A replication about cause–effect linkage benefits and managers’ strategic judgments

My instrument is framed using balanced scorecard/strategy map language because of its popularity. However, my experimental design manipulates features that I expect to generalize to most if not all SMPSs.

As discussed in Sect. 3, I manipulate causal relatedness by presenting strategic goals in goal-pairs that either come from adjacent (high causal relatedness) or non-adjacent (low causal relatedness) balanced scorecard categories. The defining nature of this manipulation is that non-adjacent goal-pairs are more distant and thus a bridging mediator cannot easily be inferred, making the linkage seems unlikely, while adjacent goal-pairs are less distant and thus a bridging mediator can easily be inferred, making the linkage seems likely.

Or, also important, this new information can increase management’s confidence in the firm’s current strategy.

The brain converts current cognitions into constructs that are encoded in memory by neurally categorizing and connecting them to other memories (e.g. Nickerson and Adams 1979; Anderson 1974; Shiffrin and Atkinson 1969; see also Otmakhova et al. 2013; Bliss and Collingridge 1993). Before examining the effect of an SPMS on memory, it is important to distinguish between working memory and long-term memory. Working memory is a temporary cache of recent cognitions that allows one to easily recall recent thoughts. Long-term memory, in contrast, is the collective body of cognitions that have been neurally categorized and connected with other memories. These categories and connections with other memories serve as cues to allow the encoded cognitions to be recalled in the future. Encoding a cognition in memory effectively means encoding it in long-term memory.

To focus their experiment on long-term memory, Cheng and Humphreys (2012) give participants a distractor task that clears their working memory before participants make information relevance judgments and strategy appropriateness judgments. Cheng and Humphreys then present participants with summaries of six fictional articles, which are construed as equivalent to articles from a reputable business journal. Participants first determine if these article summaries are relevant to the task of evaluating the firm’s strategy. Then, for articles that participants consider relevant, they determine whether the article summaries suggest the firm’s strategy is appropriate or inappropriate for the current business environment.

For my purposes, I treat causal relatedness as being effectively synonymous with cause−effect linkage strength (see Webb 2004); I use the term causal relatedness instead of the term cause−effect linkage strength because the former term has an associated literature with rich predictions to help in examining boundary conditions for Cheng and Humphreys’ (2012) results. Prior literature’s description of cause−effect linkage strength does clearly point to it also being synonymous with the subjective probability of a given cause−effect relationship. Webb (2004) says it is how much the cause goal “should lead to” the effect goal (Webb 2004, p. 941; see also King 2007 who reviews a related idea: causal ambiguity). Strong cause−effect linkages “are perceived as being plausible and informative” (p. 929) and weak cause−effect linkages are “questionable” (p. 939). Luft (2004) considers strong linkages “highly probable” and weak linkages “dubious” (p. 959).

Background knowledge refers to knowledge already encoded in memory that can be relevant to the business environment. It is not necessarily technical knowledge or knowledge that is only acquired from specialized training or education. Some background knowledge is gained from common life experiences over time, so background knowledge should be correlated with general experience.

Or, as an example from the earlier sentences about Joey, what if one knew that Joey and his neighbor are close friends who often play rough and tumble games of football after school? If so, one would likely be able to narrow down the possible bridging mediators for Sentence Pair 2 and read it as a sentence pair with high causal relatedness.

My instrument is original, but I designed it to be comparable to Cheng and Humphreys’ (2012). I create an original scenario, original strategic goals, an original distractor task, and original article summaries. Key similarities that I have held constant include the general pattern of the experiment (i.e. learn goals, complete distractor task, and then make judgments about new information), instructions about the key between-subject manipulation, and nearly identical phrasing of my dependent variable questions.

Adding these instructions to the linkages condition is consistent with similar explicit instructions provided by Cheng and Humphreys (2012).

Linking non-adjacent categories is not uncommon (see, for example, Capelo and Dias 2009; Niven 2006). To a practitioner, non-adjacent categories would not be a sure indication of low causal relatedness.

When I add an indicator variable that reflects this counterbalancing to the main hypothesis tests for my six hypotheses, it does not materially change the direction or significance of the coefficients that test those hypotheses. The indicator variable itself is not significant when added to my test of H1a, H2a, or H3a (see regressions in Table 2 Panels A and B; Table 3 Panels A and B; and Table 4 Panel A; p > 0.40 two-tailed). However, the indicator variable is significant when added to my test of H1b (see regression in Table 2 Panel C; coefficient = 0.14, p = 0.02 two-tailed), H2b (see regression in Table 3 Panel C; coefficient = 0.14, p = 0.02 two-tailed), and H3b (see regression in Table 4 Panel B; coefficient = 0.13, p = 0.03 two-tailed).

These attention checks involve no deception. I am deliberately straightforward about the correct answer participants should choose. Thus, if a participant misses an attention check, he or she is very unlikely to be paying sufficient attention to provide meaningful responses.

To test these screening protocols, I repeat my hypothesis tests (see hypothesis tests in Tables 2, 3, and 4) without these screening protocols. First, I remove IP address screening (n = 352) and find results that are almost entirely unchanged in terms of direction and significance (see Sect. 4). The only change is that my test for H1a is now significant at the 95% confidence level (linkages: p = 0.046 one-tailed). Second, I remove all screening criteria (n = 371). My H1a result is again marginally significant (linkages: p = 0.0503 one-tailed), while the other hypothesis tests are unchanged in direction and significance.

In their experimental instrument, Cheng and Humphreys (2012) explicitly describe strategy maps as portraying cause−effect relationships between strategic goals (to participants who receive a strategy map). They also give strategy map participants check questions that further emphasize how a strategy map suggests cause−effect relationships between strategic goals.

I present both questions to all participants, but I instruct them to select “N/A” for the strategy appropriateness judgment if they rate an article summary’s relevance as zero. Participants cannot advance to the next article summary if they indicate the article summary is irrelevant and select anything other than “N/A.” They also cannot advance if they select “N/A” while indicating the article summary is anything other than irrelevant. Cheng and Humphreys (2012) similarly instruct participants who rate the information as irrelevant to move onto the next question without rendering an appropriateness judgment. This is reasonable, since appropriateness judgments about information that one perceives to be irrelevant would be difficult to interpret.

For example, assume there is an irrelevant article summary, and the most correct answer is zero, meaning irrelevant. If a participant answers this item with a four, meaning highly relevant, I reverse code that participant’s response as a zero (i.e. 4 − 4 = 0). This reflects how the response is as distant from the correct answer as possible.

I test these fixed effects’ effectiveness at removing noise by repeating my hypothesis tests from Table 2 through Table 4without this set of variables included in the model. All results reported in Tables 2 through 4 remain the same in terms of direction and significance except that the coefficient that tests H2b is only marginally significant (linkages × causal relatedness: p = 0.09 one-tailed, compare with Table 3 Panel C).

I do not expect this manipulation check to be influenced by the fact that some goal-pairs have high causal relatedness and others have low causal relatedness. First, because the mixture of causal relatedness is consistent for all participants: half of the goal-pairs are high and half of the goal-pairs are low. Second, because the question explicitly asks whether the participant believes the firm expects a cause−effect relationship, not whether the participants themselves could interpret a cause−effect relationship between the goal-pairs previously presented to them.

I also record the number of business classes participant have taken . I do not include this in my analysis because I already use a more informative control variable: whether participants have a business degree.

With one exception, none of these variables differs significantly between conditions of the SPMS cause−effect linkage manipulation (p > 0.10 one-tailed in either direction). The exception is participants’ familiarity with strategy maps, which differs in a marginally significant fashion (p = 0.053, one-tailed), suggesting that participants in the linkages condition are more familiar with strategy maps (mean = 1.15) than participants in the no linkages condition (mean = 0.95). This makes sense because I describe the firm’s SPMS as a “strategy map” in the linkages condition.

Interestingly, the simple effects tests for this interaction are non-significant. I find non-significant linkages coefficients in both the low causal relatedness subsample (coefficient = −0.09, p = 0.23 two-tailed) and the high causal relatedness subsample (coefficient = 0.09, p = 0.105 one-tailed). However, this may be due to noise from erroneous strategy appropriateness judgments. That is, participants who incorrectly rate an irrelevant article summary as greater than zero relevance likely provide noisy and difficult-to-interpret judgments about the strategy appropriateness implications of that irrelevant information item. When I only consider simple effects of responses related to relevant article summaries (n = 1,152), I find a non-significant linkages coefficient in the low causal relatedness subsample (coefficient = −0.11, p = 0.25 two-tailed) and a significant linkages coefficient in the high causal relatedness subsample (coefficient = 0.15, p = 0.04 one-tailed). This latter simple effects test supports H2b as reasoned in the hypothesis development section. This noise-reduction simplification (of only considering strategy appropriateness judgments of relevant article summaries) has no effect on the direction or non-significance of my results for H1b, but it makes the three-way interaction from my later H3b results significant at the 95% confidence level (p < 0.01 one-tailed).

Choi et al. (2012, 2013) even find that participants substitute performance measures for strategic goals, interpreting these measures of progress toward strategic goals as substitutes for the strategic goals themselves.

The phrase Chesterton’s fence refers to a quotation from G. K. Chesterton where he compares societal institutions, norms, or laws to a fence that one encounters along a road—a fence that one does not know the purpose of (Chesterton 1930). Chesterton cautions against tearing down such fences without understanding why it was there in the first place: it likely serves a function significant enough to justify its construction. Or, by way of another analogy, one should hardly try renovating a home by simply removing walls he or she does not like. Some of those walls are likely to be load bearing. Likewise, practitioners might look at experimental results—which are constrained by the figurative fences of the experiment’s context—and think they should easily translate to real world results. They are unlikely to be aware of which contextual features were preconditions for the experimental results unless replications examine these preconditions directly.