Skip to main content
main-content
Top

Hint

Swipe to navigate through the articles of this issue

Published in: Journal of Business and Psychology 1/2022

Open Access 13-01-2021 | Original Paper

In the Eye of the Beholder: How Self-Other Agreements Influence Leadership Training Outcomes as Perceived by Leaders and Their Followers

Authors: Karina Nielsen, Susanne Tafvelin, Ulrica von Thiele Schwarz, Henna Hasson

Published in: Journal of Business and Psychology | Issue 1/2022

Abstract

Based on Yammarino and Atwater’s self-other agreement typology of leaders, we explored whether leaders’ and followers’ agreement influenced their ratings of leadership behaviors after training where leaders received multi-source feedback to stimulate behavior change. We used a prospective study design including 68 leaders and 237 followers from a Swedish forest industry company. Leaders underwent training to increase their transformational leadership and contingent reward styles and reduce management-by-exception passive and laissez-faire leadership. We found that self-other agreement influences followers and leaders reporting changes in leadership styles. We also found that although some leader types were perceived to improve their leadership behaviors, leaders and followers reported differential patterns in which types of leaders improved the most. Our results have important implications for how feedback should be used to support training to achieve changes in leadership styles.
Notes
The original online version of this article was revised: Professor von Thiele Schwarz's surname was misspelled Schwartz. The correct spelling is Schwarz.
A correction to this article is available online at https://​doi.​org/​10.​1007/​s10869-021-09735-6.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Introduction

Approximately 34% of leadership training programs do not achieve their intended outcomes (Avolio, Reichard, Hannah, Walumbwa, & Chan, 2009). Effective leadership training requires not only passive learning, but changes in leadership behaviors (Kirkpatrick, 1994; Nielsen, Randall, & Christensen, 2017). Leadership training often relies on leaders receiving feedback on their leadership style as a means to raise their self-awareness and promote leadership behavioral change (Slater & Coyle, 2014), and often in the form of multi-source feedback (Lacerenza, Reyes, Marlow, Joseph, & Salas, 2017). It is widely acknowledged that leaders and followers do not always agree on their leaders’ behavior (for a meta-analysis, see Lee & Carpenter, 2018). A critical issue in multi-source feedback research is the extent to which leaders and followers agree on the leader’s behaviors; this is also known as self-other agreement (SOA) (Fleenor, Smither, Atwater, Braddy, & Sturm, 2010).
The extent to which leaders and their followers agree on the leaders’ behaviors prior to training may have implications for leaders’ motivation to change their behaviors as a result of training and their followers’ acknowledgement of any attempts to change behaviors. In the present study, we aim to understand how pre-training SOA influences the extent to which leaders change their leadership styles as a result of training when 180-degree feedback (leaders’ own ratings and followers’ ratings) is integrated into training. To the best of our knowledge, there have been no studies on how SOA affects perceptions of leaders’ leadership style as a result of training. Understanding the links between feedback as input during leadership training and how different types of leaders develop in response to such feedback may help us understand differential changes in leadership styles. We focused on understanding how different SOA leader types predict changes in leadership style. We developed hypotheses as to which leader types may improve the most, as perceived either by themselves or by their followers. Such understanding may help predictions of how leaders react to training and feedback and provides valuable insights into how leadership training including feedback may be designed and what supportive interventions may need to be developed to ensure a successful training outcome.
The present study extends and contributes to existing theory and research in four ways: First, we explore the links between pre-training SOA and leaders’ and followers’ ratings of leadership styles post-training. Despite feedback being strongly related to changes in leadership behaviors, no differences between single source feedback and multi-source feedback have been identified (Lacerenza et al., 2017). We propose that the impact of multi-source feedback may depend on self-other agreement. Lee and Carpenter (2018) highlighted that previous research on SOA has been mostly cross-sectional with limited attention to how SOA may influence leadership-training outcomes. A few studies have focused on the effects of SOA feedback. Atwater and Brett (2005) found that favorable attitudes toward multi-source feedback led to higher levels of motivation after receiving feedback. Mackie (2015) found that over-estimators rated themselves lower on transformational leadership after receiving coaching. It should be noted that coaching may help leaders develop a more realistic self-image and become more self-aware whereas leadership training focuses on changing leadership behaviors. To the best of our knowledge, how the integration of SOA feedback into leadership training impacts changes in leadership behaviors has yet to be explored.
Second, the study contributes to our understanding of how SOA impacts changes in leadership style across a range of full-range leadership styles (Avolio, 2011). Full-range leadership includes both desirable, constructive behaviors, such as transformational leadership and contingent reward that have a positive impact on follower’s performance and well-being and undesirable, passive leadership styles, such as laissez-faire leadership and management-by-exception-passive (MBEP), which can have a negative impact on followers’ performance and well-being (Arnold, 2017; Bass & Riggio, 2006; Harms, Credé, Tynan, Leon, & Jeung, 2017; Hoch, Bommer, Dulebohn, & Wu, 2018; Inceoglu, Thomas, Chu, Plans, & Gerbasi, 2018; Skakon, Nielsen, Borg, & Guzman, 2010; Wang, Oh, Courtright, & Colbert, 2011). Previous leadership training has focused on whether certain leadership behaviors can be improved (Lacerenza et al., 2017); however, it is an equally interesting question whether undesirable behaviors can be reduced. Leaders exert both transformational and passive leadership at different times (Mullen, Kelloway, & Teed, 2011), and so increasing transformational leadership does not guarantee that passive leadership behaviors such as laissez faire leadership reduce. In the present study, the training also aimed at reducing passive leadership behaviors, and we therefore studied whether laissez faire leadership and MBEP behaviors can be reduced.
Third, Yammarino and Atwater (1997) suggested that leaders who either agree or disagree with their followers on their leadership style are characterized by different traits, and these traits influence how they respond to feedback. They suggested four categories of SOA leaders: over-estimators, under-estimators, in-agreement, good leaders, and in-agreement, poor leaders. We use the Yammarino and Atwater (1997) categorization and suggest that in-agreement, good leaders rate themselves high on transformational leadership and contingent reward and low on undesirable leadership styles such as laissez faire and MBEP leadership and their followers agree. In-agreement, poor leaders rate themselves low on transformational leadership and contingent reward and high on laissez faire and MBEP leadership and their followers agree. Over-estimators (of their leadership competence) rate themselves higher on transformational leadership and contingent reward and lower on laissez faire and MBEP leadership than their followers, and finally, under-estimators (of their leadership competence) rate themselves lower on transformational leadership and contingent reward and higher in laissez faire and MBEP leadership than their followers. Previous SOA studies have primarily focused on whether leaders and followers agree rather than whether they agree the leader is “good” or “poor” (Amundsen & Martinsen, 2014; Lee & Carpenter, 2018). We extend this literature to understand how positive and negative in-agreement influence changes in leadership styles post-training.
Fourth, previous studies have primarily focused on whether leadership training can lead to changes in leadership styles as rated by followers (Barling, Weber, & Kelloway, 1996; Parry & Sinha, 2005). As followers and leaders do not always agree on the leadership style pre-training, leaders and followers may also differ in their perceptions of changes in leadership style post-training (Sosik, 2001; Tekleab, Sims Jr., Yun, Tesluk, & Cox, 2008).

Hypothesis Development

The context triggers behaviors (Tett & Guterman, 2000). Feedback at the beginning of a training program allows for meta-cognitive activities such as reflecting on which changes in leadership behaviors are required and which elements of training may be particularly useful for leaders to alter their behaviors (Ford, Smith, Weissbein, Gully, & Salas, 1998).
Feedback helps leaders gain insights into their current leadership styles and identify any discrepancies between actual and intended behaviors (Maurer, 2002).
We base our study on Yammarino and Atwater’s (1997) model of SOA, which proposes that underlying traits of over-estimators; under-estimators; in-agreement, good leaders; and in-agreement, poor leaders influence how they react to feedback. Since the seminal paper of Yammarino and Atwater (1997), a body of research has explored the links between personality and SOA (e.g., Bergner, Davda, Culpin, & Rybnicek, 2016) and how SOA influences organizational outcomes (e.g., Amundsen & Martinsen, 2014). We build our hypotheses not only on the predictions of Yammarino and Atwater (1997) but also on this more recent research.

Leaders’ Self-ratings of Leadership Behaviors Post-training

In-agreement, good leaders are likely to respond positively to feedback as they recognize and accept followers’ ratings (Lee & Carpenter, 2018; Yammarino & Atwater, 1997). Leaders who are in agreement with their followers about their leadership behaviors are believed to be able to identify their own strengths and weaknesses and set appropriate goals for themselves as to how to improve their behaviors; they have high levels of self-awareness (Fleenor et al., 2010; Lee & Carpenter, 2018). In-agreement, good leaders may feel encouraged to increase their transformational leadership and contingent reward and reduce laissez faire and MBEP leadership post-training because they receive positive feedback during training. Positive feedback is more accurately recalled than negative feedback, acts as a reinforcer of behaviors, and encourages recipients to set specific goals (Ilgen, Fisher, & Taylor, 1979). As these leaders possess high levels of self-efficacy (Yammarino & Atwater, 1997), they are likely to take on board any tools and methods to change their behaviors learned during training, and they may set realistic goals as to what behaviors need to change and how. In-agreement, good leaders are intuitive and may better understand what changes are needed (Bergner et al., 2016). Atwater and Brett (2005) found that self-efficacious leaders were more likely to engage in follow-up activities after receiving 360-degree feedback. High internal locus of control (Fleenor et al., 2010; Whetten & Cameron, 2007) means these leaders feel in control over their behaviors toward followers. Individuals engage in behaviors that reinforce their positive image (Korman, 1976). As in-agreement, good leaders receive positive feedback, they may continually seek feedback from followers, and as a result, they are likely to rate themselves higher on transformational leadership and contingent reward and lower on laissez faire and MBEP leadership post-training because training provides them with input on how to change their behaviors.
Over-estimators may perceive they successfully change their leadership styles. Over-estimators show little concern for others’ perceptions (Moshavi, Brown, & Dodd, 2003) and may be less inclined to seek consultation from followers on what might be desirable leadership behaviors (Berson & Sosik, 2007). Over-estimators are high in self-monitoring and rely little on others as a source of feedback (Miller & Cardy, 2000). Over-estimators are believed to have an exaggerated sense of self-grandor and independence, and thus, they may be less likely to take followers’ feedback on board (Berson & Sosik, 2007). Their grandiose self-perception is not likely to be corrected as they will ignore the feedback given during training.
In a field experiment, Atwater, Waldman, Atwater, and Cartier (2000) found that over-estimators reduced their commitment to followers, and Brett and Atwater (2001) found that such leaders reacted to negative feedback with anger. When leaders do not believe their followers have a realistic view of their behaviors, critical evaluations are less likely to influence their behaviors (Bernardin, Dahmus, & Redmon, 1993). Over-estimators may perceive they are successful in increasing their transformational and contingent reward leadership behaviors and reducing their laissez faire and MBEP leadership behaviors when given advice on how to so, disregarding the feedback of followers. Over-estimators possess narcissistic traits (Fleenor et al., 2010) and in an experiment, Robins and John (1997) found that narcissists only rated themselves even higher after viewing their performance on video. Over-estimators’ self-grandeur prevent them from seeking consistency with followers (Korman, 1976), but they may feel that training has made them even better leaders (Robins & John, 1997). Over-estimators are less likely to report changes than in-agreement, good leaders as they see less need for development (Yammarino & Atwater, 1997).
Under-estimators may perceive they benefit from training. Bratton, Dodd, and Brown (2011) found that under-estimators score higher on adaptability, and thus, they may be more willing to learn from training. The Pygmalion effect (Rosenthal & Jacobson, 1968) suggests that followers’ positive expectations of the leader may have a positive influence. During training, leaders are given the tools and methods to change leadership behaviors, and under-estimators may be motivated to become a better leader. Amundsen and Martinsen (2014) suggested that feedback builds these leaders’ self-confidence and helps them understand their position better. Training may build their confidence to try these behaviors at work (Tekleab et al., 2008). Under-estimators may thus increase their ratings post-training as they come more self-aware of how to improve their transformational leadership and contingent reward behaviors and gain confidence they can reduce their laissez faire and MBEP behaviors after having been presented with the methods and tools to do so. As a result of the unexpected positive feedback, they may increasingly seek feedback from followers, which may help them improve their behaviors. Under-estimators’ self-beliefs about their ability to be good leaders may increase; however, they are likely still to suffer from self-doubts (Yammarino & Atwater, 1997) and lack the confidence to fully engage with training, which may limit development in their self-ratings post-training.
Finally, in-agreement, poor leaders may not perceive they benefit from training. These leaders’ self-confidence is low, and they rate themselves negatively (Tekleab et al., 2008). The Golem effect (Babad, Inbar, & Rosenthal, 1982) would suggest that followers’ poor ratings only confirm the leaders’ own negative self-view. Underachievers possess poor self-concepts and readily accept negative feedback (Ilgen et al., 1979). Negative feedback from followers that they are poor leaders may take an additional toll on leaders’ self-confidence, and they continue to view themselves as poor leaders (Fleenor et al., 2010; Tekleab et al., 2008). In-agreement, poor leaders may suffer from learned helplessness (Amundsen & Martinsen, 2014). Although they do recognize themselves as weak leaders, they are unable to change due to low self-efficacy and poor self-confidence (Fleenor et al., 2010; Tekleab et al., 2008).
Despite training, in-agreement, poor leaders may be less likely attempt to change their behaviors as they withdraw from their followers (Amundsen & Martinsen, 2014). Exploring performance as the outcome, Smither et al. (1995) found that leaders did not improve their performance after receiving feedback from their followers when they agreed with this feedback, even when this feedback was poor. We therefore hypothesize the following:
Hypothesis 1:
In-agreement, good leaders will report the greatest increases in transformational leadership post-training; over-estimators will report the second highest increases; under-estimators will report the third highest increases; and in-agreement, poor leaders will report the least increases in transformational leadership.
 
Hypothesis 2:
In-agreement, good leaders will report the greatest increases in contingent reward post-training; over-estimators will report the second highest increases; under-estimators will report the third highest increases; and in-agreement, poor leaders will report the least increases in contingent reward.
 
Hypothesis 3:
The greatest reductions in management-by exception-passive leadership post-training will be reported by in-agreement, good leaders; the second greatest reductions for over-estimators; third greatest reductions for under-estimators; and in-agreement, poor leaders will report the least reductions in MBEP leadership.
 
Hypothesis 4:
The greatest reductions in laissez faire leadership post-training will be reported by in-agreement, good leaders; the second greatest reductions for over-estimators; third greatest reductions for under-estimators; and in-agreement, poor leaders will report the least reductions in laissez faire leadership.
 

Followers’ Ratings of Leadership Behaviors Post-training

According to social cognition theory, people develop schemas to mentally organize and store information (Fiske, 1992). Followers observe their leader’s style and store this information in cognitive schemas that they use to predict and judge the leadership behavior of the leader (Ilgen & Feldman, 1983). In other words, followers’ ratings of their leaders pre-training predict how they view their leaders post-training. If followers have positive perceptions of their leaders pre-training, they are more likely to react favorably to any attempts to change leadership behaviors (Fleenor et al., 2010). For over-estimators and in-agreement, poor leaders, followers’ negative schemas will only be revised if leaders make major changes to their behaviors (Fiske, 1992; Ilgen & Feldman, 1983).
We propose followers of in-agreement, good leaders will report increased transformational leadership and contingent reward and reduced laissez faire and MBEP leadership. In-agreement, good leaders typically have a good relationship with and are valued by their followers (Berson & Sosik, 2007) and followers of in-agreement, good leaders feel supported by their leader Sosik and Godshalk (2000). As followers have positive schemas of their in-agreement, good leader, they are likely to react positively to their leader’s attempts to increase transformational leadership and contingent reward and reduce their laissez faire leadership and MBEP. As followers know leaders have been on training, they may additionally appreciate their leaders’ commitment to improve, and this could enhance their positive leadership schemas. They will thus agree with their leaders that the in-agreement, good leaders have improved.
Under-estimators may be perceived as benefitting from training by their followers. As under-estimators are receptive to feedback from followers (Amundsen & Martinsen, 2014), they react to feedback and strive to increase their transformational leadership and contingent reward behaviors and reduce their laissez faire and MBEP leadership behaviors, especially as such leaders tend not to become complacent or overconfident (Amundsen & Martinsen, 2014). Moshavi et al. (2003) found that followers of under-estimators were satisfied with their leaders.
Under-estimators are humble to the opinions of followers (Aarons, Ehrhart, Farahnak, Sklar, & Horowitz, 2017; Amundsen & Martinsen, 2014). Humility is related to agreeableness, which has been found to be related to transformational leadership and contingent reward (Bono & Judge, 2004). Humility makes under-estimators likeable (Amundsen & Martinsen, 2014). McKee, Lee, Atwater, and Antonakis (2018) found that followers of agreeable leaders over-rated their leader on instrumental leadership. Under-estimators’ willingness for self-improvement and desire to meet followers’ expectations (Yammarino & Atwater, 1997) may invoke followers’ positive appraisals post-training. As a result of followers’ positive schemas of their leader, they welcome any attempts to increase transformational leadership and contingent reward and reduce laissez faire leadership and MBEP, and they thus develop even more positive cognitive schemas of their leader post-training. Bratton et al. (2011) found that followers of under-estimators rated their leader higher on transformational leadership than over-estimators and in-agreement, poor leaders. Under-estimators are less likely to be rated higher than in-agreement, good leaders as they lack confidence in trying out new challenging behaviors.
In-agreement, poor leaders who score themselves low on transformational leadership and contingent reward and high on laissez faire and MBEP leadership and their followers agree may not be perceived to change much by their followers. Followers have negative schemas of these leaders and may not appreciate any attempts to change leadership styles. Although followers may be aware that their leaders have been on training, their negative schemas of their leader mean they anticipate that leaders are unable to benefit from training. Unless very drastic changes in leadership styles can be observed, followers are unlikely to change their ratings of their leader. Such dramatic changes are unlikely to happen as these leaders have poor self-efficacy (Amundsen & Martinsen, 2014). Followers will thus agree with these leaders that little progress have been made.
Followers are unlikely to report any changes to the leadership styles of the over-estimator leader. Bashshur, Hernández, and González-Romá (2011) suggested when leaders’ ratings are higher than those of their followers, the leaders enact passive leadership as they fail to understand the needs of their followers. It has been suggested that such leaders may be hostile towards their followers (Yammarino & Atwater, 1997). Sosik and Godshalk (2000) found that followers whose leaders overestimated themselves on transformational leadership reported low levels of support. If followers have in the past perceived their leader as arrogant and unapproachable (Fleenor et al., 2010), they are likely to continue to view their leader in this way. Atwater and Yammarino (1992) found a negative correlation between over-estimators and follower ratings of their transformational leadership behavior.
Followers see over-estimators as uncaring and self-centered (Sosik, 2001) and less emotionally competent (Wang, Wilhite, & Martino, 2016). Followers of estimators are the least satisfied with their leaders (Moshavi et al., 2003). Even if these leaders do make attempts to increase their transformational leadership and contingent reward and reduce their laissez faire and MBEP leadership, followers are unlikely to react positively to these attempts as their schemas categorize their leader as uncaring and self-centered, and major changes in behaviors may be required to change these schemas. They are thus likely to disagree with their over-estimator leaders that they have changed their behaviors. We thus developed the following hypotheses:
Hypothesis 5:
Followers will report the greatest increases in transformational leadership post-training if their leader is an under-estimator; the second highest increases if the leader is an in-agreement, good leader; third highest if the leader is an in-agreement poor leaders; and the least increases in transformational leadership will be reported by followers whose leader is an over-estimator.
 
Hypothesis 6:
Followers will report the greatest increases in contingent reward post-training if their leader is an under-estimator; the second highest increases if the leader is an in-agreement, good leader; third highest if the leader is an in-agreement, poor leader; and the least increases in contingent reward will be reported by followers whose leader is an over-estimator.
 
Hypothesis 7:
Followers will report the greatest reductions in management-by exception passive leadership if their leader is an under-estimator; the second greatest reductions will be reported by followers of in-agreement, good leaders, although the third greatest reductions will be observed by followers of in-agreement, poor leaders. Followers of over-estimators will report the least reductions in MBEP.
 
Hypothesis 8:
Followers will report the greatest reductions in laissez faire leadership if their leader is an under-estimator; the second greatest reductions will be reported by followers of in-agreement, good leaders, while the third greatest reductions will be observed by followers of in-agreement, poor leaders. Followers of over-estimators will report the least reductions in laissez faire leadership.
 

Methods

Setting, Sample, and Procedure

The present study took place in the forest industry (Tafvelin, Hasson, Holmström, & von Thiele Schwarz, 2019) using a prospective study design. All leaders in the organization (N = 101) participated. Leaders were asked to invite five followers to provide ratings of their leadership behaviors, resulting in an average of 3.5 followers (range 1 to 5) responding to the questionnaires. Leaders were asked to select both followers they felt close to and had regular contact with and followers with whom they had less frequent contact. Leaders and their followers were sent an e-mail with a personal link to an online questionnaire together with information about the training and its evaluation. The study was approved by the local ethics review board.
For the purpose of the present study, we only included data from respondents who consented for their data to be used in research. In total, 68 leaders and their 237 corresponding followers, yielding a response rate of 77 % for the leaders, and 82% for followers (237 out of 290 invited followers). The majority of participants were men (leaders 74%; followers 85%), and the average age was 46 years for both leaders and followers (leaders SD 8.70, followers SD 8.87). Leaders had an average tenure of 5 years (SD 5.52) and followers an average tenure of 23 years (SD 9.9 years).

The Training

The leadership training was initiated by the organization and formed part of an overall cultural change initiative in the company. Management wanted to foster an organizational culture focused on learning where followers took responsibility for the success of the company rather than merely operating their machines. Management saw full-range leadership as an important component in this change. The training was delivered by organizational psychologists from an occupational health company. It was on the recommendation of the occupational health company that the organization invited the research team to evaluate the training. Leaders received 20 days of training over a period of 16 months. Training was extensive as it was developed to support the wider organizational cultural change.
The training was conducted in cross-departmental groups with 20 leaders in each group. The exception was a 30-min individual session where each leader was provided with an individual feedback report on their leadership styles and offered the opportunity to discuss it with one of the consultants. The report included graphs showing the means and variation of all rating sources and norms for the scales. Leaders were not provided information on which group they belonged to, but information were provided on how they compared with the MLQ norms (3 or higher on transformational leadership, between 2 and 3 on contingent reward, and between 0 and 1 on MBEP and laissez faire) to get an idea on what they might want to develop. This individual session was offered after the first training session where a theoretical explanation of the full-range leadership model was presented. All leaders participated in the individual feedback session. The feedback session was provided to give room for individual reflection and questions and aimed to build motivation for action plans developed during leadership training.
Multiple training methodologies were employed in line with recommendations for effective leadership training (Cacioppe, 1998; Lacerenza et al., 2017). The training was based on full-range leadership theory (Avolio, 2011) combined with a functional perspective emphasizing how leader behaviors influence follower motivation and behaviors, based on operant psychology (Skinner, 1963). This approach was chosen to align with the organization’s leadership strategy. The training started with a theoretical block (14 days), followed by a practical block for the remaining time (6 days). The theoretical block included didactic sessions focusing on full-range leadership, organizational change, and follower motivational processes, thus relating leadership to follower motivation, particularly during organizational change. The practical block focused on leaders developing action plans and following up on their implementation. Leaders were expected to apply learning on leadership, organizational change, and motivational processes working through a practical case together with their followers. Action plans focused on topics such as facilitating collaboration and improving information exchange.

Measures

Leaders and followers completed the questionnaire 1 month before training and immediately after the leadership training was completed.
Leadership behaviors were measured using the Multifactor Leadership Questionnaire (MLQ-Form 5X) (Avolio & Bass, 2004). Transformational leadership was assessed with 20 items covering the four subcomponents of transformational leadership: idealized influence, inspirational motivation, intellectual stimulation, and individualized consideration. In line with the original MLQ scale, the contingent reward, MBEP, and laissez-faire leadership dimensions were each measured with 4-item scales. For all scales, ratings were made an a 5-point response scale ranging from 1 (never) to 5 (frequently, if not always), where leaders rated how often they engaged in each behavior, and followers rated how often the manager they were rating engaged in each behavior. The internal consistency of the MLQ subscales at baseline was as follows: transformational leadership 0.91 (followers) and 0.84 (leaders), contingent reward 0.80 (followers) and 0.52 (leaders), MBEP 0.58 (followers) and 0.62 (leaders), and laissez-faire 0.78 (followers) and 0.59 (leaders).

Analyses

To test how SOA influences leaders’ and followers’ perceptions of changes in leadership behaviors post-training, we used polynomial regression with response surface analysis (Edwards, 1994; Humberg, Nestler, & Back, 2019). This analysis enables us to examine the combined impact of two variables on a third, but at the same time retaining information about the differences between the variables. It is the recommended analysis when SOA is the independent variable (Edwards, 2002) as it keeps measures of self- and other ratings separate, at the same time as also incorporating higher-order terms such as squared and interaction terms which enable tests of more elaborated effects (Humberg et al., 2019). We followed the three-step procedure outlined by Shanock, Baran, Gentry, Pattison, and Heggestad (2010), also recommended by Gibson, Cooper, and Conger (2009).
First, agreement and disagreement between leaders and followers was investigated to confirm whether the level of disagreement was sufficient to warrant further analysis. At least 10% of the leaders need to disagree with their followers, with disagreement defined as at least 0.5 SD of the standardized mean score on the two predictors, to make further analyses meaningful (Fleenor & Prince, 1997). To classify the leaders into the four categories, we therefore standardized the score for self and followers, and a leader with a standardized score on the self-rating half a standard deviation above their followers’ scores was categorized as an over-estimator. A leader with a standardized self-rated score half a standard deviation below their followers’ scores was categorized as an under-estimator. Leaders within these limits were categorized as in agreement with followers (Fleenor, McCauley, & Brutus, 1996). Leaders who were in agreement and rated by their team above the mean score were classified as in-agreement, good, whereas leaders who were in agreement with their followers, but rated by the team below the sample mean was classified as in-agreement, poor. This classification, based on theory, is only used for descriptive purposes and to aid interpretation of the response surface analysis, and not used in the polynomial regressions where continuous variables are used.
Second, polynomial regression analyses were conducted, one for each of the four leadership styles for both follower and leader ratings. These analyses were performed on scale-centered variables to aid interpretation of the findings (Edwards, 1994). Time 2 ratings of leadership behavior were regressed on leaders’ ratings, followers’ ratings, the cross product of leaders’ and followers’ ratings, and the square of leaders’ and followers’ ratings of the leaders’ leadership behaviors at Time 1. If the predictors explain significant variance in the outcome variable (i.e., R2 of the polynomial regression is significant), further analyses are justified, which includes calculating the four surface test values: a1, a2, a3, and a4, based on unstandardized regression coefficients (Atwater, Ostroff, Yammarino, & Fleenor, 1998). Tables 4 and 5 show the equations for the surface tests and the results.
In the third step, the surface test values were plotted in graphs. The four surface test values represent the slopes and curvature of two lines. The first line, the “line of perfect agreement,” runs diagonally from the nearest to the farthest corners of the graph. a1 is the slope along the “line of perfect agreement” and represents how agreement between the predictors relates to the outcome. a2 is the curvature along the line for perfect agreement and shows whether this relationship (between agreement and outcome) is linear or non-linear, that is, if the outcomes differ depending on whether the ratings are high and in agreement or low and in agreement. The second line, called the “line of incongruence,” runs diagonally from the left to the right corner. The slope along the line of incongruence is reflected by a3 and the curvature by a4. Similarly, as for the line of perfect agreement, the curvature shows how disagreement between predictors relates to the outcome and subsequently the slope if the direction of disagreement matters.
To justify aggregation of the follower data to the team level, intraclass correlation coefficients (ICC(1)s) and within-group agreement (rWG(j)) statistics were calculated. As presented in Table 1, the ICCs were all positive, and all except one were significant. The mean rWG(j)s were above 0.80 for all scales. Overall, the analyses support the aggregation of follower ratings.
Table 1
Subscale intraclass correlation coefficients (ICC) and within-group agreement
Subscale
Number of items
ICC
Mean rWG (j)
Median rWG (j)
Range rWG (j)
Transformational leadership
20
    
Time 1
 
0.27*
0.82
0.87
0.80
Time 2
 
0.24*
0.83
0.88
0.93
Contingent reward
4
    
Time 1
 
0.22*
0.82
0.86
0.52
Time 2
 
0.17*
0.86
0.88
0.72
Management by exception passive
4
    
Time 1
 
0.05
0.82
0.90
0.64
Time 2
 
0.14*
0.85
0.90
0.92
Laissez-faire leadership
4
    
Time 1
 
0.20*
0.84
0.90
0.73
Time 2
 
0.29*
0.88
0.93
0.76
K = 68 leaders
*p < .05

Results

Descriptive statistics and correlations between study variables are presented in Tables 2 and 3. Table 3 shows the correlations between leader and follower ratings of the four leadership behaviors at Time 1 are all non-significant, indicating that variation exists between the ratings of leader and followers and that perceptual distance analyses are warranted.
Table 2
Means, standard deviations, and correlations among variables
Scale
Mf
SDf
Ml
SDl
1.
2.
3.
4.
5.
6.
7.
8.
1. TL T1
2.24
0.50
2.63
0.34
 
0.58**
0.14
− 0.16
0.68**
0.40**
0.15
− 0.16
2. CR T1
2.16
0.57
2.50
0.44
0.84**
 
0.19
− 0.11
0.46**
49**
0.15
− 0.06
3. MBEP T1
1.00
0.41
1.10
0.55
− 0.08
− 0.08
 
0.25*
0.10
0.12
0.49**
0.13
4. LF T1
0.76
0.54
0.81
0.45
− 0.52**
− 0.51**
0.49**
 
− 0.03
0.13
0.05
0.39**
5. TL T2
2.43
0.45
2.75
0.36
0.76**
0.64**
− 0.11
− 0.41**
 
0.70**
0.17
− 0.26
6. CR T2
2.40
0.53
2.64
0.48
0.69**
0.66**
− 0.14
− 0.44**
0.82**
 
0.25*
− 0.20
7. MBEP T2
0.99
0.42
1.08
0.55
− .04
− .04
0.61**
0.42**
− 0.05
− 0.05
 
0.14
8. LF T2
0.77
0.54
0.64
0.39
− .47**
− 0.49
0.27*
0.71**
− 0.54**
− 0.50**
0.40**
 
TL transformational leadership, CR contingent reward, MBEP management by exception passive, LF laissez-faire leadership, Mf mean followers, SDf standard deviation followers, Ml mean leaders, SDl standard deviation leaders. N = 75 leaders and their followers. Follower ratings below the diagonal, leader ratings above the diagonal
*p < .05; **p <.01
Table 3
Means, standard deviations, and correlations among leader and team rated variables
Scale
1.
2.
3.
4.
5.
6.
7.
8.
1. TL leader
 
0.09
0.69*
0.11
0.18
0.01
− 0.26*
− 0.19
2. TL team
− 0.01
 
0.09
0.82**
0.08
− 0.05
− 0.21
− 0.54**
3. CR leader
0.58**
− 0.08
 
0.14
0.25*
0.04
− 0.20
− 0.08
4. CR team
− 0.03
0.84**
0.08
 
− 0.11
− 0.05
− 0.13
− 0.50**
5. MBEP leader
0.14
− 0.10
0.19
− 0.06
 
0.25*
0.14
0.03
6. MBEP team
0.01
− 0.08
0.04
0.08
0.18
 
0.14
0.40**
7. LF leader
− 0.16
0.04
− 0.11
0.10
0.25*
− 0.05
 
0.19
8. LF team
− 0.28*
− 0.52**
− 0.22
− 0.51**
0.08
0.49**
0.12
 
TL transformational leadership, CR contingent reward, MBEP management by exception passive, LF laissez-faire leadership, N = 75 leaders and their teams. Time 1 ratings below the diagonal, time 2 ratings above the diagonal
*p < .05; ** p < .01
In line with the procedure outlined by Shanock et al. (2010), we first analyzed the level of agreement between leaders’ and followers’ perceptions of leadership before training. For transformational leadership, 19% of leaders fell in the in-agreement, good leader category, whereas 11% fell were in-agreement, poor leaders; 33% were over-estimators; and 37% were under-estimators. For contingent reward, 17% were in-agreement, good leaders; 11% were in-agreement, poor leaders; 37% were over-estimators; and 35% were under-estimators. For MBEP leadership, 16% were in-agreement, good leaders; 15% were in-agreement, poor leaders; 32% were over-estimators; and 37 % were under-estimators. Finally, for laissez faire leadership, 20% were in-agreement, good leaders; 9% were in-agreement, poor leaders; 36% were over-estimators; and 36% were under-estimators. Thus, the discrepancies in leader and follower ratings were larger than 10% (Fleenor & Prince, 1997), warranting polynomial regression analyses.
We analyzed the impact of SOA pre-training on leaders’ self-ratings of leadership styles after training. Four polynomial regressions were performed, one for each leadership style (see Table 3). Self-other agreement on transformational leadership before training explained significant variance in leaders’ self-ratings of transformational leadership after training. The same was true for all other leadership styles. We therefore calculated surface test values, a1–a4, for all four leadership styles at Time 2, which are presented in Table 4. The surface test values were then used to graph and interpret the results from the polynomial regressions.
Table 4
Polynomial regression analyses and surface values for leader ratings of leadership
 
Subscales of MLQ time 2
Subscale of MLQ time 1
TL
CR
MBEP
LF
Constant
2.14**
2.34**
1.50**
1.18**
Leader-rated (b2)
1.18**
0.92**
0.36
0.28
Team-rated (b1)
0.09
0.00
− 0.35
0.14
Leader-rated squared (b5)
− 0.39
− 0.45
0.03
− 0.03
Leader-rated * team-rated(b4)
− 0.06
− 0.01
− 0.14
0.02
Team-rated squared (b3)
0.13
0.15
− 0.31
− 0.01
R2
0.58**
0.33**
0.33**
0.18*
Surface tests
    
a1 = (b1 + b2)
10.27**
0.92**
0.01
0.42
a2 = (b3 + b4 + b5)
− 0.32
− 0.31
− 0.42
− 0.02
a3 = (b1 − b2)
10.09**
0.92**
0.71
10.15
a4 = (b3 − b4 + b5)
− 0.21
− 0.29
− 0.15
− 0.06
TL transformational leadership, CR contingent reward, MBEP management by exception passive, LF Laissez-faire leadership
*p < .05; **p < .01
Hypothesis 1 proposed that the highest increases in transformational leadership post-training would be reported by in-agreement, good leaders; second highest for over-estimators; third highest for under-estimators and the lowest for in-agreement; poor leaders as rated by leaders themselves. Testing Hypothesis 1, the slope along the line of perfect agreement was positive and significant (a1 = 1.27, t = 5.41, p < .001) suggesting that in-agreement, good leaders reported that they increased more than in-agreement, poor leaders. The slope along the line of incongruence was positive and significant (a3 = 1.09, t = 5.25, p < .001), indicating that over-estimators perceived that they increased their transformational leadership more than under-estimators post-training. As seen in Fig. 1, we were unable to differentiate between in-agreement, good leaders and over-estimators, or between in-agreement, poor leaders and under-estimators. Thus, the pattern of results for these leadership styles partially supports Hypothesis 1: In-agreement, good leaders rated that they increased their transformational leadership more than in-agreement, poor leaders and over-estimators reported they increased these styles more than under-estimators; however, in-agreement, good leaders and over-estimators increased these styles equally, as did in-agreement, poor leaders and under-estimators.
A similar pattern was found for Hypothesis 2 testing self-rated increases in contingent reward. The slope along the line of perfect agreement was positive and significant (a1 = 0.92, t = 4.31, p ≤ .001), suggesting that in-agreement, good leaders reported that they increased their contingent reward behaviors more than in-agreement, poor leaders. The slope along the line of incongruence was positive and significant (a3 = 1.09, t = 5.25, p < .001), indicating that over-estimators perceived that they increased more than under-estimators post-training. Again, as seen in Fig. 2, we were unable to differentiate between in-agreement, good leaders and over-estimators, or between in-agreement, poor leaders and under-estimators. Thus, Hypothesis 2 was partially supported.
Testing Hypothesis 3, for self-rated development of MBEP, the slope along the line of perfect agreement was non-significant (a1 = 0.01, t = 0.01, p = .991), and we thereby failed to detect any differences between in-agreement, good and in-agreement, poor leaders. In addition, the slope along the line of incongruence was non-significant (a3 = 0.71, t = 1.36, p = .177) suggesting no difference between over-estimators and under-estimators of passive leadership post-training. Therefore, no support was established for Hypothesis 3.
The results for Hypothesis 4 and self-rated laissez faire leadership were similar to Hypothesis 3. The slopes along both the line of perfect agreement (a1 = 0.42, t = 0.65, p = .516), and the line of incongruence (a3 = 0.15, t = 0.34, p = .736) was non-significant, leading to the rejection of Hypothesis 4.
To test Hypotheses 5–8, we analyzed the impact of SOA before training on the followers’ ratings of leadership post-training. Again, four polynomial regressions were performed, one for each leadership style (see Table 4). The predictors explained significant variance post-training. We therefore calculated surface test values, a1a4, for all four leadership behaviors post-training, which are presented in Table 5.
Table 5
Polynomial regression analyses and surface values for follower ratings of leadership
 
Subscales of MLQ Time 2
Subscale of MLQ Time 1
TL
CR
MBEP
LF
Constant
2.28**
2.30**
1.88**
1.05**
Leader-rated (b2)
0.01
0.22
0.05
− 0.74
Team-rated (b1)
0.74**
0.63**
1.02**
0.26**
Leader-rated squared (b5)
0.04
− 0.17
− 0.06
− 0.14
Leader-rated *team-rated (b4)
0.09
0.01
− 0.03
− 0.37
Team-rated squared (b3)
− 0.05
− 0.05
0.24
− 0.04
R2
0.58**
0.45**
0.44**
0.55**
Surface tests
    
a1 = (b1 + b2)
0.75**
0.84**
10.07*
− 48
a2 = (b3 + b4 + b5)
− 0.11
− 0.20
0.15
− 0.54
a3 = (b1 − b2)
− 0.73*
− 0.41
− 0.98**
− 1.00*
a4 = (b3 − b4 + b5)
0.08
− 0.22
− 0.20
0.02
TL transformational leadership, CR contingent reward, MBEP management by exception passive, LF Laissez-faire leadership
*p < .05; **p < .01
Hypothesis 5 proposed that followers would report the greatest increases in transformational leadership for under-estimators; second highest for good, in-agreement leaders; third highest for in-agreement, poor leaders; and lowest increases for over-estimators. Testing Hypothesis 5, the slope along the line of perfect agreement was positive and significant (a1 = 0.75, t = 2.66, p = .001) suggesting that in-agreement, good leaders increased their transformational leadership more than in-agreement, poor leaders. The slope along the line of incongruence was negative and significant (a3 = − 0.73, t = − 3.01, p = .004), indicating that under-estimators increased more than over-estimators post-training. As seen in Fig. 3, under-estimators increased their transformational leadership more than in-agreement, good leaders. Contrary to expectations, over-estimators were perceived to increase more than in-agreement, poor leaders by their followers. The pattern of results for transformational leadership only partially supports Hypothesis 5.
The results for Hypothesis 6, regarding followers’ ratings of improvements in contingent reward, revealed that the slope along the line of perfect agreement was positive and significant reward (a1 = 0.84, t = 4.02, p < .001), suggesting that in-agreement, good leaders increased their contingent reward leadership more than in-agreement, poor leaders. Contrary to the findings of transformational leadership, the slope along the line of incongruence was non-significant (a3 = − 0.41, t = − 1.95, p = .054), and we therefore failed to detect any significant differences between over-estimators and under-estimators. As seen in Fig. 4, contingent reward post-training is highest for the in-agreement, good leaders, followed by under-estimators, over-estimators, and lowest for in-agreement, poor leaders as rated by followers. Hypothesis 6 was therefore partially supported.
Testing Hypothesis 7 and reductions in MBEP, the slope along the line of agreement was positive and significant (a1 = 1.07, t = 2.13, p = .037), suggesting that in-agreement, good leaders (i.e., low levels of MBEP) reduced their MBEP more than in-agreement, poor leaders (i.e., high levels of MBEP) as rated by their followers. The line of incongruence was negative and significant (a3 = − 0.98, t = − 2.72, p = .008) indicating that under-estimators reduced their MBEP leadership behaviors more than over-estimators according to followers. As seen in Fig. 5, the lowest ratings of MBEP after training are found for in-agreement, good leaders followed by under-estimators. In addition, it was not possible to detect any differences between over-estimators and in-agreement, poor leaders in post-training MBEP leadership behaviors. Our pattern of findings therefore partially supports Hypothesis 7.
Finally, testing Hypothesis 8 and reductions in laissez faire leadership, the slope along the line of perfect agreement was non-significant (a1 = − 0.48, t = − 0.75, p = .455), suggesting no difference between in-agreement, good or in-agreement, poor leaders. The line of incongruence was negative and significant (a3 = − 1.00, t = − 2.29, p = .025), indicating that under-estimators reduced their passive leadership behaviors more than over-estimators. As presented in Fig. 6, the highest ratings of laissez-faire leadership after training were found for over-estimators, whereas lower ratings were found for under-estimators, and in-agreement, good and in-agreement, poor leaders. Thus, Hypothesis 8 was partially supported.

Discussion

In the present paper, we aimed to extend current theory and research on how SOA may influence the effects of leadership training when multi-source feedback is integrated into training. We explored these effects in a leadership training program aimed at increasing transformational leadership and contingent reward and reducing management-by-exception passive and laissez-faire leadership. Based on the seminal paper of Yammarino and Atwater (1997) and more recent empirical research (for reviews, see Fleenor et al., 2010; Lee & Carpenter, 2018) suggesting that over-estimators; under-estimators; in-agreement, good leaders; and in-agreement, poor leaders possess different traits (Fleenor et al., 2010; Lee & Carpenter, 2018; Yammarino & Atwater, 1997), we hypothesized that there are differences in the extent to which leaders change their leadership behaviors post-training as rated by leaders themselves and their followers depending on their SOA type.
Our first to fourth hypotheses stated that leaders themselves would rate themselves differently post-training based on their SOA. We suggested that in-agreement, good leaders would rate the highest improvements, followed by over-estimators, under-estimators, and in-agreement, poor leaders. These results lend overall support for the Yammarino and Atwater (1997) assumption that in-agreement, good leaders improve the most—according to themselves, however, do not support that over-estimators have the worst outcomes. We found support that in-agreement, good leaders reported the greatest increases in transformational leadership and contingent reward compared with in-agreement, poor leaders. As suggested by previous research (Lee & Carpenter, 2018), these leaders may feel comfortable with receiving feedback and may actively use this feedback to develop action plans during training that help them develop their transformational and contingent reward leadership.
The Golem effect (Babad et al., 1982) seemed to be at play when considering leaders’ own ratings. When leaders and followers both rate the leader negatively, leaders did not rate improvements in their leadership behaviors. Receiving feedback that followers agree with oneself did not induce the same change when leaders and followers agreed that the leader was poor as when leaders and followers agreed the leader was good. One possible explanation may be that in-agreement, poor leaders distrust their ability to change. Receiving training in how to change behaviors may not necessarily increase in-agreement, poor leaders’ confidence that they are capable of change, suggesting that these leaders do not benefit much from receiving feedback during leadership training. Other explanations may be that these leaders are not motivated to change (Smither et al., 1995), or they simply do not have the ability to change despite training (Yammarino & Atwater, 1997). The prediction by Yammarino and Atwater (1997) concerning the negative outcomes of in-agreement, poor leaders was confirmed.
We found no support leaders who reported lower levels of laissez faire, and MBEP leadership rated themselves differently post-training. It is possible that leaders do not perceive they enact such undesirable behaviors or that training did not focus sufficiently on how to reduce these behaviors. The leaders in the organization in the present study were mostly externally recruited and were university graduates rather than having been promoted through the ranks. This may mean that they had a greater awareness of the importance of active leadership prior to training.
Our fifth to eighth hypotheses suggested that (a) followers would perceive that transformational leadership and contingent reward had increased the most and (b) laissez faire and MBEP leadership reduced the most for under-estimators followed by in-agreement, good leaders and then over-estimators, and finally, in-agreement, poor leaders would be perceived to improve the least by the followers. We found that with regard to transformational leadership, followers rated under-estimators to improve their behaviors to a higher extent than in-agreement, good leaders (Hypothesis 5). This finding lends support to the Pygmalion effect (Rosenthal & Jacobson, 1968); under-estimators take on board the positive feedback from their followers, and this motivates them to improve their leadership behaviors, and followers appreciate these changes. A possible explanation may be that these leaders experienced a confidence boost by receiving positive feedback from their followers. This, in combination with having received the tools and methods needed to change style and the requirement to develop action plans during training, increased under-estimators’ transformational leadership behaviors as acknowledged by their followers, who already hold positive schemas of their behaviors. Yammarino and Atwater (1997) predicted mixed results of these leaders, and it would appear that in our case, followers appreciated their attempts to change, even if leaders themselves did not perceive they changed much.
In-agreement, good leaders were perceived by followers to improve more on contingent reward compared with under-estimators (Hypothesis 6), but not so on transformational leadership (Hypothesis 6). It is possible that transformational leadership is a less concrete style compared with contingent reward, and therefore, under-estimators may benefit more from followers’ feedback enabling them to know how to increase transformational leadership behaviors. Contrary to expectations, we found that over-estimators were rated by followers to increase their transformational and contingent reward leadership more than in-agreement, poor leaders. These results suggest that self-confidence plays an important role; over-estimators have faith in their own leadership ability, and negative feedback from followers during training may inspire them to improve their behaviors, despite the prediction from Yammarino and Atwater (1997) that these leaders have the most negative outcomes. Our results suggest that these leaders may not be as narcissistic and uncaring by their followers as suggested by Fleenor et al. (2010) and that followers do perceive and rate attempts of over-estimators to change leadership styles post-training; followers at least partly change their negative schemas of these leaders.
Followers rated in-agreement, good leaders reduced their MBEP leadership the most and under-estimators the second. These findings support Yammarino and Atwater (1997) and suggest that although leaders do not perceive they reduce these behaviors, changes are observed by their followers. Testing laissez faire leadership (Hypothesis 8), we found that under-estimators reduced their laissez faire leadership more than over-estimators. It is possible that the negative feedback resulted in over-estimators becoming withdrawn, and thus, any changes to laissez faire leadership are unlikely to be observed.

Implications for Research

Overall, we found support for the assumption that leaders react differentially to training depending on their levels of agreements with their followers (Yammarino & Atwater, 1997). The underlying individual and personality traits of SOA leader types may explain whether leaders changed their behaviors post-training as perceived by both followers and leaders themselves. Our results suggest that the degree to which leaders are able to identify and react to followers’ feedback may have an impact on whether they change leadership styles as a result of training, as perceived by themselves and their followers. Our results were not straightforward. For some leadership behaviors, e.g., contingent reward, in-agreement, good leaders were found to improve the most, but for transformational leadership, followers reported more improvements in under-estimators than in-agreement, good leaders. Although progress has been made to understand the underlying traits of SOA leaders, we need to understand more about the traits of the different SOA leaders before we can make reliable predictions about how and why training outcomes differ.
We found that leaders and followers may have differential views on which type of SOA leader changed the most. In-agreement, good leaders reported the greatest changes both for transformational leadership and contingent reward. Followers only rated improved contingent reward; they perceived under-estimators to increase their transformational leadership behaviors more, as we hypothesized. For MBEP and laissez faire leadership, the different types of leaders (e.g., in-agreement good leaders) did not differ in their ratings, but followers rated the greatest decreases in laissez faire leadership in under-estimators and the greatest decreases in MBEP in-agreement, good leaders. Our results suggest that the Yammarino and Atwater (1997) typology of SOA leaders need to be refined as outcomes of SOA depend on the eyes of the beholder, be it leaders themselves or followers.

Implications for Practice

Our study has important implications for practice. Lacerenza et al. (2017) found no difference between leaders who received single-source feedback and multi-source feedback; however, our results suggest that the impact of multi-source feedback depends on the extent to which leaders are in agreement with their followers and whether such agreement is positive or negative. Feeding back leadership styles should include information about the level of agreement, including whether their followers rate them higher or lower than themselves or agree with them, not just ratings leaving leaders on training to draw their own conclusions on agreement
Our results suggest that in-agreement, poor leaders find it difficult to change their style, implying that providing tools and methods for change during training is insufficient if there is a lack of confidence, motivation, and/or ability to make changes. Additional training aimed at increasing self-confidence and self-efficacy may be needed, for example through web-based training where leaders learn about the self-efficacy and complete tasks on how they can act differently in challenging situations (Luthans, Avey, & Patera, 2008). Training these leaders in self-efficacy prior to leadership training may help ensure a successful outcome of such training. Leader development programs could include educational components focused upon the utility of multi-source feedback to encourage positive attitudes.
Furthermore, our results indicate that over-estimators continue to over-rate themselves post-training. Alternative ways of supporting leader change may be required for over-estimators, e.g., integration in to key performance indicators and additional efforts to motivate leadership style change. Over-estimators who have received unexpected negative feedback from their followers during training may need support to take on board rather than reject feedback. Feedback sessions during training may be structured so that over-estimators receive feedback in a group of leaders with the same profile. If other leaders receive similar patterns of feedback, it may seem less threatening and a group feeling of “we can change” may occur. If feedback is given or as in our study in one-to-one sessions, additional support to motivate over-estimators should be provided.
In addition to tailoring training to the four types, steps could be taken to create alignment between leaders and their followers’ perceptions. Ostroff, Atwater, and Feinberg (2004) also found that more experienced leaders tended to overrate themselves, and thus, ongoing sense checks should be put in place, where feedback is provided to leaders about their leadership styles. Adjustments to leaders’ self-ratings may be supported through regular discussions and tools supporting reflective work practices where leaders and followers jointly reflect on leaders’ behaviors and what changes need to be made. As found by Mackie (2015), coaching may help over-estimators develop a more realistic self-image.
Many companies conduct annual attitude surveys on working conditions, and often include leadership items in these. Compiling and feeding back SOA on the results of the survey may facilitate reflection in leaders. Feeding back SOA on working conditions, rather than just leadership, may feel less threatening to leaders, and they may be more open to explore disagreements with their followers. Ostroff et al. (2004) found span of control to be an antecedent of disagreement, i.e., the more followers a leader had, the greater the disagreement. Organizing work is smaller teams may thus be one way of increasing SOA; however, care must also be taken to ensure that leaders and followers interact and engage in reflections on their working conditions and leadership practices.

Strengths and Limitations

Clear strengths of our study are the prospective study design allowing us to evaluate the effects of multi-source feedback integrated into leadership training. The training was designed according to best practices: Training needs analysis, face-to-face training, spacing of sessions over time, external instructors delivering the training, practice-based learning methods (Lacerenza et al., 2017). The multi-source and prospective nature of our data means common method bias is unlikely to pose a threat to our results (Podsakoff, Mackenzie, & Podsakoff, 2012). Furthermore, the multilevel data (Hox, 2010) and the polynomial regressions, which include interactions (Siemsen, Roth, & Oliveira, 2010) also reduce this threat.
To the best of our knowledge, this is the first study to explore the impact of multi-source feedback pre-training on changes in leadership post-training and thus providing valuable insights to how the social context influences training outcomes; however, our study is not without limitations. First, the study took place in one organization, which limits generalizability. Future research should explore how our results translate to other settings.
Second, a limitation which this study shares with other studies on SOA is the statistical calculation of agreement and disagreement (Fleenor et al., 2010; Lee & Carpenter, 2018). Although this allows for an objective calculation, it fails to capture whether leaders and followers themselves experience disagreement or alignment of their perceptions of constructive and passive leadership. It may be worthwhile including such subjective measures to test whether they explain outcomes of training above and beyond these objective calculations.
Third, we measured leadership styles before and after training. SOA before training was found to be related to changes in leadership styles post-training; however, we were unable to make observations during the training as to how feedback influenced leaders’ attitudes and behaviors. We are thus unable to establish causality and the mechanisms by which SOA influences leaders’ reactions to training. Future studies should supplement before and after measurements with observations of how leaders react to feedback and how they subsequently engage with training activities.
Fourth, our outcome measure was leadership as rated by leaders and followers. This outcome is given the aim of this study (exploring how perceptions of leadership change); adding objective measures of leadership effectiveness would add another dimension to the study.
Fifth, we asked leaders to select which of their followers should participate in the study. This choice potentially leads to bias; however, an indication that leaders were not biased in their selection of their followers invited is the fact that followers did disagree with their leaders on their leadership styles.
Sixth, the relatively small sample size (68 leaders and 237 followers) potentially contributed to low statistical power in our analyses. Therefore, our results should be interpreted in light of the possibility of a Type-II error. A sensitivity analysis revealed that this sample, with a power of 0.80, can detect a medium-sized (f2 = 0.17) change in R2 going from a two main effects model to a polynomial model in terms of adding the interaction and two quadratic terms (Faul, Erdfelder, Buchner, & Lang, 2009).
Finally, we focused on full-range leadership. In recent years, transformational leadership has been criticized for lacking a clear conceptual definition, being confused/confounded with charismatic leadership, being defined in terms of its outcomes of effectiveness, and using problematic measures to capture the concept (Antonakis, Bastardoz, Jacquart, & Shamir, 2016; van Knippenberg & Sitkin, 2013). More recently, it has been argued that rather than abandoning transformational leadership altogether, the theory itself holds value (Siangchokyoo, Klinger, & Campion, 2020)); however, the application of the theory has suffered as it became dominant in the leadership field too early in its development, and it is necessary to go back to drawing board and address some of the issues. One of the strengths of transformational leadership is the strong evidence linking transformational leadership to follower well-being (Arnold, 2017; Harms et al., 2017; Inceoglu et al., 2018; Skakon et al., 2010), suggesting it does have a positive impact on outcomes not inherent in its conceptualization, i.e., performance. Furthermore, transformational leadership theory does offer explanatory value in relation to effectiveness above newer leadership constructs (Hoch et al., 2018).
In our study, we found low reliability on some of the MLQ subscales on the leader subscales. This low reliability is somewhat unexpected given the wide usage of the scale (Judge & Piccolo, 2004), however, links to the criticisms of van Knippenberg and Sitkin (2013). Given that reliability is dependent on the sample size and number of items included in a scale (Raykov & Marcoulides, 2011), the combination of the low number of leaders and few items (4 items per subscale) may explain the low reliability in our data. Low reliability may attenuate relationships between variables (Schmitt, 1996), which suggests that some relationships in our study may have been underestimated. The low reliability may also have increased the discrepancy in our data, increasing the percentage of leader and followers who were not in agreement. Siangchokyoo et al. (2020) argued that new measures to capture transformational leadership should be developed. It was not feasible to develop and validate new scales in the present study, but we urge scholars to develop and validate scales in future studies.
The original full-range theory focused on leaders’ transformation of followers (Siangchokyoo et al., 2020). We focus on full-range leadership as the organization under study had identified leaders as the vehicle for transforming their followers and that they wanted to change the leadership according to the principles of this type of leadership. Based on the analysis of what the organization wished to achieve, we focused on the dimensions of idealized influence, intellectual stimulation, inspirational motivation, and individualized consideration (Bass & Riggio, 2006). Siangchokyoo et al. (2020) argued for taking a step back and conducting quasi-experimental studies to develop the original transformational leadership theory. In the present study, we addressed this call by examining in which circumstances full-range leadership can be trained.

Conclusion

The main contributions of the present study are threefold. First, we extended the existing leadership training literature by exploring a range of leadership styles and how these may be impacted by training. We tested the extent to which it is possible to reduce undesirable, passive leadership behaviors. Followers reported this to be the case. Second, we contribute to the leadership training literature exploring the social context within which leaders successfully change their leadership styles as a result of training including feedback. We found that self-other agreement plays an important role, and these results call for future research and practice to rethink how multi-source feedback is used in training situations. Third, we extend the current literature on the outcomes of self-other agreement. We tested outcomes rated by both leaders and followers and found that depending on the source, differences were found in how effective training integrated with multi-source feedback was in changing leadership styles.

Compliance with ethical standards

Disclaimer

The research fund had no involvement in data collection, analysis and interpretation, nor in the decision for submitting this work.
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://​creativecommons.​org/​licenses/​by/​4.​0/​.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Literature
go back to reference Avolio, B. J. (2011). Full range leadership development. Thouasand Oaks: Sage. CrossRef Avolio, B. J. (2011). Full range leadership development. Thouasand Oaks: Sage. CrossRef
go back to reference Avolio, B. J., & Bass, B. M. (2004). Multifactor Leadership Questionnarie: Manual & review copy. Menlo Park: Mind Garden. Avolio, B. J., & Bass, B. M. (2004). Multifactor Leadership Questionnarie: Manual & review copy. Menlo Park: Mind Garden.
go back to reference Bass, B. M., & Riggio, R. E. (2006). Transformational leadership. Mahwah, NJ: Psychology Press. CrossRef Bass, B. M., & Riggio, R. E. (2006). Transformational leadership. Mahwah, NJ: Psychology Press. CrossRef
go back to reference Edwards, J. R. (2002). Alternatives to difference scores: Polynomial regression analysis and response surface methodology. In F. Drasgow & N. W. Schmitt (Eds.), Advances in measurement and data analysis (pp. 350–400). San Francisco: Jossey-Bass. Edwards, J. R. (2002). Alternatives to difference scores: Polynomial regression analysis and response surface methodology. In F. Drasgow & N. W. Schmitt (Eds.), Advances in measurement and data analysis (pp. 350–400). San Francisco: Jossey-Bass.
go back to reference Fleenor, J. W., & Prince, J. M. (1997). Using 360-degree feedback in organizations. Greensboro: Center for Creative Leadership. Fleenor, J. W., & Prince, J. M. (1997). Using 360-degree feedback in organizations. Greensboro: Center for Creative Leadership.
go back to reference Hox, J. J. (2010). Multilevel analysis. Techniques and applications (2nd ed.). New York: Routledge. CrossRef Hox, J. J. (2010). Multilevel analysis. Techniques and applications (2nd ed.). New York: Routledge. CrossRef
go back to reference Ilgen, D. R., & Feldman, J. M. (1983). Performance appraisal: A process focus. Research in Organizational Behavior, 5, 141–197. Ilgen, D. R., & Feldman, J. M. (1983). Performance appraisal: A process focus. Research in Organizational Behavior, 5, 141–197.
go back to reference Kirkpatrick, D. L. (1994). Evaluating training programs: The four levels. San Francisco: Berrett-Kohler. Kirkpatrick, D. L. (1994). Evaluating training programs: The four levels. San Francisco: Berrett-Kohler.
go back to reference Mackie, D. (2015). Who sees change after leadership coaching? An analysis of impact by rater level and self-other alignment on multi-source feedback. International Coaching Psychology Review, 10(2), 118–130. Mackie, D. (2015). Who sees change after leadership coaching? An analysis of impact by rater level and self-other alignment on multi-source feedback. International Coaching Psychology Review, 10(2), 118–130.
go back to reference Raykov, T., & Marcoulides, G. A. (2011). Introduction to psychometric theory. New York: Routledge. CrossRef Raykov, T., & Marcoulides, G. A. (2011). Introduction to psychometric theory. New York: Routledge. CrossRef
go back to reference Rosenthal, R., & Jacobson, L. (1968). Pygmalion in the classroom: Teacher expectation and pupils' intellectual development. New York: Holt, Rinehart, & Winston. CrossRef Rosenthal, R., & Jacobson, L. (1968). Pygmalion in the classroom: Teacher expectation and pupils' intellectual development. New York: Holt, Rinehart, & Winston. CrossRef
go back to reference Siangchokyoo, N., Klinger, R. L., & Campion, E. D. (2020). Follower transformation as the linchpin of transformational leadership theory: A systematic review and future research agenda. The Leadership Quarterly, 31(1), 101341. CrossRef Siangchokyoo, N., Klinger, R. L., & Campion, E. D. (2020). Follower transformation as the linchpin of transformational leadership theory: A systematic review and future research agenda. The Leadership Quarterly, 31(1), 101341. CrossRef
go back to reference Whetten, D. A., & Cameron, K. S. (2007). Developing management skills. Upper Saddle River: Prentice Hall. Whetten, D. A., & Cameron, K. S. (2007). Developing management skills. Upper Saddle River: Prentice Hall.
Metadata
Title
In the Eye of the Beholder: How Self-Other Agreements Influence Leadership Training Outcomes as Perceived by Leaders and Their Followers
Authors
Karina Nielsen
Susanne Tafvelin
Ulrica von Thiele Schwarz
Henna Hasson
Publication date
13-01-2021
Publisher
Springer US
Published in
Journal of Business and Psychology / Issue 1/2022
Print ISSN: 0889-3268
Electronic ISSN: 1573-353X
DOI
https://doi.org/10.1007/s10869-020-09730-3

Other articles of this Issue 1/2022

Journal of Business and Psychology 1/2022 Go to the issue

Premium Partner