19. Seeking Feedback on Learning for Change

The rapidly changing—and, at times, excessively complex—nature of development work demands diverse competences from aid agencies such as the Asian Development Bank. In addition to technical knowledge and skills, they include no less than appreciating political economy; building relationships; reading and responding to complex organizational and social predicaments; and increasing capacity to contend with uncertainty, task-compromise, and deal with difference and diversity. The learning challenges that these demands present require the ability to work more reflectively in a turbulent practice environment. There is no alternative: to remain relevant and effective, an organization’s rate of learning must be at least equal to—but preferably greater than—the rate of change in the environment.

Learning for Change in ADB (2009) was published to help deliver the increased development effectiveness that Strategy 2020, ADB’s long-term strategic framework, seeks. It broadly defined a learning organization as a collective undertaking, rooted in action, that builds and improves its own practice by consciously and continually devising and developing the means to draw learning from its own (and others’) experience. Usefully, it assimilated the manifold dimensions of the learning organization and marked out generic roadblocks to learning. Notably, Learning for Change in ADB specified how action across organization, people, knowledge, and technology—the learning organization model it created—can energize and support individual, team, and cross-functional learning, and in return be enriched by learning.

Organizations that neither invite nor cherish feedback from personnel, clients, audiences, and partners are working in a vacuum. Feedback is the answer to their travails. These Knowledge Solutions showcase the details of the internal Learning for Change survey that ADB launched in 2010 to gauge perceptions of competencies to learn for change in ADB.

Across departments, 47% of the respondents were from the regional departments,⁴ 12% from the knowledge departments,⁵ and 41% from other departments (Fig. 19.3). Departmental response rates varied considerably—from 2 to 75%, with 7 out of the 23 responding departments showing relatively high response rates of 20% or over.

Feedback from staff members on each statement deserves dedicated attention. However, overall, survey results (Sects. “The Organization Subsystem–The People Subsystem by Staff Category”) indicate that, among the four subsystems, staff members perceived ADB to be most competent in relation to the technology subsystem; it needs most improvement in the people subsystem. Dropping the “don’t know” (or 6) responses and inverting the Likert scale, where 5 is equivalent to “strongly agree” (hence becoming the ideal score) and 1 represents “strongly disagree,” the mean score for the responses for all 10 items under the technology subsystem is 3.32 while that for the people subsystem is 3.02 (Fig. 19.4). The organization and knowledge subsystems have mean scores of 3.26 and 3.17, respectively.

Taking into account the average percentage of respondents who collectively “strongly agree” and “agree” to the 10 statements per subsystem, the rankings of the subsystems are retained. The technology and people subsystems are first and last at 53% (Table 19.4) and 38% (Table 19.2), respectively. The organization subsystem ranks second at 48% (Table 19.1) while the knowledge subsystem is positioned third at 43% (Table 19.3).

Among the 10 statements under the organization subsystem, respondents “strongly agree” and “agree” most with two statements (59% for each): (i) “there is an inspiring vision for learning and an organizational learning strategy that clearly communicates that learning is critical to organizational success” (statement 1), and (ii) “emergent learning is encouraged by creating opportunities for informal sharing of knowledge and experience” (statement 9). The majority (51%) also agrees that “leaders take an exemplary leading role in creating and sustaining a supportive learning culture” (statement 2). The highest rate of neutrality (36%) is toward the statement, “Failures and unintended outcomes are the focus of constructive discussions. When such incidents involve clients, care is taken to protect their reputation” (statement 10). This is also one of two statements for which the highest percentage of “don’t know” responses overall (13%) is found (probably because many respondents would be unable to answer the second part of the statement regarding the protection of client reputation). Finally, there is highest rate of disagreement (34%) with the statement, “adequate resources are allocated for learning in terms of time, allocation, specialist support staff, budgets for knowledge management infrastructure, formal and informal communities of practice and other value networks, and learning and development programs” (statement 6). Table 19.1 presents the survey results for the organization subsystem.

The responses to statement 1 demonstrate a strong sense of communicated vision concerning the importance of learning for ADB’s success. This is a critical issue and provides a strong platform for building ADB’s organizational learning capacity. The responses to statement 2 concerning leadership’s role in creating a supportive learning culture are encouraging. The fact that the number of respondents who “agree” with this statement outnumber those who “disagree” with it by over two to one, with a sizeable share being “neutral,” suggests that many ADB leaders are successfully creating a supportive learning culture for their colleagues. It is also significant that statements 1 and 2 have the highest “strongly agree” responses (13 and 12%, respectively). This indicates a firm belief on the part of those respondents which suggests that there may be some examples of particularly good practice to disseminate and learn from. Statement 4 concerning sanctioned informal organizational structures reveals the widest gap in the organization subsystem between those who “strongly agree” (4%) and those who “agree” (43%). This suggests the need to examine current areas of good practice and make these examples more widely known within ADB. In statement 7, the noticeable clustering around “neutral” may be explained by a lack of understanding of the terms “planned” and “emergent” when applied to learning and a reluctance to admit to this in the “don’t know” category. Statement 10 on “failures and unintended outcomes” reveals the highest level of “neutral” responses (36%). This may be explained by the limited awareness of many respondents concerning the second half of the statement on the protection of client reputations.

Regarding the 10 statements under this subsystem, respondents generally “strongly agree” and “agree” (52%) with the statement, “leadership (based on the possession of expertise and knowledge) is expected from staff members at all levels in the organizational hierarchy” (statement 10). ADB staff are mostly “neutral” (32%) about the proposition that “staff members successfully use a wide range of opportunities for individual and team-based learning and development” (statement 7). The highest percentage of respondents (44%) “strongly disagree” and “disagree” that ADB “uses a wide range of formal and informal rewards and incentives for contributing to organizational learning and knowledge development (e.g., career advancement, increased income, informal peer status, additional time provided for study, and public acknowledgment for innovative contributions made)” (statement 9). Table 19.2 presents the survey results for the people subsystem.

In general, the statements in the people subsystem are likely to represent aspects of organizational learning with which respondents are most familiar. As a result, one would expect to see the highest prevalence of “strongly agree” and “strongly disagree” responses to these statements. What is interesting, however, is that 7 out of the 10 statements cluster noticeably around the “agree”–“neutral”–“disagree” responses with these three taking up 78% for statement 9, 80% for statement 6, 87% of responses for statements 2 and 4, and a peak of 88% of responses in statement 3. The broad balance between “agree”–“neutral”–“disagree” responses suggests a very varied perception of experiences and the potential for significant improvements in this dimension of organizational learning in ADB. Adopting a learning charter might help establish ground rules and common reference points.

Segregating responses by staff category—namely, local staff⁶ and professional staff—it is evident that local staff feel more positively about the people subsystem, with a greater share of them agreeing to more statements than professional staff (Sect. “The People Subsystem by Staff Category”). For example, over half of local staff “strongly agree” and “agree” collectively to five statements—namely, statements 1, 4, 5, 8, and 10—while the highest rate of agreement for professional staff is 44% for statement 10 only. Further, no professional staff “strongly agree” with statements 2, 6, 7, and 9. Finally, there is a difference of 25% points and above between the local and professional staff for those agreeing with statements 2, 4, 7, and 8. It would be interesting to inquire in depth why this is so.

In terms of the knowledge subsystem, the statement to which the highest percentage of respondents “strongly agree” and “agree” (64%) is “there is widespread recognition that while knowledge is created in the minds of individuals, knowledge development thrives in a rich web of professional networks among individuals” (statement 1). The good majority (59 and 51%, respectively) also “ strongly agree” and “agree” that in ADB (i) “there are creative opportunities for knowledge to be developed and shared with others by facilitating networks between individuals” (statement 3), and (ii) “the design and delivery of products and services demonstrate how effective the organization is at applying what it has learned about the nature of good practice” (statement 4). ADB staff were found to be largely “neutral” (32%) to the statement, “adoption of after-action reviews and retrospects to learn from experience has been successful” (statement 10). Incidentally, this is the other statement (as mentioned in paragraph 6) to which the highest rate (13%) of “don’t know” responses was recorded (explicable, perhaps, by lack of familiarity with the concepts used). On the other hand, respondents most “disagree” and “strongly disagree” (44%) that “the organization has a resilient organizational memory and is not vulnerable to the loss of important knowledge when staff members move to other jobs in the organization or leave” (statement 8). This reveals a significant concern about loss (or potential loss) of organizational memory. This should be of real concern to the Human Resources Division, which provides the “last resort” means of capturing knowledge and expertise from departing staff. However, better means of addressing this would be the introduction of exit interviews in offices and departments and a personal commitment from each staff to consider: “What knowledge and experience should I be passing on to colleagues so that they and ADB gain from my work in case I move on to another job?” Table 19.3 presents the survey results for the knowledge subsystem.

The balanced responses to statement 5 suggest that further investigation would be fruitful here to examine what systems and infrastructure for knowledge management need to be developed, better understood, or made more effective. The responses to statement 6 reveal a very diverse range of views. It is a complex statement that would benefit from further investigation.

Under the technology subsystem, 74% “strongly agree” and “agree” that “information and communications technology is successfully used to keep people informed and aware of corporate developments” (statement 4). Further, more than half of the respondents believe in the following statements: (i) “There is a thorough and shared understanding of the value of information and communications technology for knowledge management and learning” (statement 1, 54%), (ii) “Information and communications technology facilitates but does not drive or constrain knowledge management and learning in the organization” (statement 2, 63%), (iii) “Information and communications technology is successfully used to create and sustain learning communities” (statement 3, 52%), and (iv) “Creative use of information and communications technology is high. At least five of the following have been successfully adopted: shared document drives, intranet pages, online communities and networks, wikis, and other means of collaborative document production, blogging, online storytelling, lessons learned databases, staff profile pages, online webinars, podcasts, and social network mapping” (statement 9, 54%). The most “neutral” responses are found in the statement, “information and communications technology is successfully used to enable people to identify internal sources of expertise” (statement 8). The highest percentage of respondents (although only 26%) “strongly disagree” and “disagree” that “sufficient opportunities are provided for staff members to learn how to make use of available information and communications technology for learning and sharing” (statement 10). Table 19.4 presents the survey results for the technology subsystem.

The use of technology for knowledge management and organizational learning seems to emerge as a success story in most responses. It would be interesting to understand in depth why this is so. Positive perceptions may owe in part because the uptake and use of technology in ADB does not require the support of managers or the creation of a supportive learning environment in one’s team—one can use the technology if it is there and one knows how. The responses to statement 3 represent a very positive assessment of the contribution of technology to learning communities. Statement 4 shows a very significant recognition by respondents of the value of technology in keeping them informed of corporate developments. While this may not in itself lead to improved organizational learning, personnel who are aware of their place in the wider organization are generally considered to be more likely to contribute their knowledge for its collective good. The responses to statement 8 suggest a need for greater use of technology for internal peer support. Peer assists require an understanding of where expertise resides (not only in terms of current roles and responsibilities but also in light of previous experience). Staff profile pages surely have a role to play.

The largest proportion of respondents (74%) “strongly agree” and “agree” that “ADB has made successful use of information and communications technology to keep people informed and aware of corporate developments” (technology subsystem, statement 4), while the highest rate of disagreement (44%) is with the statement that ADB has a “resilient organizational memory and is not vulnerable to the loss of important knowledge when staff members move to other jobs in the organization or leave” (knowledge subsystem, statement 8).

Three statements pertaining to actions taken by ADB ex-post (e.g., evaluations, adoption of after-action reviews and retrospects) consistently generated relatively high frequencies of “don’t know” responses. To wit, these statements are (i) “Failures and unintended outcomes are the focus of constructive discussions leading to new approaches. When such incidents involve clients, care is taken to protect their reputation” (organization subsystem, statement 10, 13%); (ii) “Evaluations are carefully designed with learning (as well as accountability) in mind. Systems ensure that the outputs of internal and independent evaluations are made widely available; carefully examined; and used to influence decision making and planning, question orthodox thinking, and trigger creativity and innovation” (knowledge subsystem, statement 6, 11%); and (iii) “Adoption of after-action reviews and retrospects to learn from experience has been successful” (knowledge subsystem, statement 10, 13%). This reflects minimal awareness of existing policies or the lack thereof.

The survey response rate of 9%, given likely survey fatigue in ADB, is acceptable if not robust and compares reasonably with what is usually considered good for an online survey (10%). What are very positive are the high response rates from some departments: these demonstrate what is possible when both participant interest and management encouragement are present. The survey mean scores per subsystem are all above 3 (the score that represents “neutral”), which indicates a somewhat favorable yet uncertain view of ADB’s capacities, barring exceptions. It is worth remembering that, using a five-point Likert scale, to reach a mean score of 4 would require a significant number of “strongly agree” and “agree” responses to balance “neutral”, “disagree”, and “strongly disagree” feedback so any organization is highly unlikely to achieve a mean score of 4, let alone the ideal score of 5. This needs to be emphasized in any interpretation of results. In these circumstances, indicators such as the percentage of “strongly agree” and “strongly disagree” responses become significant. Moreover, one should bear in mind that mean scores can obscure significant differences in distributions of responses between statements.

Organizations do not change—the people in them change, and then change their organization. Surveys provide the starting point for effective interventions. On organizational learning, ADB now has a baseline against which to gauge perception of progress, noting that cultural change can take time to anchor. Since the value of any survey increases considerably when there are two or more data sets to compare, therefore, it is recommended that the survey be repeated, perhaps annually, primarily for purposes of organizational learning, not performance measurement. ADB should also ponder what is likely to encourage higher response rates: communicating key findings from the survey⁷; involving staff in office-and department-led improvements plans to secure buy-in and build lessons into systems, the principal recommendation from the meeting of 3 September, 2010 with ADB’s knowledge management coordinators⁸; sharing evidence that the survey has led to actions being taken⁹; and requesting management to both take part and encourage personnel to participate in future surveys, thereby enabling deeper analysis at office and departmental levels.¹⁰