Broadening the Study of Participation in the Life Sciences: How Critical Theoretical and Mixed-Methodological Approaches Can Enhance Efforts to Broaden Participation

INTRODUCTION

Researchers, policy makers, scientific funding agencies, companies, and educational institutions and practitioners have invested significantly in the recruitment of women and people from historically underrepresented racial and ethnic backgrounds into science, technology, engineering, and mathematics (STEM) fields over the past 40 years. During this period, some headway has been made in degree matriculation by women and people from underrepresented minority (URM) backgrounds. Some broad disciplinary fields (e.g., the life sciences) experienced higher levels of participation than others (e.g., engineering). In 2012, women earned 58% of bachelor’s, 54% of master’s, and 47% of doctoral degrees in the life sciences (National Science Foundation [NSF], National Center for Science and Engineering Statistics, 2015). In 2014, graduates from URM backgrounds earned 19% of bachelor’s, 17% of master’s, and 11% of doctoral degrees in these disciplines, with 14% of bachelor’s, 13% of master’s, and 7% of doctoral degrees belonging to women from URM backgrounds (WebCASPAR, 2016).

While degree matriculation in the life sciences is near gender parity and larger proportions of life sciences degrees are being awarded to graduates from URM backgrounds, much work remains to broaden participation in the life sciences. Despite women’s greater than 50% rate of undergraduate matriculation, research on the higher education experiences of life sciences majors illustrates ongoing gender gaps in exam performance and participation in whole-class discussions in introductory biology courses (Eddy et al., 2014). Research also illustrates participation in peer discussions in introductory biology courses is influenced by social identities (Eddy et al., 2015). Participation differs in terms of the roles students feel most comfortable taking on, with women preferring to be in collaborative roles; men preferring to be in leadership roles; and Asian-American students, students from URM backgrounds, and international students preferring listening roles more often than their counterparts. Participation also varies with regard to whether students experience anxiety in those discussions, with women and international students experiencing more anxiety in large discussions than their counterparts. This work also described variation in the value students found in peer discussions, with women reporting higher values in peer discussions when they had an ally in the group and lower values without the presence of a friend. These findings demonstrate that what is meant by “participation” in the life sciences extends beyond numeric representation. Numeric representation alone has not erased gender disparities, and many racial and ethnic disparities still exist in life sciences educational spaces.

Improvements in degree matriculation across different social groups have not yet translated into equitable workforce outcomes. Many of the investments made to broaden participation have not been effectively paired with retention and career progression strategies. Based on cross-tabular analyses of publicly available data from the NSF’s 2013 Scientists and Engineers Statistical Data System (SESTAT), shown in Figure 1, among those in the workforce, 77% of women who have their highest degrees in the life sciences are working in fields that are categorized outside the life sciences compared with a slightly lower attrition rate for men, 76% of whom are working outside the life sciences (NSF, 2016). Of women with their highest degrees in the life sciences, 39% are working in fields that are categorized outside STEM altogether. For men in the aggregate, this figure is 49%, a data point I will return to in the next paragraph. Figure 1 also illustrates that 83% of people from URM backgrounds who have their highest degrees in the life sciences are working in fields outside the life sciences, and 48% are working outside STEM (compared with 46% for those who identify as white and 28% for those who identify as Asian).

These patterns become even more complex when we consider within-group differences that account for gender and race simultaneously (Figure 2). For example, the larger attrition rate for men in the life sciences out of the STEM workforce largely applies to white and Asian men. The 2013 SESTAT data also show that this larger attrition rate for men is largely accounted for by the movement of white and Asian men into leadership roles both in academia and in industry. These leadership roles, including company ownership, are not considered STEM roles within the data. These gaps mean considerable talent is being left out of the life sciences innovation enterprise. Perhaps even more importantly, the gaps illustrate social welfare issues that inhibit full participation in the breadth of workplace choices and the reach of scientific benefit to all communities across the social spectrum.

For women and people from URM backgrounds working in fields categorized within the life sciences workforce, research reveals stratification in movement into senior and upper-level leadership positions in academia, government, and industry. For example, within academia, women comprise 46% of assistant professors, 31% of associate professors, and only 23% of full professors in the life sciences. Among the 80 biotechnology companies that filed initial public offerings in 2014, only six had women CEOs, and 20% of these companies had no women in any leadership position (Association for Women in Science, 2015). Much research points to the myriad stratification, cultural barriers, and biases women and people from URM backgrounds experience. These hindrances impact sense of scientific identity, self-efficacy, and fit. They also influence hiring; space and resource allocation; salary and compensation package composition; evaluation; recognition and awards; research grant funding; promotion; tenure; access to key professional networks and mentors; movement into leadership roles; access to venture capital, startup funds, angel funds, and core knowledge for scientific commercialization; and more (Hill et al., 2010; Lincoln et al., 2012; Blume-Kohout, 2014). This work paints a picture of the complex landscape of experiences; yet the quantitative research conducted on the life sciences workforce often glosses over this complexity.

This research methods Essay introduces critical theoretical and mixed-methodological approaches that provide deeper and contextualized understandings of the life sciences pathways and experiences of women and people from URM backgrounds. First, I draw on multidisciplinary research to discuss critical theory and methodologies. This interdisciplinary approach has benefited a variety of fields, such as education, legal studies, and gender studies. I illustrate how it allows us to reflect upon how we collect, measure, interpret, and analyze data, providing novel alternatives for the study of participation in life sciences. Then, I demonstrate the benefits of these approaches for researchers who study diversity and inclusion issues in the life sciences through examples from two critical mixed-methods studies of prominent issues in STEM participation and recognition. The first study pairs critical discourse analysis of the STEM workforce literature, data, and underlying surveys with quantitative analyses of STEM pathways into the workforce. This example illustrates the necessity of questioning popular models of and assumptions about retention. It also demonstrates the importance of intersecting demographic categories and reveals patterns of experience both within and between groups whose access to and participation in STEM we aim to improve. The second study’s critical approach applies research on inequities in prizes awarded by STEM professional societies toward organizational change. This example uses data from the life sciences professional societies to show the importance of placing data within context to broaden participation and understand the challenges in sustaining the changes we seek. Such approaches may aid life sciences education researchers in creating sustainable and desirable change without reproducing problematic systemic stratifications and inequities.

CRITICAL THEORETICAL FRAMEWORKS AND METHODOLOGICAL APPROACHES

Critical theory originated in the 1920s and expanded Marxist critiques of capitalism to the realm of social and cultural criticism (Habermas, 1971). As a self-reflective framework that is connected to social justice aims, critical theory sheds light on “hidden power arrangements, oppressive practices, and ways of thinking” to transform unjust social structures (Baez, 2007, p. 19). True to its history of emancipatory goals, critical theory pays close attention to the use and reproduction of language, knowledge production, power, and oppression. It also encourages looking through ideology “for hidden subtexts, omissions, and answers to questions unposed in order to disrupt, destabilize, and denaturalize those ideologies” (Metcalf, 2014, p. 79).

Contemporary critical theory examines the inseparable, interconnected, and complex nature of identity itself. It relegates importance to engaging holistically the many fluid intricacies, histories, and contexts involved in the relationship between concepts and subjects. For example, critical theory encourages examination of the relationship between the concept gender and the people who identify with that concept. This relationship focus is particularly important among identity concepts traditionally theorized and researched individually, such as gender and race (Crenshaw, 1991; Baez, 2007; Kinzie, 2007).

Feminist science studies, as a branch of critical theory, is particularly useful for framing research on broadening participation in the life sciences. Feminist science studies applies the critical theoretical lens to science as a social and cultural text and draws upon multidisciplinary knowledge from anthropology, sociology, cultural studies, economics, legal studies, education, history, philosophy, political science, and feminism. Here, feminism is not just about the study of women and/or gender but also focuses on the ways in which systems of power, oppression, and subject formation are entangled with one another. This framework provides space for questioning science’s history of hierarchy and exclusion, bias and knowledge construction in science and engineering scholarship, the existence of objectivity, STEM pedagogy and curricula, and the gendered, classed, sexualized, and racialized assumptions underlying scientific work (Haraway, 1991; Fausto-Sterling, 2000; Hammonds and Subramaniam, 2003; Harding, 2006).

Feminist science studies scholars have demonstrated the historical and ongoing connections between STEM and many sociopolitical issues such as environmental harm, global capitalism, militarism, research “proving” the inferiority of and/or reproducing stereotypes about women and people of color, and the inequitable lack of transfer of scientific and technological advancements to the world’s marginalized populations. For example, Harding (2006) explains three major historical practices within science with racially and gendered discriminatory consequences that continue to have impacts in the present. The first sought to define, divide, and rank race and sex by comparative science measures popular from the beginning of the nineteenth century until well after World War II, including craniology, which persistently found the “natural” inferiority of non-Europeans, Jews, women, and other marginalized groups. The second practice involves scientific and technological misuses and abuses, including medical experimentation without informed consent, such as the Tuskegee syphilis experiments and the harvesting and commercialization of HeLa cells; Nazi eugenics; discriminatory testing and uses of reproductive technologies on Puerto Rican women and mainland women of color; and “the environmental racism that disproportionately locates toxic industries and dumps in nonwhite neighborhoods and Third World societies” (p. 19). The third, which is the issue that efforts to broaden participation aim to address, is the practice, regardless of intent, within STEM social structures of excluding, marginalizing, and restricting women and people of color to lower-level jobs. Feminist science studies scholarship continues to interrogate the impacts of this history and the ways in which bias manifests itself within scientific work today. It guides us to question our social scientific measures and metrics, whose story they tell, in which contexts, and for what purposes.

Many critical theorists tend to use qualitative methodologies in their work because of the rich and complex context the methodology can provide. The theorists have also demonstrated, despite the problematic aspects of our scientific and quantitative histories, that it is possible and necessary to conduct scientific and quantitative work from critical and socially just perspectives.

Critical Quantitative and Mixed-Methodological Approaches

In fields and disciplines such as higher education, sociology, and gender studies, in which many scholars engage in critically transformative research, policy, and practice, there is an emergent interest in work that includes critical quantitative analysis, regardless of whether the study is entirely quantitative or it pairs qualitative and quantitative work in a critical mixed-methodological approach (Baez, 2007; Browne, 2007; Carter and Hurtado, 2007; Faircloth et al., 2015; Wells and Stage, 2015). Quantitative research and the large-scale and big data that often accompany it are frequently used to make policy and programmatic decisions. It is particularly important that this work be conducted from a critical perspective that does not propagate inequities and further marginalize or render invisible the experiences of historically marginalized communities. Researchers can use critical perspectives to see patterns on a larger scale while participating in a study design that reflects on the construction of variables; the social, political, historical, institutional, and economic contexts under which they were created; and the potential interpretations and consequences of those interpretations for respondents, communities, policy making, and future research (Metcalf, 2014).

For life sciences education researchers, this methodological approach encourages us to be intentional and purposeful in how we collect, measure, analyze, and interpret data. It allows for a deeper look at our survey design; the purpose and context surrounding each survey item and element; whether our variables and measures are adequately capturing the experiences and self-identities of life sciences students and workers; and how our interpretive and analytical choices influence what part of the picture we see, what suggestions we make and for whom. This approach pushes us to see and acknowledge how past and present research, our own included, might be limited in accounting for those very experiences we seek to understand most in our efforts to broaden participation in the life sciences. Critical quantitative research gives us room, despite these research and data limitations, to construct new measures, to find new ways to engage with and contextualize the data, and to be innovative and transparent in the work that we do. In the following sections, I use two examples from my own research to demonstrate the strengths of this approach for inquiry.

Example: The Study of STEM Pathways

Using a critical quantitative approach does not mean that we must conduct large-scale surveys ourselves; this example illustrates the usefulness of the critical quantitative approach even when the data that form the basis of our analyses is pre-existing. In this study, I applied a critical theoretical approach to reading the literature surrounding the STEM workforce as well as the NSF’s SESTAT surveys and data that form the basis of those workforce studies, with a focus on how STEM pathways and demographic identities are understood and measured. This approach guided me in looking at the social, political, and economic contexts surrounding the surveys and studies and how those contexts can shape the collection and analysis of data, the discourses in the presentation of findings, and the assumptions embedded therein. It also allowed me to look at who and what possibilities are missing from the surveys, the data elements collected, and the interpretation of findings. I then applied those qualitative findings to quantitative analyses of the NSF’s 2006 restricted-access SESTAT data ( www.nsf.gov/statistics/sestat/) to compare different ways of measuring pathways into the STEM workforce (for full details on this study and to see the full regression models and equations, see Metcalf, 2011).

As a result of the critical qualitative analysis of the STEM workforce literature, I found that the most pervasive model for understanding STEM pathways in these workforce studies is the pipeline model. It was developed by the NSF in the 1970s as the political and economic landscape was riddled with Cold War fears and emerging emphases on technological competition. This model was designed by engineers and the National Research Council’s Committee on the Education and Utilization of the Engineer to quantify and predict the number of scientists and engineers needed to fulfill national competitiveness needs through supply-side and linear flow modeling as follows: “Q₁ + ∑f_i + ∑f_o = Q₂, where Q₁ = the number of people in stock at the beginning of the period, ∑f_i = the sum of flows into the stock, ∑f_o = the sum of flows out of the stock, and Q₂ = the number of people in stock at the end of period” (National Research Council, 1986, p. 29; Lucena, 2005, p. 105). In particular, the model conceptualized those who did not flow along the prescribed path as leaks, with U.S. women and minorities (often as a singular group) being discussed as pipeline leaks most often.

This model has discursively survived for decades despite critiques of its validity and usefulness (Husu, 2001; Hammonds and Subramaniam, 2003; Lucena, 2005; Metcalf, 2010, 2011, 2014). Some criticism highlighted its failed predictions; focus on supply without regard to demand; poor measurements; inability to account for varied and nonlinear career paths; tendency to homogenize people, fields, sectors and stages; and lack of consideration to systemic change, structural inequality, and power relations (Metcalf, 2010, 2011, 2014; Gibbs and Griffin, 2013). For example, in 1992, the NSF’s original pipeline studies underwent review during hearings of the House of Representatives Committee on Science, Space, and Technology, in which it was found that legitimate criticism of the model had been suppressed during its design. During the hearings, the model was criticized for its flawed measurements and the failure of its workforce shortage predictions to manifest (Lucena, 2005; Metcalf, 2010, 2011). More recent critiques of the supply-side focus of this model have argued that a singular focus on the supply-side neglects demand-side analysis, especially when the demand-side is influenced by organizational resistance to change and contains persistent barriers to the entry of historically underrepresented groups into STEM fields (Etzkowitz et al., 2000; Black and Stephan, 2005; Metcalf, 2010, 2011). Despite these shortcomings, many STEM workforce researchers and policy makers continue to focus on the supply-side and base the motivation for their work on the assumption that there will be a mass shortage of scientists and engineers (Metcalf, 2010, 2011).

Given the limitations of this model and its pervasiveness in STEM workforce studies and discourse, I specifically sought to compare the pipeline model measurements for retention with other possibilities afforded, yet previously unexplored, by the data set most often used in the workforce studies, the NSF’s restricted-access SESTAT data. As discussed below, I also kept in mind the limitations within the data set itself and how those limitations connected back to the pipeline narrative.

The data within this integrated system are typically collected biennially through three national-level surveys: the National Survey of College Graduates, the National Survey of Recent College Graduates,¹ and the Survey of Doctorate Recipients. At the time of the study, the most recent data were collected in 2006 (n = 105,064) and have a target population of U.S. residents with at least a bachelor’s degree who are 75 years old or younger and are either educated or working in science and engineering. Even in this description, we already begin to see a picture of who is excluded from the definition of science and engineering: those who are engaging in science and engineering work without a formal education or who received an associate’s or technical degree. While the individual surveys that are compiled to construct SESTAT ask respondents about community college attendance and associate’s degrees, these data are only included in SESTAT when the respondent also has a bachelor’s degree. A large body of higher education research shows gender, class, and racial disparities in community college versus bachelor’s-granting institution attendance and in transferring between these institutions, especially for STEM students (Crisp et al., 2009; Malcom, 2010; Packard et al., 2011; Reyes, 2011; Hagedorn and Purnamasari, 2012). Excluding data on these experiences from SESTAT in turn limits our ability to understand the pathways into STEM taken by many of the historically underrepresented groups whose participation we seek to broaden: first-generation college students, students from low socioeconomic backgrounds, and men and women of color.

In addition, the NSF defines science and engineering fields through five categories: 1) computer and mathematical sciences; 2) biological, agricultural, and other life sciences; 3) physical and related sciences; 4) social and related sciences; and 5) engineering. Teaching science and engineering in university settings is included as science and engineering work, while teaching science and engineering in K–12 settings is not. Within this definitional divide also comes a gender divide. By this definition, the kinds of science and engineering education that “count” happen in university settings and not in K–12 spaces, where women currently comprise more than 75% of educators (U.S. Department of Labor, Bureau of Labor Statistics, 2015). Other fields not included in the definition are business, business economics, management, administration, health fields and related occupations, social services and related fields and occupations, sales and marketing, technologies fields and occupations (including computer programming) and arts and humanities fields and occupations. While this definitional set is for science and engineering and not STEM, the exclusion of the technologies, particularly computer programming, is curious, given the ways in which science and technology work are often paired. This is the case not only in fields like computer science but also in emerging fields, like biotechnology, and in the increasing push to commercialize scientific work through technology. This definitional move limits our ability as researchers to accurately measure the ways in which scientific work is conducted.

These limitations also extend to demographic categorization and aggregation. Many of the studies on the STEM workforce, largely influenced by the pipeline model, tend to compare men in the aggregate with women in the aggregate, measure gender using male/female dichotomous sex categories, and rarely consider the experiences of women of color, let alone women from disaggregated racial and ethnic backgrounds with low numeric participation in STEM (Lal et al., 1999; NSF, 2004; Abriola and Davies, 2006; Metcalf, 2010, 2011, 2014). The SESTAT survey and data-set construction limits analytical possibilities by containing data on binary sex rather than gender or gender identity. Gender is a social category referring to the roles, behaviors, activities, and attributes that a society considers normative for men and women (Macey, 2000). It is distinct from sex, which is a category often assigned at birth as female, male, or intersex based on physical and/or biological characteristics. Gender identity is one’s sense of self as a man, woman, or a blend of both or neither. One’s gender identity may differ from one’s assigned sex. The term “cisgender” refers to those whose gender identity socially agrees with their assigned sex. The term “transgender” refers to those whose gender identity socially differs from their assigned sex. “Gender-nonconforming” is an overarching term referring to those whose gender identities reside outside of the options “man” or “woman.” In including only male/female categories, the surveys do not reflect the experiences of transgender and gender-nonconforming respondents and do not really capture “gender” as a social category.

SESTAT also contains narrowly defined racial and ethnic categories. The surveys prevent respondents from indicating that they only identify as Hispanic. The data set also compiles the racial and ethnic categories that respondents have selected to provide a minority indicator variable (i.e., minority vs. nonminority) to researchers for ease of analysis. Researchers have the option to create their own series of indicator variables from the data; yet they are encouraged otherwise through the prefabricated variables and a quantitative history that places importance on statistical significance over the exploration of meaningful and interesting patterns. This history and prefabricated variable tend to result in research in which racial and ethnic minorities are lumped together and treated as a homogeneous social group rather than creating an analytical space that accounts for the rich and complex social histories of the different racial and ethnic groups comprising the category “minority.”

For the purposes of this example, the above limitations are taken into consideration and compared in the following regression models. The models use varying degrees of aggregation of demographic independent variables and two different measures of retention as dependent variables. The first replicates the pipeline model narrative that measures retention by way of pipeline leaks: obtaining a degree in a field that is defined as science and engineering and continuing on to work outside a field defined as science and engineering constitutes a “leak” and means that the person was not retained. The second makes use of another possible measure of retention afforded by the SESTAT data. In this case, retention is measured by how closely related one’s science and engineering degree is to one’s occupation, regardless of whether that occupation counts as science and engineering.

Retention: Leaks = Obtain Science and Engineering Degree → Work in Non–Science and Engineering Job.

To look at the odds of “leaking” by demographic factors, I conducted three binary logistic regressions that measured the dependent retention variable following the pipeline model. For the first regression, I followed the traditional pipeline model assumptions that tend to oversimplify identity characteristics by focusing dichotomously and mutually exclusively on females² and minorities. The second binary logistic model disaggregated identity characteristics where possible. To further complicate identity measures and consider the experiences of, for example, women of color, the third model is a full-interaction binary logistic regression model that interacted the sex variable with each of the independent variables included in the second model. This model, shown as one model for males and one for females in Table 1, was intended to represent the complicated ways in which sex as a social construct overlaps and is intertwined with other identity and social factors.

Table 1, model 1, shows the expected pipeline model results: when holding all other variables constant, women and minorities have higher odds of leaking between obtaining their highest degrees and entering into the workforce than their respective counterparts. However, by disaggregating the race/ethnicity variables and creating interaction terms between sex and race/ethnicity, we find a complicated counternarrative. Models 2–4 illustrate that not all URM groups have equal likelihoods of leaking and allow us to make within-group comparisons. Model 2, for example, shows that black, non-Hispanic respondents have the highest odds of leaking, which are about seven times higher than American Indian/Alaska Native respondents. Non-Hispanic Native Hawaiian/Pacific Islanders have slightly lower odds of leaking relative to those who are white, non-Hispanic and slightly higher odds than those who are Asian, non-Hispanic. Models 3 and 4 continue to unpack this picture for us by allowing us to look at the experiences of women and men from URM backgrounds. For females and males, those from non-Hispanic and American Indian/Alaska Native, black, Native Hawaiian/Pacific Islander, and multiracial backgrounds have higher odds of leaking than their white and Asian counterparts. However, the scale of the likelihood is also variable by sex. For example, black, non-Hispanic males have greater odds of leaking than do females and non-Hispanic American Indian/Alaska Natives, and multiracial females have higher likelihoods than do males. These findings allow us to see that critically considering gender, sex, and race provides us analytical space for complex variation in retention likelihoods, even when the underlying measures are limited. By creating interaction terms between the sex and race categories available in the data, we can begin to see patterns of and nuances within intersectional social experiences that are often unseen when these categories are considered separately.

Retention: Gain Science and Engineering Knowledge Utilized in Workplace.

To consider the odds of retention as alternatively measured by closeness of relationship between one’s degree and one’s occupation (degree–occupation relatedness), I used an ordered logistic regression of the data on those who were employed at the time of the survey. This measure of retention accounts for perceived use of one’s degree knowledge in one’s day-to-day work. As in the binary logistic models, I first modeled the pipeline assumptions about how to account for identity via a basic set of indicator-independent variables. Then, I expanded the model to account for disaggregated identity characteristics.

Table 2, model 1, shows that, when holding all other predictors constant, females are more likely than males and minorities are less likely than nonminorities, to have high degree–occupation relatedness. In stark contrast to the pipeline model measure of retention in Table 1, which showed that females and minorities were significantly less likely to be retained, we see here that females are more likely to be retained and minorities only slightly less likely to be retained. Model 2 also shows that it is possible for individuals to be likely to leak but also likely to have high degree–occupation relatedness. For example, non-Hispanic American Indian/Alaska Native and Hispanic respondents have higher likelihoods of leaking than white, non-Hispanic respondents, but they also have greater likelihoods of having higher degree–occupation relatedness than their counterparts. In many cases, those who were least likely to be retained in the pipeline models depicted in Table 1 were the most likely to be retained when we measure retention as degree–occupation relatedness, as seen in Table 2.

Discussion.

Regardless of intention, many workforce studies and their underlying data sets have a tendency to greatly oversimplify the measurement and understanding of retention outcomes, especially for targeted underrepresented groups. Applying a critical framework to how retention and identity are conceptualized, regardless of whether the mode of analysis falls into qualitative and/or quantitative camps, is just one step toward deepening our understanding of retention issues. Increasing the depth and breadth of our understanding in turn helps generate a richer pool of research-based resources to aid interested parties in achieving, rather than unintentionally undermining, their ultimate equity-based goals. When our overall goal is to broaden science and engineering to include the participation of historically underrepresented groups, we must conduct research that accounts for the spectrum of experiences of those groups. Including the experiences of multiply marginalized groups, such as women of color, and groups that have been historically excluded from analyses because of low n values, such as Native Americans, is key in accounting for this spectrum. This knowledge helps us make thoroughly informed decisions about policy, practice, and organizational change rather than reproducing the status quo.

Many subjective choices are made in the construction of large-scale data sets used to inform much national science and engineering policy, and these choices greatly influence the likelihood of retention outcomes. What counts as science and engineering, and who counts as a scientist and engineer, is socially constructed and varies according to whom you ask. In her work on how engineering has been defined historically by engineers, Pawley (2007) explains,

The definitions of science and engineering work and degrees as determined by NSF for SESTAT have shifted over time and by social and political context. Considering these contexts alongside the where/who and where not/who not questions are important pieces of the research on broadening participation.

Despite these limitations in existing data, with a critical lens, we can still do much to analyze the data in novel and more representative ways. Life sciences researchers using critical approaches are likely to find interesting patterns when exploring variation in how they define and aggregate/disaggregate the field set that counts as life sciences education and work, differing measures of retention, and the demographic categories used and analyzed for those whose experiences they wish to understand.

Example: AWARDs and Recognition

The second study is an ongoing longitudinal critical mixed-methods study of 18 STEM professional societies and the ways in which they allocate their awards to STEM workers. For the purposes of this essay, I will focus on the data from the seven life sciences societies,³ since it is most relevant to this journal’s audience (for greater detail, please see Lincoln et al., 2012; Popejoy and Leboy, 2012).

In this NSF-funded AWARDs study, we drew on our critical theoretical framework and research knowledge about stratification and bias in STEM recognition to conduct a quantitative analysis of the awards allocations by gender and award type. Through our study of the related research, we found that the pipeline narrative was present in this arena as well and was used to argue that equity in awards would occur over time as the recruitment efforts to fix the supply-side were presumed to one day manifest in more women working in STEM. To test this, we used several different baseline measures for the potential pool of award candidates, including: PhDs in the discipline, actual nominees, number of members in the society, and faculty in the discipline. We divided the awards into categories surrounding their value to STEM fields: research and scholarly awards are more highly valued and are connected more specifically to promotion and tenure outcomes, while teaching and service awards are less valued and connected to the caregiving activities of the field. We performed an ordinary least-squares regression analysis of the percentage of women award winners over an award period lasting two decades. In so doing, we found that women received a significantly and disproportionately high number of teaching and service awards and a disproportionately low number of research and scholarly awards relative to their representation in each of the baselines (see Figure 3 for data on the life sciences awards from 1991 to 2014). Our analysis revealed that, despite the growth in award recognition for women during this period, women received a relatively small percentage of scholarly awards compared with teaching awards. Contrary to the pipeline assumption, this disparity actually grew in the 2000s as the growth of women’s receipt of teaching and service awards outpaced the increase in scholarly awards they received.

In addition, social science research on recognition indicates research done by women is often overlooked in favor of that of men, which is more frequently seen as notable (Rossiter, 1993; Lincoln et al., 2012; Popejoy and Leboy, 2012; Knobloch-Westerwick et al., 2013; Grunspan et al., 2016). This is known as the Matilda Effect (Rossiter, 1993; Lincoln et al., 2012; Popejoy and Leboy, 2012; Knobloch-Westerwick et al., 2013). To test for this, we conducted logistic regression analyses for the odds that a man will win scholarly awards and found that men were more than eight times more likely to win scholarly awards than women when accounting for their proportion in the nomination pool and were twice as likely to win scholarly awards regardless of their representation in the nomination pool. We also found that the presence of women on the awards committee benefits women’s odds of winning, but the benefit is erased if the committee is chaired by a man. These findings suggested that a large degree of bias was at play in the current awards allocation processes within the professional societies.

As part of our interventions and partnership with the professional societies, with the support and presence of the society and awards committee leadership, we then presented these findings as evidence that further action needed to take place. We shared the findings in conjunction with unconscious bias trainings and further data gathering by the participants as a review of their processes. We then brainstormed, within the context of each society, how to make revisions toward systemic change. The kinds of changes included: continuing bias trainings, creating more diverse selection committees, revising the language in calls for awards nominations and in the selection criteria, and creating greater transparency.

Since our interventions occurred, it appeared as if some progress had been made. However, using the critical lens to revisit the data and consider the context of the timing of the interventions and to disaggregate the longitudinal data accordingly, we found a different pattern that speaks greatly to the limits of interventions on sustainable change (Figure 4). These intervention periods occurred twice, with 2 years in between. In the award cycle immediately following our interventions, we found that the awards allocations became more equitable. But, between interventions, the progress gained had diminished substantially and then, after the second intervention, moved closer to equity again. This allowed us to see that, even with the heavy involvement of the societies in reshaping their processes within their own organizational contexts, continued and repeated efforts toward establishing new cultural norms within the professional societies is necessary. Otherwise, as awards committees shift membership, leadership turns over, and time constraints occur, committees will fall back on old and problematic patterns of behavior. Our ongoing work is exploring how to make these changes more sustainable, particularly in environments with relatively high leadership turnover rates.

Discussion.

The critical approach suggested here allowed us to find patterns in the awards data not yet seen and to push back against problematic discourses that were preventing action and change. It also provided us with contextual evidence to discuss with well-intentioned society leadership who beforehand had not recognized the gravity of disparity in awards and what could be done to address it. The self-reflective aspects of the framework required we continue to revisit our data and consider them thoroughly and with as much of the surrounding context as possible. This gave us the opportunity to see the sustainability issues in our intervention model for change and has sent us in new directions for inquiry and practice.

CONCLUSION

This essay highlights the importance and usefulness of critically looking at the ways in which we collect, measure, interpret, and analyze data. For life sciences education researchers, whose work influences institutional policies, programs, and practice, such an approach has the transformative ability to expose and create space for altering rather than reproducing problematic institutional arrangements, stratifications, and inequities. For life sciences education researchers seeking to broaden participation, using a critical theoretical lens and methodological approach can guide us in continuing to revisit our data, discourses, practices, pedagogical approaches, and modes of inquiry. Such an approach and line of questioning is imperative if we want to realize lasting change and finally see movement toward equity.

FOOTNOTES

REFERENCES