Research that addresses complex challenges often requires contributions from the social, life and natural sciences. The disciplines that contribute subject response data, and more specifically qualitative analyses of subject response data, to interdisciplinary studies are characterised by low consensus with respect to methods they use a diversity of terms to describe those methods and they often work from assumptions that are foreign to readers in the natural and life sciences. The first contribution this paper makes is to demonstrate that the forms of reporting that may be adequate for communicating quantitative analysis do not provide teams that include members from natural, life and social sciences with useful accounts of qualitative analysis. Our second contribution is to discuss and model how to report four methods appropriate for qualitative contributions to interdisciplinary projects.
Hinweise
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
1 Introduction
There are strong arguments to combine quantitative analysis and qualitative analysis within the social sciences (Babbie 1989; Creswell and Clark 2000; Johnson Onwuegbuzie and Turner 2007). Research that addresses complex challenges, such as adaptation to the effects of climate change, often involves teams from the social, life and natural sciences. These interdisciplinary studies frequently demand teams to integrate qualitative analysis of subject response data with quantitative analysis of direct measures of natural phenomena. Further, reports of these studies are often presented in journals whose reporting formats anticipate quantitative analysis of direct measurements for natural and life science readers. We have found specific guidance on the design of interdisciplinary research (e.g. Tobi & Kampen 2018), on how to make it meaningful for policy (e.g. Kampen and Tamás 2014) and we have found a large number of guidelines for the reporting of both quantitative and qualitative analysis for both disciplinary researchers and for those times when interdisciplinarity is limited to the social sciences. Despite our own and our peers’ efforts, we have not found guidelines for the presentation of the qualitative analysis of subject response data that well serve integration into the reports of interdisciplinary studies published in journals that are read outside of the social sciences.
Our purpose with this paper is to strengthen inter-disciplinary science by improving the adequacy of the reports of analysis of qualitative subject response data within reports of interdisciplinary studies. In the next section we demonstrate the need for these guidelines by describing and faulting existing reporting practices. The guidance we then offer is presented through the use of a model case. The analysis methods we present in this model case were selected for their relevance to interdisciplinary research addressing environmental challenges.
Anzeige
2 The transparency of reporting in interdisciplinary research
In preparation for this manuscript we downloaded four years of papers that contained both ‘interdisciplinary’ and ‘interview’ in their titles, keywords and abstracts (N = 1160 papers).1 The term ‘interdisciplinary’ was selected as we were certain that authors’ self-identification would be a strong indicator of interdisciplinarity and the term ‘interview’ was selected as the alternatives we considered, such as ‘qualitative’ produced high false-positive rates. We recognize that this search strategy likely excluded many studies which compromises the generalisability of our findings. We then used automatic coding in Atlas.ti to identify all paragraphs that contained both words ‘analysis’ and ‘interview’ (n = 1033 paragraphs) to quickly identify those papers that contained a substantial discussion of the methods used to analyse interview data and a location within papers where that discussion is certain to be found. We then used random selection from these paragraphs to identify papers for examination. We continued to randomly select papers for examination until five in a row produced no novel observations (n = 79 papers).
In all of the papers examined, researchers reported that they identified and aggregated themes in order to present patterns. The description given these efforts generally mirrored the account given of their analysis of quantitative data. For example, many reported ‘thematic content analysis’ which appears to be as informative as ‘multiple logistic regression.’ These two are neither equivalent nor are they similarly informative. The term ‘multiple logistic regression’ references a specific set of analysis procedures and assumptions about which there is well-known consensus. Thematic content analysis, however, involves two distinct steps neither of which benefits from the consensus supporting interpretation of the term ‘multiple logistic regression’. The first step in thematic content analysis is the attachment of codes to text that capture meaning. This step, coding, is akin to measurement or data processing in the natural and life sciences. The codes applied are the equivalent to the pH value recorded by a researcher when using litmus strips to measure acidity in surface water or the calculation of BMI based on data provided on weight and height.
Staying with the step of coding, which tended to be far better discussed than synthesis, in the articles we reviewed it was consistently clear that researchers identified themes, but it was not clear where those themes came from. Unlike chemistry, where budding scientists are taught consistently how to read litmus strips so the reader knows what procedures lie behind a stated pH value, there is no consensus in the social sciences that we know of that allows a reader to infer from ‘thematic content analysis’ an unequivocal understanding of how researchers identified units of text as meaningful and then determined what speakers meant by what they said. Certainly, many of the articles we reviewed used multiple raters and negotiation to improve reliability, but inter-rater agreement does not improve transparency in the manner required to shed light on validity.
Turning now to synthesis, we did not often find interpretable discussion of mechanisms by which the text strings coded by researchers were combined so as to produce the patterns that were reported. In the quantitative world, this would be the same as presenting the manipulation of data as a ‘cluster analysis’ without any further specification of the math and the criterion used to support identification of the patterns reported. Those cases that provided an interpretable account tended to be informed by well referenced use of grounded theory in which the processes and logic behind a line of argument and refutational synthesis are clearly stated.
Anzeige
In summary, analysis of qualitative subject response data and quantitative direct measures data arise from disciplines that vary dramatically in their level of consensus. Therefore, qualitative analysis requires far more detailed reporting than is normally found in the accounts given of quantitative analysis. In the following section we introduce and then provide and discuss model reports for the analysis of narrative subject-response data in research that is both mixed-methods and interdisciplinary.
3 Model case
3.1 Material and Methods
3.1.1 Material
To demonstrate appropriate presentation of the qualitative analysis of subject response data within reports of interdisciplinary studies we used transcripts of eleven semi-structured interviews that were part of an interdisciplinary study in the domain of socio-technical studies. The questions that we use to provide model presentations here are (a) how do international graduate students use ICT technology to maintain ties with their household members and (b) what is the meaning of ‘household’ as experienced by those students. This material, relatively unstructured interview data, is typical of that subjected to qualitative analysis in interdisciplinary research.
Both the interviews and the transcripts were done in Spring 2012 by Jarkyn Shadymanova, at the time research fellow at the Sociology of Consumption and Households Group, Wageningen University, the Netherlands. Interviewees were African graduate students of Wageningen University who were interviewed in English. In the remainder of this paper the numbers P1 to P11 are used to refer to these interviewees. As should be found in such reports, a demographic description of the interviewees is presented in Annex one.
3.1.2 Methods
For this paper we exercised simple forms of four methods of analysis, aspects of which we have often found to be silently combined by researchers who are contributing to interdisciplinary studies: content analysis, metaphor analysis, domain analysis and membership categorization analysis. For each method we provide exemplar texts for a ‘materials and methods’ and for a ‘results’ section that are preceded by an introduction to the method and followed by a discussion of the method and its reporting.
Each of the methods we have chosen to model and discuss is understood and used in diverse ways. Silverman (2015), for instance, mentions content analysis, membership categorization analysis, conversation analysis, discourse analysis, semiotics and workplace studies. Flick (2014) speaks of grounded theory coding, thematic coding and content analysis, conversation, discourse and hermeneutic analysis. Coffey and Atkinson (1996) distinguish narrative analysis, metaphor analysis, and domain analysis. Bernard (1988) addresses narrative, discourse and content analysis. In addition, often ethnography or feminist research is mentioned (Bernard 1988; Grbich 2012; Silverman 2015).
There are also inconsistencies in discussion of coding. Most authors see coding as essential for qualitative analysis. However, they differ in the way they see coding in relation to analysis. Miles and Huberman explicitly state: “Coding is analysis” (Miles and Huberman 1994 p. 56). Others see coding and analysis as two distinct phases, where the latter is of a higher level of abstraction. Flick, citing Strauss and Corbin, 1990, distinguishes coding and ‘axial coding’, where the “Axial coding is the process of relating subcategories to a category” (Flick 2014 p. 311).
In our many years of instruction at the graduate level, our students have consistently recognized on their own that these taxonomies overlap, that the terms included in each are not mutually exclusive and that each taxonomy partitions practice in slightly different ways. In addition to making it impossible to infer from a label such as ‘thematic content analysis’ what was actually done, this lack of consensus also makes it impossible for an author who has transparently reported their analysis to defend against a detractor who argues, from a different definition of the method named, that their analysis is lacking some crucial dimension. The lack of consensus that characterizes methodological texts on qualitative data analysis brings us to our most basic recommendation: transparent report of qualitative analysis requires justification and detailed description of each of the analytic steps followed and the assessment of the appropriateness of such analysis must turn on examination of analytic steps in context and not the label assumed.
3.2 Content Analysis
3.2.1 Introduction
Content analysis is a ‘technique for making inferences by systematically and objectively identifying special characteristics of messages’ (Holsti 1969, p. 608). As used in this study, content analysis aimed to examine textual data through the systematic application of pre-determined categorization codes and then determining frequencies of text fragments in each category (Silverman 2015).
Content analysis is typically used to answer questions of the form ‘who, what, when, where, how and how often’. All kinds of qualitative data can be subjected to content analysis: newspaper clippings, literary works, e-mails, pictures, audio clips, blogs, movies, scientific articles, answers to open questions in a survey, and, of course, also interview or focus groups discussion transcripts.
Coding within content analysis is done top-down, on the basis of a predefined protocol with a coding scheme, derived from the theoretical framework of the researcher as informed by a review of relevant literature. Initial coding schemes are often tested and amended based on their performance in a sample of the data.
Coding consistently imposes an operationalization of the conceptual framework of the researcher on the data in a manner that may be inconsistent with the framing used by research subjects. If repondents consistently use the same terms to describe analytically relevant concepts, automatic coding may be used. Compared to manual coding, automatic coding has three advantages. It allows for the coding of larger data sets, it increases completeness and it eliminates human error. Nonetheless, automatic coding requires careful consideration. For instance, the term ‘internet’ can appear in semantic units indicating a problem (lack of access) as well as a mode of–successful–communication.
In our example the research question for content analysis was: What are the characteristics of each respondent’s household and how do they communicate–with what tools, how often and how long–with their relatives back home?
3.2.2 Report of method
In order to determine household characteristics we used top-down content analysis. The coding frame used was based on an earlier scheme (Casimir and Tobi 2011) and extended with a list of ICT (Information Communication Technology) devices derived from Shadymanova’s interview guide. The coding scheme was segmented following the research questions: which people are part of the household, what is shared (resources, activities, expenditures), which ICT tools are used to communicate with the household, and how often are they used. The coding protocol was tested on two randomly chosen interviews, found inadequate and modified such that it adequately anticipated the full diversity of ICT tools used and household compositions. Manual coding was used rather than automatic as tests of automatic coding did not identify all relevant text strings and did not consistently associate appropriate codes with found text strings. For instance, the search string ‘internet access’ could indicate both the possibility of internet access and the absence of it.
3.2.3 Report of results
3.2.3.1 Coding
The coding phase resulted in Table 1, where the first two columns contain the coding scheme. The third column gives a summary of results.
Table 1
Results of content analysis, based upon coding scheme
Facets
Indicators
Results: number of interviewees indicating…
Household composition
Size
1 (single): 3
2 (with partner, no children): 1
3 (one child): 2
4 (two children): 2
5 (three children): 1
6 (three children+ maid/man): 2
Generations/extended family
6
Headship
single: 3
married: 6
with partner: 2
Sharing resources mentioned
Accommodation
4
Income
2
Durables
See ICT tools below
Non-durables (including ICT facilities)
See below
Other assets
Not asked
Sharing activities mentioned
Food preparation and eating
4
Care
1
Decision making
2
Sharing expenditures mentioned
Accommodation
5
Durables
Not asked
Non-Durables, including foods
Not asked
ICT tools
cell phone
11
Chat
11
Internet
10
Facebook
11
e-mail
10
instant messaging (Skype)
11
Twitter
4
Voice over IP (VoIP)
3
Frequency of communication1)
every day
7
twice or three times a week
5
once a week
4
once per month
2
Duration per time
10 min or less–cell phone or VOIP
8
half an hour–Skype or Facebook
5
one hour–Skype or Facebook
1
two hours or more–Skype or Facebook
1
1Numbers add up to more than 11, since interviewees communicated both with cell phone and Skype or Facebook; and they indicated frequencies for communication with partner, children and other relatives
3.2.3.2 Analysis
Analysis consisted of an overview of frequencies of the codes applied (column 3 of Table 1). Six of the interviewees had one or more children, five of the interviewees were single. Most frequently mentioned as shared within the household were: sharing a roof, sharing consumption (food), sharing income and expenditures.
To communicate with their household back home, all interviewees used a mobile phone, e-mail and instant messaging (Skype). Six of the eleven interviewees had contact with their household back home every day, two interviewees twice or three times per week, and three interviewees once a week. The remaining interviewee–who was single without children–had contact with her relatives once per month. Duration of communications varied from a few minutes to three hours or more. The latter only when Skype was available.
3.2.4 Discussion of content analysis and its reporting
Top-down content analysis provided a description of manifest features within the data identified by the researchers at the outset as relevant to their study. The method allowed the researchers to extend the initial coding scheme that appeared adequate with respect to the research question, such that it became adequate with respect to the data. Results however are limited to the deductively imposed framework used and will not report findings that call into question the appropriateness of that framework.
3.3 Metaphor analysis
3.3.1 Introduction
Metaphor analysis uses systematic examination of elicited or spontaneous metaphors to identify latent conceptualizations (Schmitt 2005). According to Coffey and Atkinson (1996) metaphors are grounded in socially shared knowledge: “Particular metaphors may help to identify cultural domains that are familiar to the members of a given culture or subculture; they express specific values, collective identities, shared knowledge, and common vocabularies” (Coffey and Atkinson 1996, p. 86). Metaphors require and reflect shared meanings. “In terms of data analysis (…) we can explore the intent (or function) of the metaphor, the cultural context of the metaphor, and the semantic mode of the metaphor” (Coffey and Atkinson 1996, p. 85). Metaphor analysis is well suited for questions such as: ‘how do people depict a situation?’, or: ‘how do they describe a process?’ Data could be any kind of text, talk or visual. Metaphor analysis is particularly relevant when research questions require researchers to identify and make explicit implicit aspects of data, for example, when communication is highly coded as is often found in exploration of sensitive topics.
A metaphor analysis involves identification, classification and inductive examination of metaphors to draw inferences regarding the structure and significance of the conceptual metaphors of which they are an instance (Low and Cameron 1999). As metaphors are manifested in ways that are context-dependent, their analysis often starts with bottom-up coding. Top-down coding for metaphors is only indicated when researchers have a specific interest in pre-determined forms of metaphors (e.g. path metaphors, battle metaphors, animal metaphors).
The research question for our example is: what implicit perceptions and/or feelings do respondents have with respect to their households and their travel from that household, expressed through flowery language.
3.3.2 Report of method
Following Coffey and Atkinson (1996), we started metaphor analysis with building a protocol that would allow multiple researchers reliably to identify instances of flowery language. We tested this protocol and then coded the text for instances of flowery language. Once so identified, we then examined text coded as flowery in detail for instances of metaphors. For this coding we operationalized ‘metaphor’ as any instance where a term used can also be used in a different context and where hearers’ knowledge of that use in that different context alters their interpretation. Once all instances of flowery language were scrutinized for metaphors, we identified the source context for metaphorical terms, detailed the additional meanings that may be implied through use of that term, and then selected from those possibilities the one that was most probable.
3.3.3 Report of results
3.3.3.1 Coding
Coding identified metaphors in more than half of the transcripts. Respondents used non-literal descriptions when discussing topics that may involve emotion, such as distance (e.g. ‘another planet’), connection (e.g. ‘blood ties’) and surprise or impact (e.g. ‘shock’). The complete list of metaphors found is presented in Table 2.
Table 2
List of all metaphors found
Metaphors; metaphorical expressions in italic
How the thing is going; how things are going
It is the meaning of life
They have a network
Sometimes I felt like forcing the people
And you develop some strategy for that
In terms of I am not cut off
You do not feel free sometimes with other people
How the person will accept or will be open to discuss to you
But I do not want to say that Dutch people are not open because I experienced other situations where they were even more open than I expected
But also the old people in general they feel they show more open than the young people who never go to other countries, so this kind of things sometimes shock
You do not like to be hurt
You will not see me on Facebook very often
So it is like calling from another planet, the calling from here
And I stayed with my daughter until my husband came
(she was in the Netherlands, and her daughter in Ghana)
So I told her Big Brother will be watching you
I am into the domestic stuff
He is my path
At first when I just arrived I was shocked
I recall the cultural shock
3.3.3.2 Analysis
On review, we decided that it was inappropriate to undertake analysis beyond the identification of potential metaphors. Each of our respondents came from a distinct cultural context so each could be expected to have their own distinct repertoire of metaphors. The narratives examined did not arise in natural conversation within their context but in interaction with an interviewer who comes from a different context, so it is not clear that respondents would have drawn on the repertoire found in their home context. As metaphor use is tied to both language and context, and interviews were held in a foreign country in both respondents’ and interviewer’s second language, we could not identify possible, let alone the most probable, meaning.
3.3.4 Discussion of metaphor analysis and its reporting
Metaphor analysis is useful when research questions require interpretation of a narrative that goes beyond the strictly literal meaning of terms. Since we did not have enough information at our disposal–as explained in Sect. 4.1., we cannot discuss results or come to conclusions.
3.4 Domain analysis
3.4.1 Introduction
Domain analysis was created by ethnographers to help them understand how the communities they were studying structured their world. “Domain analysis involves a search for the larger units of cultural knowledge called domains (…). In doing this kind of analysis we will search for cultural symbols which are included in larger categories (domains) by virtue of some similarity.” (Spradley 1979, p. 94). Spradley distinguished four elements in the domain structure. The first is the so-called folk terms that informants use. These terms have semantic relationships–the second element–with ‘cover terms’, a name for a category of cultural knowledge, which are the third feature of domains. Finally, every domain has a boundary: informants know what is part of the domain and what is not (Carballo-Cárdenas Mol and Tobi 2013).
Codes are derived from the ‘folk’ terms used by the respondents in interviews (Borgatti and Halgin 1999) using in-vivo coding. “The systematic use of in-vivo codes can be used to develop a ‘bottom-up’ approach to the derivation of categories from the content of the data” (Coffey and Atkinson 1996, p. 32). Data can be any kind of text or script, both naturally occurring and elicited text, talk and visuals.
The research question in our example was: How do respondents talk about their ‘household’ and their communication with home?
3.4.2 Report of method
Domain analysis was created in order to allow researchers to describe how respondents structure their worlds on their terms. Following Coffey and Atkinson (1996) our first step was to code the ‘folk terms’ with which the interviewees expressed their ideas about their household and the communication with that household. The second step was to identify those words or expressions that clearly indicated distinct domains (cover terms). Cover terms were inductively identified through clustering of folk terms. Clustering was indicated, in the first instance, by proximity between terms. For example, ‘sharing’ is subdivided into sharing a roof, sharing expenditures, etc. Once proximity associations were exhausted, terms were then clustered using refutational and then confirmatory arguments. For example, the term ‘meaning of life’ was tested against ‘sharing’ and ‘significance of household’ and found to fit least poorly and acceptably with ‘significance of household.’ Our last step was then to identify how other descriptive terms related to the cover terms just identified. Examples of such semantic relations are membership, causation or sequence.
3.4.3 Report of results
3.4.3.1 Coding
When talking about their household, interviewees referred to the sharing of several aspects–sharing a roof, food, income, expenditures, time, household organization–and to the significance of households, including feelings of connectedness and emotional aspects. Terms used were, for instance, “the people who live together”, “who share food”, “who live under the same roof”; and: “it is an emotion”, “it is protecting”, “it is important”, “it is the ‘holy thing’”. Or: “the meaning of life”. Also mentioned: “It is the place where I feel important and valuable”.
When talking about communication, the interviewees referred to the content of communications. The content of communication ranged from sharing practical information–some repair that has to be done, financial issues–to conversations about how their beloved ones are doing, in particular: how children are doing in school. Also, medical problems with children or parents were discussed, or business shared with relatives. Sometimes no specific topic was mentioned, but the interviewees indicated that they wanted to communicate with home because they felt lonely, or because they wanted to hear their mother’s voice, since he or she missed her.
Interviewees did also talk about their communication style. One of the interviewees indicated that ICT creates circumstances for communication which may ask for a different style: “You have to be more kind, support them. When I am there, I tend to be more rigorous” (P4).
3.4.3.2 Analysis
In the analysis phase, cover terms were defined and the folk terms were related with semantic relationships to these cover terms. In some cases, the cover terms were divided into sub-terms, for instance ‘sharing’ is subdivided into sharing a roof, sharing expenditures, etc. Figure 1 presents the result of both the coding phase and the analysis phase for the domain ‘Household’. In Fig. 2, the content types (cover terms) and expressions (folk terms) of the domain ‘Communication’ are shown.
Fig. 1
Domain household with folk terms (uncolored boxes), semantic relationships (labels on the arrows) and cover terms (filled boxes, with initial capitals)
Fig. 2
Domain communication, in particular content of communication
×
×
3.4.4 Discussion of domain analysis and its reporting
Domain analysis allowed us to answer questions about how respondents structured their world. It was possible efficiently and reliably to identify folk terms through in-vivo coding. Decisions on categorization of folk terms and semantic relationships between folk terms and cover terms required subjective judgement that would be difficult to reproduce, but was quite easy to transparently document. For example, rather than proceed with the classification structure just presented, we could have opted for subcategories such as practical reasons (sharing information, asking where people are), communication for its own sake (when feeling lonely, for instance) and other reasons for communication. Transparent presentation of these subjective judgements should be reported as an annex within or as supplemental material accompanying a standard-length journal article.
3.5 Membership Categorization Analysis
3.5.1 Introduction
Membership categorization analysis (MCA), introduced by Sacks in 1972, identifies the categories interviewees use to classify people and how these categories are routinely attached to particular kinds of attributes and activities (Silverman 2015). The rules applied to attribute individuals to categories are called Membership Categorization Devices, MCDs (Schegloff 2007). MCA does not ask people how they categorize, but investigates how people “use social categories to account for, explain, justify and make sense of people’s actions” (Fitzgerald and Housley 2015, p. 6). Each set of categories is a collection and categories may belong to more than one collection: professor is part of the collection students/administrators/staff, i.e. university community, but also part of the occupational collection plumber/doctor/secretary/undertaker (Schegloff 2007).
In the actual execution of MCA, the fundamental goal is to identify how respondents order categories and their attributes in their social world. Data analyzed through MCA may be any kind of text, talk or visuals. We used Membership Categorization Analysis to identify how respondents determined if a given individual was a member of their household.
3.5.2 Report of method
Membership categorization analysis was developed to identify from natural speech how subjects classify others and what characteristics are associated with those classes (Silverman 2015). In our analysis we used MCA to identify the rules by which our subjects classified others as member of their household: membership classification devices. Shadymanova elicited data appropriate for this analysis by asking respondents the following questions: “Could you tell me about your household?” and “What is a household for you?”. We identified each instance where an individual was classified as either a member or not a member of the household and then coded explanatory text. These fragments of explanatory text were then examined for justifications for the classification just given. Justifications were then clustered by similarity and each cluster of justification was then described as a membership classification device.
3.5.3 Report of results
3.5.3.1 Coding
Respondents appeared to use several terms when indicating whether individuals were members of their household or not. Interviewees used terms as ‘being family’, ‘having blood ties’. Several found sharing a roof, sharing income, or expenditures, sharing food or household chores justification to include people in the category ‘household’. In several cases, household membership was related to feeling responsible (“I am paying their school fees / their rent, because I am feeling responsible for them”). Also, emotional terms were used (“She feels like family”). Some of the interviewees realized that their cultural background influences their rules of inclusion:
If I consider household as composed of those who are close to me somehow, I will say that I have one wife; I have three kids; and I have a lady who helps us at home, who helps my wife. So that is my household, my small household. But in Africa, let’s say in my country, (…) [the] household is part of a larger household: (…) aunts, (…) one brother and some sisters. (P1)
One interviewee said: “household is the space you are sharing and supplying for daily needs” (P4). Since he was the one who paid for food and rent, he was part of that household, also when he was abroad at the moment. Sharing a roof was also for P3 a reason to include people; however, being absent was no reason to exclude people as a member of the household.
Several interviewees expressed awareness of the existence of different definitions of households by saying: my wife and my children constitute my nuclear household, but in our culture–or Africa, or in my country–we include siblings, aunts, grandparents. Also, nephews or nieces for whom they took care by paying rent or school fees could be included or excluded from the household, depending on the definition used.
The second column of Table 3 contains the terms used by the interviewees.
Table 3
Membership Categorization Devices (MCD) to include or exclude someone as household member
MCD
Terms used by interviewees
Remarks
Being family
having blood ties
married
descendants of grandfather
Sharing
a roof
Being present is a criterion to include, being absent not necessarily a criterion to exclude
food
income
expenditures
decision making
care taking
household chores
Emotional criteria
feeling responsible
Often related to one of the other criteria (f.i. sharing expenditures)
“she feels like family”
Cultural criteria
in our culture…
In particular used by interviewees who mentioned awareness of the existence of different definitions of household
in Africa
in my country
3.5.3.2 Analysis
After the coding was done, the rules of inclusion and exclusion used by the interviewees were classified into categories of Membership Categorization Devices, given in the first column of Table 3. The third column contains further remarks.
3.5.4 Discussion of MCA and its reporting
MCA produced a list of criteria (Membership Categorization Devices) that justify classification of individuals with respect to their membership in the category ‘household’. It gave the authors a new understanding of respondents’ public construction of their understanding of relationships. Significantly, interviewees’ notions of ‘household’ were mutually inconsistent and many interviewees used more than one device (e.g. blood and familiarity) when determining membership in their household.
4 Discussion
4.1 Comparison of the four methods modelled
In the previous section we modelled and discussed what is required to usefully report individual qualitative analysis methods within interdisciplinary studies. In this sub-section we step back and comparatively discuss the methods we modelled.
Only content analysis identified the variables of interest (i.e. specified codes) before starting coding and analysis. The other three approaches identified variables inductively through ‘bottom-up coding’, either using the terms used by the interviewees (e.g. the ‘folk terms’ in the domain analysis, and the flowery language of interviewees in the metaphor analysis), or identifying rules of inclusion and exclusion in the membership categorization analysis. In those three methods, once variables were identified, they were converted into an analytic framework that was deductively applied to the remaining texts and checked in an iterative cycle with the texts already coded.
Content analysis, as conducted here, was counting the number of times the terms determined to be relevant appeared in the text. Although not presented here, analysis may be continued through the use of theoretically motivated descriptive and correlational statistics which would be then reported according to the norms governing reporting of quantitative analysis. We were able to report this analysis method transparently because its operation relied on deductive application of a clearly declared coding scheme. In our example, the content analysis gave information on household composition, what households shared, ICT-tools used, and frequency and duration of communication with those tools.
Domain analysis, as used in this example, permitted us to work from manifest features of the transcripts to identify cognitive structures used by respondents in the interview. Domain analysis may be used on a substantial set of interviews, with special attention to the presence or absence of subgroups’ use of folk terms. For the purposes of simplicity, in this example we did not examine the interaction between domain and metaphor analysis. In practice, the folk-terms identified as key within a domain analysis may be themselves or may be closely associated with metaphors. In these cases, the additional layers of meaning associated with terms by metaphor analysis must be carried forward through the domain analysis as the often-normative shadings that come with metaphors may be analytically relevant.
Membership Categorization Analysis added to the domain analysis by identifying who was a member of the household and who not, while in the domain analysis the emphasis was on the significance of the household, without taking into account who were part of it and what made them part. MCA, like domain analysis, may be used in analysis of substantially sized data sets, although both seem less suitable for large data sets than content analysis.
We were not able to produce an adequate metaphor analysis. Like domain analysis and MCA, metaphor analysis requires repeated close reading of transcripts. As the metaphors studied are found precisely at the intersection of language and culture as they collide in an interview setting that was, in this case, foreign to both, we could not apply thematic codes with the sort of confidence possible with content analysis. While we were able to identify that there was a metaphor, we could not produce an unambiguous description of the range of possible associated meanings nor could we reliably describe the rules that govern association of these meanings with the flowery language we identified. Were we reporting a metaphor analysis in a context where no part of the research aspired to be a valid account, as is appropriate in some exploratory studies or in those where the assumptions required for valid descriptions are not met, we may have chosen to proceed by associating our own meanings with identified metaphors. In the context of a project that is both inter-disciplinary and mixed-methods, however, it is not appropriate for researchers to silently include speculation as data. When there is reason to believe that the words used by respondents have connotations that are analytically relevant, it is certainly appropriate to recognize those connotations. Identification of these connotations, however, would require explicit design of a transparently reported distinct research effort to develop a formal ruleset for the identification and interpretation of the analytically relevant metaphors found in the narratives examined.
In our experience, and as suggested in the discussion of domain analysis given above, it is rarely possible to answer a socially relevant research question through use of a single method of qualitative analysis. Contrary to what we have observed to be common practice, it is not appropriate to report, for example, only that a ‘frame analysis’ was undertaken where that term describes interactive application of several constitutive methods. Each of these constitutive methods, and the means by which the data arising therefrom are combined, should be described separately. The level of detail required to support this sort of description may very well not fit either the norms or the space afforded in current publication fora. In those cases, the reporting of qualitative analysis useful for interdisciplinary teams will require publication of supplemental material.
4.2 Coding and analysis
Coding requires segmentation of a narrative into units of meaning that are hopefully compatible with the conceptual framework within which the research questions were formulated and appropriate for the sort of analysis required to answer that question. When reporting qualitative analysis of narrative data for interdisciplinary teams, this segmentation and then the association of these segments with codes should not be presented as analysis as these two steps most closely approximate the work done by a respondent when she provides a value in response to a structured survey item or when a researcher records the value displayed on an instrument. With this in mind, a transparent discussion of coding is not an adequate report of qualitative analysis. Once a narrative is segmented in a manner that fits the researcher’s conceptual framework, the texts so coded are data appropriate for analysis. How the narrative fragments are interpreted once coded is determined by the nature of the data analysis method chosen. For example, within MCA text coded as ‘category bound activity’ is interpreted and used quite differently than text coded as ‘path metaphor’ within a metaphor analysis. In the absence of a well established shared lexicon, the mechanisms and content of the interpretations made through analysis should be reported in detail. In order to be interpretable by inter-disciplinary teams, it may be better to report coding as ‘data processing’ and the manipulation and interpretation of the coded narrative fragments as ‘data analysis.’
The results of content analysis, domain analysis and MCA may usefully be presented in the form of a table or graph and in this article we showed examples of both. The graphs were produced within a qualitative data analysis program, in this case Atlas.ti. Presenting results in a way that does not solely rely on ‘typical’ quotes is recommended. When quotes are used, the justification for their selection, as well considered in discussions around annotation for transparent inquiry, must be reported (https://qdr.syr.edu/ati).
4.3 Transparency and appropriateness
It may not be possible to transparently report qualitative analysis of subject response data but this should not encourage use of transparent but inappropriate methods. While we strongly encourage explicit coding in order to improve transparency, we recognize that even with the systematic approaches we took, transparency in reporting analogous to that found in purely quantitative interdisciplinary studies, is not always possible. In several instances a given semantic unit could reasonably be recognized by two or more codes that the scheme used presented as mutually exclusive and we were unable to complete a metaphor analysis though subjective attribution of meaning by researchers may be necessary. In keeping with the principles of annotation for transparent inquiry (https://qdr.syr.edu/ati), when only one reading is carried forward, such decisions should be transparently documented through applying all possible codes within the analysis software used and then using comments to provide discussion supporting the decisions taken.
4.4 Compatibility with contributions from the natural and life sciences
Qualitative analysis of subject response data within interdisciplinary studies is, appropriately, reductive. Some authors, for instance St. Pierre and Jackson (2014) argue that coding ought to be avoided entirely. They state that lecturers “teach analysis as coding because it is teachable” (St. Pierre & Jackson, 2014, p. 715) and reject the many textbooks and university research courses which, according to them, support the positivist, quasi-statistic analytic practice, reducing words to numbers. We agree that coding is a reductive exercise and that coding and analysis can be distinguished. We, however, think this critique is not relevant as it makes epistemic assumptions that are not appropriate for inter-disciplinary mixed-methods research. Research on environmental challenges, for example, is funded to inform practice and the measure of this research, ultimately, is predictive validity. For this, researchers must assume that the world described is somewhat stable, that descriptions thereof will converge, that the data they gather represents something more than instrument effects and that it is possible to reduce the complexity of the world sufficiently to render a useful representation. If the purpose of qualitative inquiry within interdisciplinary efforts is to complement and extend quantitative findings, it is appropriate to adopt a compatible stance. The assumptions necessary to support such reductive analysis, as long discussed (e.g. Bergdahl 2019; Shankman et al. 1984) may not hold in some circumstances and naïve combination of fundamentally different data does gross disservice to both.
5 Conclusion
In this paper we first demonstrated that the forms of reporting qualitative analysis in interdisciplinary research often do not provide readers with sufficiently detailed accounts of qualitative analysis. Secondly, to mitigate this problem we presented reporting models for four methods of analysis selected for their relevance to interdisciplinary research addressing environmental challenges. Qualitative analysis of narrative subject response data requires a high level of detail in reporting. Clear separation and transparent accounts of both coding and analysis are crucial for qualitative contributions to interdisciplinary mixed-methods research.
Further literature reviews on qualitative contributions to interdisciplinary research is needed to get a better understanding of the strengths and weaknesses of methods used and reported. By definition, literature reviews rely on publications, whether grey, white or peer-reviewed. Therefore, we think our contribution to the reporting of qualitative contributions to interdisciplinary research is essential to both the interdisciplinary readership for transparency and replication purposes, and the study of qualitative methods within interdisciplinary research.
Acknowledgements
We thank Jarkyn Shadymanova for making available her interviews for this research.
Declarations
Conflict of interest
The authors declare that they have no competing interest.
Open AccessThis article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Appendix 1: demographic description of interviewees
A basic demographic description of the interviewees is provided in Table
Table 4
Demographics on interviewees and order of analysis
Content analysis
Metaphor analysis
Domain analysis
Membership categorization
P 1: Male (35–39), Benin
1
2
3
4
P 2: Male (35–39), South Africa
1
4
3
2
P 3: Female (40–44), Kenya
2
4
3
1
P 4: Male (40–44), Benin
1
3
4
2
P 5: Male (40–44), Ghana
3
4
2
1
P 6: Male (30–34), Tanzania
4
2
1
3
P 7: Male (35–39), Cote d'Ivoire
2
3
1
4
P 8: Male (30–34), Tanzania
4
1
3
2
P 9: Male (30–34), Rwanda
3
4
1
2
P10: Male (45–49), Ghana
2
4
1
3
P11: Female (40–44), Ghana
3
2
1
4
4. The order of the data analysis methods was randomized and differed across interviewees to reduce order effects. For the analysis, we used Atlas.ti, Computer-Aided Qualitative Data Analysis Software, version 7.5.6—18.
