Tensions in Representing Behavioral Data in an Electronic Health Record

Health records come in many different forms. More formal conceptualizations such as medical charts or electronic health records help to standardize practices of health care teams, while informal practices of jotting down notes on paper or exchanging information support the ad hoc aspects of team work. Fitzpatrick (2004) uses the term ‘working record’ to differentiate from a passive information repository. A health record ‘at work’ takes various forms because it is maintained and evolves through practical aspects of delivering health services. For example, components of the working record can take varying forms when authored by different members of the team, and for different intended audiences, lifespans, locations, or purposes (Fitzpatrick 2004). Health records can enforce standardization of practices, but they can also be designed to provide flexibility and adaptivity for local and immediate needs of teams (Fitzpatrick 2004; Berg 2004).

One practical local need is managing subjective data, which are not easily standardized. Subjective aspects of patient care and coordination are just as important as objective ones (Erikson 1958), but they are difficult to represent in the record (Mentis 2010). To explore these aspects, we focus on behavioral health, describing local needs and demands through fieldwork studying care teams supporting children with behavioral challenges. The individualized nature of these services involve a subjective process of identifying behaviors of concern, then operationalizing, measuring, and monitoring them (Marcu et al. 2013, 2016). Children’s behaviors are recorded manually based on direct observation, and monitored to determine the effectiveness of interventions.

Over two and a half years, we studied this process among care teams by examining their evolving use of the working record in practice. Our focus on the role of records among teams providing psychiatric care for children represents an understudied context. Building on limited work in this area, we examine the role of schools within the ecosystem supporting children’s behavioral health. Complementing Saario et al. (2012), who studied coordination across organizations, we focus on the intraorganizational practices of record keeping by school-based teams. These care teams monitor and address behaviors that affect a child’s ability to learn and develop interpersonal relationships. Early intervention is most effective, and schools play an important role in prevention and intervention (Greenwood et al. 2008). Under U.S. law, schools are required to identify needs and provide services for children that can be classified as having behavioral disabilities (Yell et al. 2018). Children with autism spectrum disorders, conduct disorders, anxiety disorders, neurological impairment, emotional impairment, and other types of diagnoses can exhibit disruptive behaviors that may lead to services within a school setting. School-based behavioral health teams include such professions as special educators, school psychologists, psychiatrists, behavior analysts, therapists, and social workers.

Behavioral health services for children have a rich set of dependencies across stakeholders and settings in a child’s daily life—especially between healthcare and education—which has made them a focus of sociotechnical research (Amir et al. 2015; Hayes et al. 2010; Marcu et al. 2014). Care coordination is complex because of the necessity to navigate across different systems of care (Council on Children with Disabilities 1999). Coordination is ‘the process of managing dependencies among activities,’ and what it looks like varies across settings (Malone and Crowston 1994). Behavioral health involves what Kaziunas et al. (2019) call ecologies of care, across the formal healthcare system and community services such as those provided by faith-based groups. Their ethnographic account describes how coordination across an ecology of care could help individuals manage their health, but instead there tends to be fragmentation. Behavioral health is additionally dependent on managing one’s health at home (Barbarin et al. 2015), and this is particularly important for children and others who have caregivers playing an active role in their care. Children’s behavioral health involves a range of stakeholders, from parents and other caregivers in the home, to school staff responsible for their care during the day, and behavioral specialists in other community and clinical settings. When children exhibit behaviors that are concerning, these stakeholders must coordinate across organizational and professional boundaries. Coordination is so interdependent and challenging that there can be tensions and even breakdowns, as our prior work showed at the boundary between home and school (Marcu et al. 2019).

In this paper, our focus on the role of behavioral records for coordination shows how those who were responsible for creating and maintaining records for behavioral monitoring struggled with the local demands of this work, and wanted technological supports. We took an action research approach to investigate their paper-based record keeping practices, design a novel electronic health record, and conduct a deployment study of our resulting prototype. Throughout these stages of our research, we uncovered tensions at the team and organizational levels around how behavioral data should be collected, represented, and used. Accordingly, this paper addresses the following research question: What are the tensions involved in representing behavioral data in an electronic health record, for use in care coordination? We contribute rich descriptions of practices with behavioral data, and a study of technology-in-use, explaining how tensions in representing behavioral data in an electronic health record have to do with balancing intra- and inter- organizational information needs. Our study of these teams addresses the paucity of literature focused on allied health professionals (Fitzpatrick and Ellingsen 2013).

Figure 1 illustrates the methods used in this study, which unfolded over three phases. In the following section, we describe the setting in which this study took place. Then, we describe how formative fieldwork began and continued throughout the study with ongoing data analysis, as we designed a prototype and studied its use in a deployment.

Records served to document instances of behavior throughout the school day, using the frequency or duration of a certain type of behavior. Record keeping practices were integrated into the work of care teams because capturing naturalistic behavior was important for measuring and monitoring a child’s progress. For example, if the care team was helping a child manage his anger, they would review records over time to look for related changes in behavior, such as more instances of describing his emotions, more instances of asking for a break when frustrated, fewer uses of inappropriate language, or less time needed for the child to calm down during an emotional outburst.

The challenges of representing behavior in the record included applying standardization and quantification to subjective aspects of a person’s health; and the need to observe naturalistic behavior, which required mobility and could lead to a significant amount of data transfer. Below, we outline key practices and artifacts of record keeping, and later we describe their role in the tensions we identified.

Certain members of the care team carried the record keeping responsibilities, because this required direct observation of behavior. The rest of the care team was involved in some way with ongoing treatment decisions, for which they made use of the behavioral data in assessing the effects of interventions on behaviors. Behavioral goals were set to help monitor progress over time and also report to those outside of the organization, including parents and auditors. Interactions across care team member roles occurred through (or sometimes around) the record, and often tensions resulted from the ways records were created, shared, and used in care coordination.

Our formative fieldwork gave us an understanding of the challenges involved in representing behavioral data in the working record. A prototype computerized working record, Lilypad, was also a product of our formative fieldwork and action research in partnership with three care teams.

Insights from the deployment study helped us understand how the transition to use of Lilypad began to meet the needs of record keepers who had been looking for technological support—while also highlighting the tensions that persisted when it came to capturing behavioral data within records. We discuss how these tensions appeared before and after the deployment. We also reflect on the design decisions behind Lilypad and to what extent they affected how care teams used behavioral data in coordinating care.

Taking an action research approach, we examined how behavior is represented in the health record. We focused on the challenges of record keeping during observation of behavior, leading to the design and deployment of our Lilypad prototype. Investigating the transition from a paper-based record to Lilypad, and studying technology in use, we found persistent tensions surrounding representation and use of data in the record. We identified tensions related to the need to structure work with and around the record, while maintaining the flexibility and unstructured aspects of work that are key in behavioral health.

Bowers et al. discuss structured and unstructured work as interrelated, underscoring ‘the multiple considerations which impinge upon the acceptability of technology in actual contexts–considerations which often require very difficult trade-offs’ (Bowers et al. 1995). They describe these trade-offs based on how technologies help to organize workflow from within the organization and workflow from without the organization. Although we did not examine workflow, our study similarly explored intraorganizational and interorganizational work that has both structured and unstructured aspects. The record keeping practices we observed with behavioral data are largely not able to be standardized or sequential, and therefore do not fit definitions of workflow (Bowers et al. 1995; Dourish 2001). But we find that Bowers et al.’s discussion of tradeoffs aligns with the tensions that we identified. We therefore adapt their intra- and interorganizational distinction in order to explain tensions between the creation of records from within and the creation of records from without.

Records from within involved care team members reflecting on a child’s behaviors individually and collectively in order to make adjustments to care based on the child’s progress on behavioral goals. Often, however, these adjustments could happen through anecdotal reports based on trust in other team members’ observations and professional judgement. As a result, creating records from within was less focused on capturing useful behavioral data, even though the records were generally valued and many processes revolved around the records. On the other hand, records from without were driven by accountability to stakeholders outside of the organization, related to billing, insurance, auditing, as well as placement decisions that balanced financial considerations with the individual needs of each child. Records from without were therefore seen by care teams to be more consequential, not only for financial accounting of work, but also because external members of the care team were much less familiar with the work of the organization and the behaviors and needs of each child.

For the purposes of reflecting on a child’s progress intraorganizationally in the context of their program, care teams preferred leveraging the knowledge and expertise of those members of the team with most proximity to the child. Other work has shown that early childhood interventionists don’t feel they need to collect behavioral data for their own understanding, because they know the child well (Sandall et al. 2004). Our study shows this trust of observation, memory, and professional judgement extends to others on one’s care team. That is, care team members may prefer secondhand accounts and interpretation of a child’s behavioral progress over data. Meanwhile, even though care teams are concerned with interorganizational perceptions of what goes on in their program, creating records from without is so different to them from creating records from within, that the two become difficult to reconcile. Records from within tended to be created and used used more organically as part of reflection and other unstructured team work. Records from without were structured by reporting requirements set externally, and standardized practices and tools set internally by administrators and supervisors.

Arguably, designing and deploying the Lilypad system together could have been a potential opportunity to reconcile records from within and records from without. However, we prioritized ease of use and adoption for those team members most central to record keeping practices, and did not actively work to redesign the practices themselves. We knew that the organization had made previous attempts to transition away from paper-based records, unsuccessfully, and some on their staff were wary of another attempt. Focusing on feasibility, we also knew that existing reporting software was dictated from without, for example through a client management system for billing to insurance. The long-term vision was thus for Lilypad to eventually become interoperable with such systems, meeting local needs for record keeping from within first, and later facilitating workflows for reporting interorganizationally.

With increasing digitization of healthcare work, the difficulty of incorporating novel records such as Lilypad or even a basic EHR (Kaziunas et al. 2019) in behavioral health can be explained by the challenges of representing behavioral data, ‘since data work often involves interpretation and assessments based on sense-making’ (Bossen et al. 2019). There is a significant need to rely on a multidisciplinary care team for interpreting an individual’s behavior and negotiating action with and around data in the record. We contrast the nature of this coordination with medicine, which uses data in the record at a larger scale for measuring organizational performance or even comparing services nationally (Pine and Mazmanian 2015). Behavioral data, on the other hand, is more suited to measuring individual treatment progress, and even representing that progress effectively can be difficult.

Medicine also draws from a single (or main) community of practice in determining how data will be used for accountability and decision-making (Pine and Mazmanian 2015). With regard to accountability, behavioral data are used to subjectively describe the nature of an individual’s behavior so as to justify the need for treatment, given that objective measures of physical or biological symptoms are often not possible. Accountability has more to do with documenting a need and rationale for why services are billed, as opposed to performance measures or protocols, which are difficult to define. Even when a behavior can be quantified, this requires a process of operationalizing the behavior of interest in a way that is difficult to standardize. Moreover, as multiple disciplines collaborate on a care team, each of them brings a different set of standards and each comes from a community of practice that may record and use behavioral data differently. The role of the record in coordination is therefore to enable ongoing interpretation and negotiation with one individual’s behavioral data across communities of practice.

Dourish suggests that workflow technologies could serve to visualize work, whereby ‘the role of the system is to find and present the order in the work, rather than, as traditionally conceived, to prespecify and enforce it’ (Dourish 2001). Standardization and structure are key challenges with representing behavioral data, which is why natural language processing is one approach that has been applied with unstructured clinical notes in electronic health records, though it still carries significant limitations (Townsend 2013; Perera et al. 2013; Xu et al. 2007). To find and present order in the work through visualization, we could learn from how Lilypad was appropriated for these purposes. For example, when a care team used their Apple TV to project the Lilypad interface on a wall-mounted display in the meeting room, even without a dedicated visualization, a mental health therapist with little knowledge of the child’s case was able to review recent behavioral data and reflect on what it might mean about the child’s progress. This required a simple chronological log of the child’s behaviors, pairing frequency and duration data with qualitative notes. Rather than sophisticated analytics or support for managing a large amount of data, Lilypad users wanted real-time access to review details of a child’s behaviors in order to do their work. It may be that adjusting to an electronic health record over time would lead to use of more complex visualizations, but meeting these more straightforward data needs first enabled adoption and supported the collaborative work of the care teams.

In this three-phase study, we took an action research approach to understanding challenges with paper-based record keeping practices with behavioral data, then designing a novel electronic health record, and finally studying its use in a 24-week deployment. Our Lilypad prototype was designed to support the capture of behavioral incidents with three forms of data: frequency, duration, and qualitative notes. We have described how care teams struggle to distribute responsibility for managing behavioral data in the working record, with some members of the team more burdened and some more dependent. We identified five persistent tensions in representing behavioral data, which reveal the difficulty of manually capturing behavior in the record due to not only the nature of data that cannot be standardized, but also the ways that intraorganizationl use of data can conflict with interorganizational use of data. Our deployment succeeded in facilitating transition from a paper-based to electronic health record, but in many ways did not change these tensions or always help care teams use data differently. Addressing these tensions requires understanding how data are captured and used from various perspectives across the care team, and rearranging workflows with data to provide more opportunities for interpretation and negotiation.