State Mental Health Authority Level of Involvement in Coordinated Specialty Care Clinics and Client Outcomes

Coordinated specialty care (CSC) is a recovery-oriented, team-based treatment model that provides evidence-based services for people with first-episode psychosis (1–7). CSC seeks to intervene early and improve symptoms, social functioning, and quality of life (4, 5). In the United States, most CSC programs receive some funding through the Community Mental Health Services Block Grant (MHBG) set-aside program for first-episode psychosis (1, 8). The state mental health authority (SMHA) in each state or territory is responsible for administering the MHBG set-aside funds to CSC programs. SMHAs are state governmental agencies designed to ensure the delivery of services to individuals with mental health conditions (9). Typically, SMHAs coordinate and directly operate mental health services (e.g., direct psychiatric treatment and supports for housing, employment, and education) and allocate funds to community providers for mental health services not provided by the SMHA. SMHAs vary widely in how they are organized, staffed, and managed within state governments (8–10).

Since 2014, SMHAs have administered the MHBG set-aside funds for first-episode psychosis (8, 11). In fiscal years 2014 and 2015, Congress directed each state to set aside 5% of its MHBG dollars to support evidence-based programs for individuals with serious mental illness, including psychotic disorders. The Substance Abuse and Mental Health Services Administration (SAMHSA), which administers the MHBG, provided guidance to states to encourage a focus on effective treatment for early psychosis, with particular emphasis on CSC (1). In 2016–2020, SAMHSA directed states to increase their investment in CSC by setting aside 10% of their MHBG funds. The law that funds the MHBG was amended to provide states with a supplement to cover the cost of the required 10% set-aside (9).

SMHAs take a variety of approaches to administering the MHBG set-aside funds. For example, SMHAs in Maryland, New York, Ohio, and Oregon work closely with academic institutions to implement CSC programs (8, 12). In contrast, other states contract with third parties to manage the distribution of funding and oversight (13). In 2015, a study of 12 state administrators of the MHBG set-aside funds found differences in how involved and prescriptive SMHAs were in determining the CSC model selected, the programs’ target populations, training needs, and use of funds (10); however, strong state guidance remained one of the facilitating factors for implementing CSC via use of the set-aside funds (10).

The literature on mental health systems change documents the importance of SMHA involvement for implementing evidence-based practices (EBPs) (14–19). When SMHAs administer funding but have little involvement in the implementation process, broad adoption of the practice is often limited (16–19). When SMHAs do not actively promote or engage in decision making around implementing EBPs, recruiting participating clinics is often more difficult and clinic staff turnover is higher (20). However, little is known about how state-level involvement is related to individual client outcomes. Specifically for CSC, it remains unclear whether the level of SMHA involvement with the set-aside–funded CSC programs predicts the success of these programs or plays a role in the lives of clients served.

States received funding increases for CSC in 2021 and 2022 through the COVID-19 Relief Supplement Allotment (21) and American Rescue Plan Supplement (22). With this increased investment in CSC, it is important to examine whether SMHA involvement plays a role in CSC program effectiveness. The purpose of this study was to investigate whether a relationship exists between the level of SMHA involvement with CSC programs and clinical outcomes of clients served within these programs.

Methods

Sample

We collected data from 34 CSC programs included in a larger study examining the use of MHBG set-aside funds for early psychosis care. These CSC programs were selected from a pool of 244 CSC programs identified by SMHAs in their 2017 federal reporting on the distribution of MHBG set-aside funds. From that pool, 10 senior researchers selected 40 potential CSC programs that were actively providing CSC services and that represented diversity in geographic distribution, had a CSC program model (e.g., OnTrackNY, EASA, NAVIGATE), received technical assistance at program start-up, and had an urban-rural mix. Study team members contacted these 40 programs, 34 of which agreed to participate in the study. The 34 study sites were located across 21 U.S. states and one U.S. territory (see Figure 1), and SMHAs of these states and territory were included in the study. All aspects of this study were reviewed and approved by the Westat Research Ethics Board.

Data Collection

Client outcomes.

Staff members at the 34 study sites collected outcome data on clients receiving CSC services on entry into the program and every 6 months, for up to 18 months. Outcome data collection ran from January 2018 to July 2019. Demographic and background information included age, gender, race-ethnicity, income, duration of untreated psychosis before the person entered the program, and start of CSC program enrollment for each client. Sites submitted deidentified data through a secure online portal.

Outcome measures included the Modified Colorado Symptom Index (CSI) (23, 24), a 14-item client self-report scale rating the frequency of experiencing psychiatric symptoms in the past month from 0, not at all, to 4, at least every day, with good internal consistency (0.92) and test-retest reliability (0.71) (23); the Terrible to Delighted Quality of Life Scale (25), a one-item self-report rating of how clients feel about their life generally on a scale ranging from 1, terrible, to 10, delighted; Global Functioning: Social Scale (26, 27), a one-item clinician-completed measure regarding a client’s social contact and interactions in the past month ranked on a scale ranging from 1, extreme social isolation, to 10, superior social and interpersonal functioning; and the Global Functioning: Role Scale (26, 27), a one-item clinician-completed measure regarding a client’s functioning in occupational, educational, or homemaker roles in the past month, scored on a scale ranging from 1, extreme role dysfunction, to 10, superior role functioning. The functioning scales have good interrater reliability (intraclass correlation coefficients of 0.85–0.95) and evidence of construct and discriminant validity (26).

SMHA staff interviews.

Between November 2018 and May 2019, senior members of the study team (S.S.G. and a nonauthor team member) conducted 1-hour, semistructured telephone interviews with the 22 SMHAs responsible for administering MHBG set-aside funding to the 34 study sites. One to five SMHA staff members selected by the SMHA participated in the interviews. After receiving a description of the study, respondents gave verbal informed consent for participation. Discussion included the process for administering MHBG set-aside funds, selection of CSC programs, and SMHA involvement with programs. Interviews were not limited to the role of the SMHA with the 34 study sites and included discussion of SMHA involvement with all MHBG-funded CSC programs within the state.

To develop an indicator of SMHA involvement with CSC programs, we used data from SMHA responses to five questions selected to reflect two main dimensions of involvement: level of ongoing communication with CSC programs and degree of state-level control regarding the way the programs operate, including hiring decisions, CSC program model selection, and admission criteria. We selected these items on the basis of existing work (16, 17) and consultation with experts who work with both the SMHAs and the CSC programs. An important factor for item selection was that federal guidance to states on CSC implementation did not include specific recommendations regarding these areas (10, 28). As such, these five questions revealed variability among SMHAs (see Results). For each “yes” response, the SMHA received 1 point, for a maximum of 5 points. Possible scores ranged from 1 to 5, with higher scores indicating a higher degree of involvement with CSC programs. A senior member of the study team (T.C.D.), who did not participate in the telephone interviews, rated the SMHA responses.

Analyses

We used multilevel random-effect modeling to examine whether the level of SMHA involvement was associated with client outcomes. For each of the outcome measures, only clients who completed a baseline assessment and at least one follow-up assessment were included in the analyses (N=362–435 clients, depending on the measure). To account for the potential effect of attrition from the two groups of study participants (those with only baseline assessment vs. those with follow-up data), a propensity score (PS) approach (29) was used to create a response PS for each participant. We estimated the probability of being in the group with follow-up data on the basis of the available participant-level and site-level core covariates. These covariates included length of time between interviews, gender, race-ethnicity, income, age, duration of untreated psychosis, program fidelity score (30), staff turnover rate, urbanicity and poverty level of the service area, number of participants within sites, and the site-level compositions of participants in terms of race-ethnicity, gender, and income. The PS were included in the analyses to control for the potential association of missing the individuals who had only a baseline score.

The outcome measures were the change scores for the CSI (23, 24), Terrible to Delighted Quality of Life Scale (25), and Global Functioning: Social and Role Scales (26, 27). The models controlled for baseline assessment scores, the PS, client-level covariates (demographic variables from the baseline assessment, including duration of treatment, gender, race-ethnicity, income status, age, and duration of untreated psychosis), and site-level covariates (CSC fidelity score, turnover rate, urbanicity and poverty level of the service area, number of participants within sites, and the site-level compositions of participants in terms of race-ethnicity, gender, and income). The model’s level 1 included client-level covariates, and level 2 included site-level covariates. The models allowed intercepts to vary across the sites.

Results

The indicators of SMHA involvement with CSC programs exhibited variability (Table 1). SMHA involvement ranged from 1 to 5 (mean=2.9). Table 2 presents client change scores, in terms of average change (difference between final assessment score and baseline assessment score), for each level of SMHA involvement. In general, the higher the level of SMHA involvement, the larger was the magnitude of the change scores (reflecting positive change).

Table 3 lists the regression coefficients of the SMHA score for the levels of involvement. The model’s first level (score of 1) was used as the reference group for the four models. The regression coefficients of the control variables for each model are not shown. Each of the four groups (SMHA scores of 2–5) were compared with SMHAs that scored 1 on the involvement scale. As shown in the table, clients in CSC programs with SMHAs that were the most involved (level 5) had significantly lower final assessment CSI scores, compared with clients at clinics with SMHA level 1 involvement (β=−3.65, p=0.036), indicating a larger reduction in symptoms for clients at clinics with greater SMHA involvement. Similarly, clients at clinics with SMHA level 5 involvement had significantly higher Global Functioning: Social and Role Scale scores, compared with clients at clinics with SMHA level 1 involvement (β_social=1.32, p<0.001; β_role=1.13, p=0.019), indicating larger improvements in social and role functioning of clients served at clinics with greater SMHA involvement.

Even when the analysis controlled for demographic variables and PS, the most involved SMHAs, scoring a 5, were associated with clients who had the largest improvements in terms of symptoms, social functioning, and role functioning. For the quality-of-life measure, clients at clinics with SMHA level 2 involvement were also found to have higher scores than clients at clinics with SMHA level 1 involvement, although the difference showed only a trend toward statistical significance (β=0.55, p=0.056).

Discussion

Our results are consistent with those of previous studies (16–20, 31–33) and emphasize the importance of SMHA involvement for implementing EBPs. In our study, SMHA involvement was characterized as frequent communication with CSC programs, involvement in hiring and training of CSC staff, and involvement in decision making about CSC program delivery. Increased SMHA involvement predicted positive client outcomes in CSC programs. This trend was true for multiple client outcomes, including improved symptoms, role functioning, social functioning, and quality of life, although improvement in quality of life showed only a trend toward significance.

Previous studies have shown that SMHAs with more involvement and stronger relationships with providers and clinics experience more success in statewide implementation of EBPs (17, 18, 20, 31–33) and sustainability of EBPs (18). A study of eight SMHAs and their role in implementing EBPs found that the most successful states had SMHAs that not only understood the components of services being offered but also directed staff members on how to implement service components (17).

Unlike in other situations where SMHAs initiate and implement EBPs within their state to address a particular need, in this study every SMHA was required to administer the funding for the MHBG set-aside. One could argue that the administration of the MHBG set-aside funding was thrust upon SMHAs, and some knew little about first-episode psychosis. Additionally, CSC itself is relatively new in the United States, compared with other countries (1, 28). The earliest U.S. programs date back to 2000 (28). CSC models and CSC service components vary across the United States. Although states such as New York, Oregon, and Washington have adopted a single programmatic model for implementing CSC, most states have not (28). In a recent study of CSC implementation in California, programs across 58 counties implemented different versions of CSC, with varying service components, and in some cases, programs differed within the same county (34). In a study of 12 states, Horvitz-Lennon and colleagues (10) noted that although the MHBG set-aside funding for CSC was part of a legislative initiative, SMHAs did not have guidance about how much programmatic involvement they should have, which strategies they should use to implement CSC, or how they should administer the funds. As such, states varied widely in terms of their involvement in prescribing CSC implementation (10).

Despite the diversity among CSC programs and state contextual factors, in our sample of 34 programs across 21 states and one U.S. territory, greater SMHA involvement predicted better client outcomes. This trend held true even after the analysis was controlled for important predictors, such as baseline assessment scores, client-level demographic variables, and site-level covariates. We note that we do not believe that the relationship between SMHA involvement and client outcomes is linear, nor does the study support a causal hypothesis. Indeed, moderate levels of SMHA involvement did not necessarily predict better client outcomes, compared with low levels of SMHA involvement. Only the highest ratings of SMHA involvement predicted positive client outcomes.

Many alternative hypotheses might explain our findings because we could not measure all the potentially relevant features of the participating sites. For example, SMHAs that are most involved may specify stricter enrollment criteria to their CSC study sites, which may lead to better outcomes because the programs have fewer clients with co-occurring and complicating conditions. Another factor may be the overall CSC program budgets, because funding may be a powerful predictor of client outcomes. We were not able to assess either of these factors.

Another possible explanation for the findings may be that states with a long history of CSC implementation may have more involved SMHAs. In our study, nine CSC programs across five states reported that they provided CSC services before the availability of the 2014 MHBG set-aside funding. In our small sample, having an older program within the state did not indicate that the SMHA was more involved, and among the five states with programs initiated before 2014, only one SMHA received the highest rating of 5. One SMHA received a score of 3, one received a score of 4, and two received a score of 2.

Additionally, the involvement of a CSC Resource Center or of academic institutions with CSC expertise might explain the study findings. Although we did not directly test this hypothesis, seven SMHAs reported that they worked closely with a third party (a CSC Resource Center, academic institution, or managed care company) that managed and trained CSC staff members. Only one of these SMHAs received a rating of 5 (highest), one received a rating of 4, two received a rating of 3, two received a rating of 2, and one received a rating of 1. We did not have enough information to fully explore how this relationship may have influenced outcomes.

These findings are exploratory, because the study had several limitations. The CSC programs included in our study were chosen, in part, because of their reputation for providing good CSC care, which could have led to a biased estimate of the effect of SMHA involvement. However, enough variability existed in the data on outcomes and involvement to reveal a relationship. Our sample may not be representative of all CSC programs in the United States. Also, the inclusion of select CSC programs within each state and not the full array of CSC programs in the state may have limited the range of client outcomes, because some programs included in this study may have had better outcomes than other programs in the state.

Although we controlled for several site-level variables, the nature of the study and the number of study sites limited our ability to assess other variables that might be equally indicative of SMHA involvement or predictive of outcomes. For example, SMHA knowledge about and experience with CSC may explain how involved SMHAs are with CSC programs; however, we did not assess this variable. Additionally, we note that only one researcher on the study team rated the presence or absence of each factor by using interview transcripts; thus, these assessments could have been biased. Finally, follow-up information was not included in the analyses to further understand processes. For example, seven (32%) of the SMHAs reported involvement in the hiring of CSC staff members, but we did not have information about the extent of their involvement (e.g., reviewing resumes or participating in interviews). Additional detail could have helped us understand which SMHA activities drove the findings.

Conclusions

This study provides suggestive evidence about the important role that SMHAs can play in improving the implementation of CSC and in supporting outcomes for service users. The findings support previous studies that emphasize the power of SMHAs in “championing” EBP implementation within states. With the recent increase in SMHA-administered funding for CSC programs, it will be important for SMHAs to recognize the potential role they have in supporting clinical improvements in clients’ lives.