Abstract
Despite the central role of training and consultation in the implementation of evidence-based psychological interventions (EBPIs), comprehensive reviews of research on training have highlighted serious gaps in knowledge regarding best practices. Consultation after initial didactic training appears to be of critical importance, but there has been very little research to determine optimal consultation format or interventions. This observational study compared two consultation formats that included review of session audio and feedback in the context of a program to train clinicians (n = 85) in community mental health clinics to deliver cognitive therapy (CT). A “gold standard” condition in which clinicians received individual feedback after expert consultants reviewed full sessions was compared to a group consultation format in which short segments of session audio were reviewed by a group of clinicians and an expert consultant. After adjusting for potential baseline differences between individuals in the two consultation conditions, few differences were found in terms of successful completion of the consultation phase or in terms of competence in CT at the end of consultation or after a 2 year follow-up. However, analyses did not support hypotheses regarding non-inferiority of the group consultation condition. While both groups largely maintained competence, clinicians in the group consultation condition demonstrated increases in competence over the follow-up period, while a sub-group of those in the individual condition experienced decreases. These findings, if replicated, have important implications for EBP implementation programs, as they suggest that observation and feedback is feasible in community mental health setting, and that employing this method in a group format is an effective and efficient consultation strategy that may enhance the implementation and sustainability of evidence-based psychotherapies.
Similar content being viewed by others
References
Baer, J. S., Rosengren, D. B., Dunn, C. W., Wells, E. A., Ogle, R. L., & Hartzler, B. (2004). An evaluation of workshop training in motivational interviewing for addiction and mental health clinicians. Drug and Alcohol Dependence, 73(1), 99–106. doi:10.1016/j.drugalcdep.2003.10.001.
Beck, A. T. (2005). The current state of cognitive therapy: A 40-year retrospective. Archives of General Psychiatry, 62(9), 953–959. doi:10.1001/archpsyc.62.9.953.
Beidas, R., Edmunds, J., Cannuscio, C., Gallagher, M., Downey, M. M., & Kendall, P. (2013). Therapists perspectives on the effective elements of consultation following training. Administration and Policy in Mental Health and Mental Health Services Research, 40(6), 507–517. doi:10.1007/s10488-013-0475-7.
Beidas, R., Edmunds, J., Marcus, S., & Kendall, P. (2012). Training and consultation to promote implementation of an empirically supported treatment: A randomized trial. Psychiatric Services, 63(7), 660–665. doi:10.1176/appi.ps.201100401.
Beidas, R., & Kendall, P. C. (2010). Training therapists in evidence-based practice: A critical review of studies from a systems-contextual perspective. Clinical Psychology: Science and Practice, 17(1), 1. doi:10.1111/j.1468-2850.2009.01187.x.
Blackwelder, W. (2004). Current issues in clinical equivalence trials. Journal of Dental Research, 83(suppl 1), C113–C115.
Branson, A., Shafran, R., & Myles, P. (2015). Investigating the relationship between competence and patient outcome with CBT. Behaviour Research and Therapy, 68, 19–26. doi:10.1016/j.brat.2015.03.002.
Brosan, L., Reynolds, S., & Moore, R. G. (2008). Self evaluation of cognitive therapy performance: Do therapists know how competent they are? Behavioural and Cognitive Psychotherapy, 36(5), 581. doi:10.1017/S1352465808004438.
Bryk, A. S., & Raudenbush, S. W. (1992). Hierarchical linear models: Applications and data analysis methods. Thousand Oaks, CA: Sage Publications Inc.
Carroll, K. M., Martino, S., & Rounsaville, B. J. (2010). No train, no gain? Clinical Psychology: Science and Practice, 17(1), 36–40. doi:10.1111/j.1468-2850.2009.01190.x.
Chorpita, B. F., & Regan, J. (2009). Dissemination of effective mental health treatment procedures: Maximizing the return on a significant investment. Behaviour Research and Therapy, 47(11), 990–993. doi:10.1016/j.brat.2009.07.002.
Cichetti, D. (1994). Guidelines, criteria, and rules of thumb for normed and standardized instruments in psychology. Psychological Assessment, 6(4), 284–290.
Creed, T. A., Stirman, S. W., Evans, A. C., & Beck, A. T. (2014). A model for implementation of cognitive therapy in community mental health: The Beck Initiative. The Behavior Therapist, 37(3).
D’Agostino, R. B., Massaro, J. M., & Sullivan, L. M. (2003). Non-inferiority trials: Design concepts and issues-the encounters of academic consultants in statistics. Statistics in Medicine, 22(2), 169–186.
Eckardt, P. (2012). Propensity score estimates in multilevel models for causal inference. Nursing Research, 61(3), 213–223.
Edmunds, J. M., Beidas, R. S., & Kendall, P. C. (2013). Dissemination and implementation of evidence-based practices: Training and consultation as implementation strategies. Clinical Psychology: Science and Practice, 20(2), 152–165. doi:10.1111/cpsp.12031.
Feldstein, A. C., Glasgow, R. E., & Smith, D. H. (2008). A practical, robust implementation and sustainability model (PRISM) for integrating research findings into practice. Joint Commission Journal on Quality and Patient Safety, 34(4), 228–243. doi:10.2165/00148365-200806040-00001.
Fixsen, D. L., Naoom, S. F., Blase, K. A., Friendman, R. M., & Wallace, F. (2005). Implementation research: A synthesis of the literature. Tampa, FL: The National Implementation Research Network, Louis de la Parte Florida Mental Health Institute, University of South Florida.
Fleming, L. M., Glass, J. A., Fujisaki, S., & Toner, S. L. (2010). Group process and learning: A grounded theory model of group supervision. Training and Education in Professional Psychology, 4(3), 194.
Franklin, M. E., Abramowitz, J. S., Kozak, M. J., Levitt, J. T., & Foa, E. B. (2000). Effectiveness of exposure and ritual prevention for obsessive-compulsive disorder: Randomized compared with nonrandomized samples. Journal of Consulting and Clinical Psychology, 68(4), 594. doi:10.1037/0022-006X.68.4.594.
Harder, V. S., Stuart, E. A., & Anthony, J. C. (2010). Propensity score techniques and the assessment of measured covariate balance to test causal associations in psychological research. Psychological Methods, 15(3), 234–249. doi:10.1037/a0019623.
Henggeler, S. W., Chapman, J. E., Rowland, M. D., Sheidow, A. J., & Cunningham, P. B. (2013). Evaluating training methods for transporting contingency management to therapists. Journal of Substance Abuse Treatment, 45(5), 466–474.
Herschell, A. D., Kolko, D. J., Baumann, B. L., & Davis, A. C. (2010). The role of therapist training in the implementation of psychosocial treatments: A review and critique with recommendations. Clinical Psychology Review, 30(4), 448–466. doi:10.1016/j.cpr.2010.02.005.
Juster, H. R., Heimberg, R. G., & Engelberg, B. (1995). Self selection and sample selection in a treatment study of social phobia. Behavioral Research Therapy, 33(3), 321–324.
Karlin, B. E., Ruzek, J. I., Chard, K. M., Eftekhari, A., Monson, C. M., Hembree, E. A., et al. (2010). Dissemination of evidence-based psychological treatments for posttraumatic stress disorder in the veterans health administration. Journal of Traumatic Stress, 23(6), 663–673. doi:10.1002/jts.20588.
Kolko, D. J., Baumann, B. L., Herschell, A. D., Hart, J. A., Holden, E. A., & Wisniewski, S. R. (2012). Implementation of AF-CBT by community practitioners serving child welfare and mental health: A randomized trial. Child maltreatment. doi:10.1177/1077559511427346.
Lewandowsky, S., & Thomas, J. L. (2009). Expertise: Acquisition, limitations, and control. Reviews of human factors and ergonomics, 5(1), 140–165.
McHugh, R. K., & Barlow, D. H. (2010). The dissemination and implementation of evidence-based psychological treatments: A review of current efforts. American Psychologist, 65, 73–84. doi:10.1037/a0018121.
Miller, W. R., Yahne, C. E., Moyers, T. B., Martinez, J., & Pirritano, M. (2004). A randomized trial of methods to help clinicians learn motivational interviewing. Journal of Consulting and Clinical Psychology, 72(6), 1050–1062. doi:10.1037/0022-006X.72.6.1050.
Milne, D. (2014). Beyond the “Acid Test”: A conceptual review and reformulation of outcome evaluation in clinical supervision. American Journal of Psychotherapy, 68(2), 213–230.
Nacasch, N., Huppert, J. D., Su, Y.-J., Kivity, Y., Dinshtein, Y., Yeh, R., & Foa, E. B. (2014). Are 60-minute prolonged exposure sessions with 20-minute imaginal exposure to traumatic memories sufficient to successfully treat PTSD? A randomized noninferiority clinical trial. Behavior Therapy, 46, 328–341.
Ögren, M.-L. E. C. (2009). Group supervision in psychotherapy. Main findings from a Swedish research project on psychotherapy supervision in a group format. British Journal of Guidance & Counselling, 37(2), 129–139. doi:10.1080/03069880902728614.
Rakovshik, S. G., & McManus, F. (2010). Establishing evidence-based training in cognitive behavioral therapy: A review of current empirical findings and theoretical guidance. Clinical Psychology Review, 30(5), 496–516. doi:10.1016/j.cpr.2010.03.004.
Raudenbush, S. W. (1997). Statistical analysis and optimal design for cluster randomized trials. Psychological Methods, 2(2), 173.
Ruzek, J. I., & Rosen, R. C. (2009). Disseminating evidence-based treatments for PTSD in organizational settings: A high priority focus area. Behaviour Research and Therapy, 47(11), 980–989. doi:10.1016/j.brat.2009.07.008.
Schafer, J. L., & Kang, J. (2008). Average causal effects from nonrandomized studies: A practical guide and simulated example. Psychological Methods, 13(4), 279–313. doi:10.1037/a0014268.
Schoenwald, S. K., Garland, A. F., Chapman, J. E., Frazier, S. L., Sheidow, A. J., & Southam-Gerow, M. A. (2011). Toward the effective and efficient measurement of implementation fidelity. Administration and Policy in Mental Health and Mental Health Services Research. doi:10.1007/s10488-010-0321-0.
Shaw, B. F., Elkin, I., Yamaguchi, J., Olmsted, M., Vallis, T. M., Dobson, K. S., et al. (1999). Therapist competence ratings in relation to clinical outcome in cognitive therapy of depression. Journal of Consulting and Clinical Psychology, 67(6), 837.
Sholomskas, D. E., Syracuse-Siewert, G., Rounsaville, B. J., Ball, S. A., Nuro, K. F., & Carroll, K. M. (2005). We don’t train in vain: A dissemination trial of three strategies of training clinicians in cognitive-behavioral therapy. Journal of Consulting and Clinical Psychology, 73(1), 106–115. doi:10.1037/0022-006X.73.1.106.
Simons, A. D., Padesky, C. A., Montemarano, J., Lewis, C. C., Murakami, J., Lamb, K., et al. (2010). Training and dissemination of cognitive behavior therapy for depression in adults: A preliminary examination of therapist competence and client outcomes. Journal of Consulting and Clinical Psychology, 78(5), 751–756. doi:10.1037/a0020569.
Singer, J. D., & Willett, J. B. (2003). Applied longitudinal data analysis: Modeling change and event occurrence. London: Oxford University Press.
Stirman, S. W., Bhar, S. S., Spokas, M., Brown, G. K., Creed, T. A., Perivoliotis, D., et al. (2010). Training and consultation in evidence-based psychosocial treatments in public mental health settings: The ACCESS model. Professional Psychology: Research and Practice, 41(1), 48–56. doi:10.1037/a0018099.
Stirman, S. W., Buchhofer, R., McLaulin, J. B., Evans, A. C., & Beck, A. T. (2009). The Beck initiative: A partnership to implement cognitive therapy in a community behavioral health system. Psychiatric Services, 60, 1302–1310. doi:10.1176/appi.ps.60.10.1302.
Stirman, S. W., Crits-Christoph, P., & DeRubeis, R. J. (2004). Achieving successful dissemination of empirically supported psychotherapies: A synthesis of dissemination theory. Clinical Psychology: Science and Practice, 11(4), 343–359. doi:10.1093/clipsy/bph091.
Stirman, S. W., Gutierrez-Colina, A., Toder, K., Castro, F., Esposito, G., Barg, F., et al. (2013). Clinicians’ perspectives on cognitive therapy in community mental health settings: Implications for training and implementation. Administration and Policy in Mental Health and Mental Health Services Research, 40(4), 274–285. doi:10.1007/s10488-012-0418-8.
Stirman, S. W., Miller, C. J., Toder, K., Calloway, A., Crits-Christoph, P., & Beck, A. T. (2012). Perspectives on cognitive therapy training within community mental health settings: Implications for clinician satisfaction and skill development. Depression Research and Treatment, 2012, 1–11. doi:10.1155/2012/391084.
Swales, M. A., Taylor, B., & Hibbs, R. A. (2012). Implementing dialectical behaviour therapy: Programme survival in routine healthcare settings. Journal of Mental Health, 21(6), 548–555.
Tracey, T. J., Wampold, B. E., Lichtenberg, J. W., & Goodyear, R. K. (2014). Expertise in psychotherapy: An elusive goal? American Psychologist, 69(3), 218.
Weissman, M. M., Verdeli, H., Gameroff, M. J., Bledsoe, S. E., Betts, K., Mufson, L., et al. (2006). National survey of psychotherapy training in psychiatry, psychology, and social work. Archives of General Psychiatry, 63(8), 925–934.
Young, J., & Beck, A. T. (1980). Cognitive therapy scale: Rating manual. Unpublished manuscript.
Acknowledgments
A portion of this work was supported by funding from the National Institute of Mental Health (R00 MH-080100; PI: Stirman). The authors wish to thanks Drs. Michael Suvak and Robert Gallop for their feedback on the statistical analyses in this paper.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Stirman, S.W., Pontoski, K., Creed, T. et al. A Non-randomized Comparison of Strategies for Consultation in a Community-Academic Training Program to Implement an Evidence-Based Psychotherapy. Adm Policy Ment Health 44, 55–66 (2017). https://doi.org/10.1007/s10488-015-0700-7
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10488-015-0700-7