Each year, millions of Americans receive evidence-based psychotherapies (EBPs) such as cognitive behavioral therapy (CBT) for the treatment of mental and behavioral health problems. Yet, at present, there is no scalable method for evaluating the quality of psychotherapy services. In research settings, human-based behavioral coding methods are used, but these are time consuming, costly, and rarely used in real-world clinical settings. Thus, EBP quality and effectiveness is unmeasured and unknown. The current, fast-track STTR proposal will develop and evaluate an AI-based software system (LyssnCBT) that will automatically estimate CBT fidelity from an audio recording of a CBT session. Importantly, the current work builds from Lyssn's previous, successful work in developing an automated system for evaluating motivational interviewing (MI), and previous research showing that AI algorithms can accurately estimate CBT fidelity. Lyssn.io, Inc., ("Lyssn") is a start-up developing AI-based technologies to support training, supervision, and quality assurance of evidence-based counseling. Our goal is to develop innovative health technology solutions that are objective, scalable, and cost efficient. Lyssn offers a HIPAA-compliant, cloud-based platform for secure recording, sharing, and reviewing of therapy sessions, which includes AI-generated metrics for MI. The proposed LyssnCBT tool will build from and be integrated into this core platform. Lyssn is partnering with Dr. Torrey Creed and the Penn Collaborative, which has a 14+ year track record of gold-standard CBT training and supervision, including more than 100 community agencies with almost 900 providers. The expertise, relationships, and amassed data -- more than 8,000 recorded sessions and more than 3,000 rated for CBT fidelity -- form the clinical foundation for the current research. Phase I will work from an existing AI-CBT prototype to develop LyssnCBT. Core activities include user-centered design focus groups and interviews with community mental health (CMH) therapists, supervisors, and administrators, which will inform the design and development of LyssnCBT. LyssnCBT will be evaluated for usability and implementation readiness in a final stage of Phase I. Phase II will conduct a field-based usability trial and a stepped-wedge, hybrid implementation-effectiveness randomized trial (N = 1,850 CMH clients) to evaluate the effectiveness of LyssnCBT to improve therapist CBT skills and client outcomes, and to reduce client drop-out. Analyses will also examine the hypothesized mechanism of action underlying LyssnCBT. The research is strongly aligned with NIMH's 2020 Strategic Plan and its emphasis on a computational approach to scaling up treatment delivery and monitoring. Successful execution will provide automated, scalable CBT fidelity feedback for the first time ever, supporting high-quality training, supervision, and quality assurance, and providing a core technology foundation that could support a range of EBPs in the future.
Public Health Relevance Statement: PROJECT NARRATIVE Evidence-based psychotherapies such as cognitive-behavioral therapy (CBT) are increasingly emphasized in professional practice guidelines, taught in training programs, sought out by consumers, and valued by payers. Yet, at present, there is no scalable method for evaluating CBT fidelity to support training, supervision, and quality assurance in community practice settings. The current study will develop and evaluate a clinical support software tool (LyssnCBT) that uses speech signal processing and machine learning to automate CBT fidelity and support high-quality CBT supervision and service delivery in community settings.
Project Terms: Adult; 21+ years old; Adult Human; adulthood; Algorithms; Mental disorders; Mental health disorders; Psychiatric Disease; Psychiatric Disorder; mental illness; psychiatric illness; psychological disorder; Client; Clinical Research; Clinical Study; Cognitive Therapy; Cognition Therapy; Cognitive Psychotherapy; cognitive behavior intervention; cognitive behavior modification; cognitive behavior therapy; cognitive behavioral intervention; cognitive behavioral modification; cognitive behavioral therapy; cognitive behavioral treatment; Communities; Counseling; Dropout; Engineering; Feedback; Focus Groups; Foundations; Future; Goals; Gold; Group Interviews; Healthcare Systems; Health Care Systems; Human; Modern Man; Interview; Lead; Pb element; heavy metal Pb; heavy metal lead; Mental Health; Mental Hygiene; Psychological Health; Methods; Methodology; Persons; National Institute of Mental Health; NIMH; Online Systems; On-Line Systems; online computer; web based; Patients; Pilot Projects; pilot study; Professional Practice; health care professional practice; healthcare professional practice; Psychotherapy; Research; Resources; Research Resources; Science; Computer software; Software; Software Tools; Computer Software Tools; Speech; Standardization; Supervision; Technology; Testing; Time; Training Programs; Training Support; Universities; Vision; Sight; visual function; Work; Treatment outcome; Administrator; Practice Guidelines; base; quality assurance; improved; Clinical; Phase; Evaluation; Training; disability; Policies; Collaborations; tool; machine learned; Machine Learning; Complex; Clinic; Protocol; Protocols documentation; System; Services; American; Performance; tech development; technology development; HIPAA; Kennedy Kassebaum Act; PL 104-191; PL104-191; Public Law 104-191; United States Health Insurance Portability and Accountability Act; Health Insurance Portability and Accountability Act; skills; motivational interview; motivational enhancement therapy; Reporting; Health Care Technology; Healthcare Technology; Health Technology; Coding System; Code; behavioral health; develop software; developing computer software; software development; Provider; clinical depression; major depression; major depression disorder; Major Depressive Disorder; Effectiveness; Preparedness; Readiness; telehealth; Data; randomisation; randomization; randomly assigned; Randomized; Strategic Planning; Community Practice; Small Business Technology Transfer Research; STTR; Monitor; Development; developmental; Behavioral; cost; digital; software systems; design; designing; Outcome; scale up; cost efficient; Consumption; innovation; innovate; innovative; speech processing; user centered design; usability; implementation science; addiction; addictive disorder; prototype; commercialization; therapeutic development; therapeutic agent development; community setting; evidence base; symptomatic improvement; improve symptom; symptom improvement; cognitive enhancement; Secure; signal processing; cloud based; phase II trial; phase 2 trial; cloud platform; cloud server; software as a service; dashboard; service delivery; practice setting; randomized effectiveness trial; effectiveness implementation study; effectiveness implementation hybrid; effectiveness evaluation; assess effectiveness; determine effectiveness; effectiveness assessment; evaluate effectiveness; services as usual
