Orange County, New York System of Care Evaluation

Orange County, New York is one of 164 Systems of Care for Children’s Mental Health communities, a grant initiative from the Center for Mental Health Services (CMHS) of the Substance Abuse and Mental Health Services Administration (SAMHSA). CHSR is leading the congressionally-mandated, mixed-methods evaluation of this initiative in Orange County.

The Systems of Care initiative, administered by the Child and Family Branch of CMHS, was launched in 1993 and is the largest child mental health services initiative ever funded. Grants have been awarded to more than 150 communities to support the development of local systems of care for children with serious emotional disturbance. A system of care is a coordinated network of community-based services and supports that are organized to meet the challenges of children and youth with serious mental health needs and their families.

Evaluation & Research

CHSR's Orange County evaluation is part of the national evaluation of all system of care communities, coordinated by ICF Macro. Study components include:

• The Descriptive Study collects demographic characteristics, social and functional characteristics, mental health diagnoses, and presenting problems on all children/youth entering the system of care.  
• The Longitudinal Child and Family Outcome Study uses a combination of questionnaires and standardized instruments regarding: children’s emotional and behavioral status, strengths, educational performance, criminal justice system involvement, living environments, caregiver strain, family functioning, service utilization, and child and family satisfaction with services. Using computer-assisted interview software, families and youth are interviewed in-person at 6-month intervals over the course of the project. 
• The Services and Costs Study examines the individual-level costs associated with providing services within a system of care environment.

Program Development

Systems of care are complex system transformation and service integration initiatives. Working with the community to develop a logic model is an effective way to bring together diverse groups around a common goal – to organize their planning, implementation, and evaluation strategies. This process utilizes the Theory of Change to help participants understand how the underlying assumptions and strategies that guide local system development are critical to producing changes in services and supports that are expected to lead to improvements for the community and its residents. After multiple iterations of the model and many discussions with a broad base of community members, parents, youth, agency administrators, and project leaders, the “final” logic model is presented here. Ideally, a logic model should never be considered “final.” Rather, it is a tool for ongoing discussion, monitoring progress, continuous quality improvement, and a vehicle for making midcourse corrections throughout an initiative.

CHSR evaluators are employing a family empowerment approach to meaningfully engage families and youth in the implementation of the research. An evaluation advisory group, comprised of providers, families, and youth, meets twice a month to develop local surveys, review and interpret findings, and provide ongoing input into the evaluation process. The evaluation team is an integral part of the local initiative as a resource for logic model development, strategic planning, forms development, and continuous feedback on process and outcomes.

Contact: Rose Greene