Designed six predictive program process and impact evaluation strategies for Greater Twin Cities United Way with a mission to build career pathways for high school students.

The United Way

  • Predictive evaluation logic models to assess the process and impact of five (5) high school career pathway academies.

  • Predictive evaluation logic model to assess the process and impact of a dual credit program for 17-20 year olds who are significantly behind in high school credits and unlikely to graduate or who have already dropped out of school.

  • Modular, scalable process and impact predictive evaluation logic models for six evaluation projects (five career academies and one dual credit program) that serve as that measurement strategy, data collection map, and communication structure for use across the Greater Twin Cities United Way (GTCUW) and multiple state agencies.

This on-going large scale evaluation project consists of a portfolio of six process and impact evaluations. Greater Twin Cities United Way (GTCUW) secured a grant to evaluate five high school career academies: (1) healthcare, (2) manufacturing, (3) IT, (4) construction, and (5) criminal justice developed by the community and regional high schools. They also received funding to evaluate a dual credit program that targets 17-20 years old, who are significantly behind in high school credits and unlikely to graduate or who have already dropped out of school. This dual credit program allows students to define a career pathway and work towards earning an associate’s degree or certificate, while completing a high school diploma.

For each of the six projects, SPP partners work across state agencies and various stakeholders such as school districts, state education departments, employers, industry specialists, workforce development organizations, and the targeted high school students.

Example program evaluation questions are:

  • To what extent has the program increased the number of students, particularly students of color and American Indian students, who graduate high school, earn post-secondary credits and credentials, and complete internships?

  • To what extent has the program grown as data driven and action-oriented collaborations?

  • Which of the practices and strategies are the most promising and actually drive the improvements being seen?

During the planning phase for each program, predictive evaluation logic models were created. The logic model is used to assess the causal (if-then) relationships between the program outputs. All six evaluation projects are currently underway. As designed, data from those who participated in the programs (experimental group) and those targeted to participate, but did not (control group) are being collected and compared to determine program outcomes. Program impact and process improvements will be determined and communicated to each stakeholder group. All six evaluations are currently under way.

Greater Twin Cities United Way (GTCUW) is a Minnesota-based non-profit agency focused on creating a better life for their community by focusing on three key areas: Stabilizing Families, Helping Children Succeed and Empowering Healthy Lives.

The challenge is to demonstrate that the six programs GTCUW supports are helping high school students continue to build their skills, postsecondary credentials, earnings and/or earning potential.

SPP collaborated with diverse stakeholder groups to design the six predictive evaluation logic models. For each program, evaluation questions and success metrics were determined. Success metrics were mapped into a predictive logic model showing the link between the program participation to success criteria to the program goals. Data collection methods were established that included data from program participants (high school students) and other functions such as the schools and state educational agencies. Since these are currently on-going evaluations, the next steps planned are to conduct data analyses employing the predictive logic models designed to compare outcomes between the students that participated in the programs and the comparison groups.

To date, the outcome has been six predictive logic models that are driving the portfolio of evaluations.