Types of Evaluation and the Questions They Can Answer
Impact refers to the global effects of your program, such as the effects on your patient’s health that are a result of your program. These can include measures like improved health outcomes, lower costs, and better care. Impact evaluations are rigorous, long-term assessments of changes or improvements that reach beyond the scope of your program.
Outcome refers to the direct results of your project or program efforts on the population served. Outcome measures are defined during project design and represent details that are measurable and may be attributable to program activities.
Outcome monitoring is the tracking of these measures during and after program implementation. Monitoring provides immediate feedback for short-term improvements and contributes to your evaluation dataset. Outcome evaluation is the process of analyzing outcome measures to see whether an effect was observed. Outcome evaluation can include comparison between groups, time series analysis, or pre-post testing.
Process evaluation focuses on the effectiveness of program implementation and highlights successful strategies or areas for improvement. It describes program elements such as the services delivered, methodology or model used, population of interest, resources required, and the qualifications of those delivering services.
Process monitoring is the act of tracking the inputs and outputs of your project to inform your progress and contribute information for evaluation. These may include who is performing the work, how often, where, and the number of clients or patients in the program. Process evaluation also typically involves accounting for the materials, services, and key resources used.
Formative evaluation, which can include a needs assessment or evaluability assessment, is used to inform project planning and design to ensure that a program or activity is feasible, appropriate, and acceptable prior to implementation. Completing a formative evaluation before starting a project helps ensure success by identifying the most likely areas of impact, populations served, evidence-based methodologies, required inputs, and available resources.
Program Evaluation at CIVHC
CIVHC’s Evaluation Team is trained to meet evaluation needs from consultation and project design to data collection, monitoring, analysis, reporting, training, and capacity building. The services and products the team provides include:
- Full-Scale Program Evaluations – CIVHC’s Evaluation Team can evaluate your project or program from start to finish including collecting and analyzing data, creating infographics, and providing final evaluation reports, executive summaries, slide sets, or other materials.
- Survey Design and Analysis – CIVHC can help you design surveys that ask the right questions in the best ways through use of existing, validated tools or the development of custom survey instruments. Evaluation analysts can also review your data and provide insights.
- Quantitative Data Collection and Analysis- We will work with you to define a set of meaningful and measurable quantitative metrics to track your program’s achievements, establish a data collection plan, and implement, analyze, and present your results. The CO APCD serves as a unique data source that can be combined with your own program data or other sources to provide comprehensive quantitative information.
- Qualitative Data Collection and Analysis – Qualitative analysis can provide rich context that may not be achieved through quantitative assessment alone and can provide valuable information for program design or revision. CIVHC can help you design and implement a qualitative evaluation strategy best suited for your program’s needs.
- Capacity Building – CIVHC staff can help you make the best use of your evaluation data through capacity building, training, program monitoring, and data numeracy education.
- Evaluation Support for Grant Writing – A strong evaluation plan in a grant proposal is critical for ensuring you have the capacity and resources for evaluation activities. Evaluation data can also be powerful when proposing a new program or redesign. CIVHC can assist with writing and reviewing these proposal components, as well as coach staff in developing these skills.