Program Evaluation

Program Evaluation

Evaluation of one sort or another is built into nearly all industries, though it could slide in under many names: “How’d we do?” surveys, annual employee reviews, or profit and loss statements. Each type of evaluation is intended to identify and measure the processes, outcomes, and impacts of a product, program, or project. When evaluating initiatives in the health care space, specialized evaluators are often used throughout a project to ensure that, after all is completed, those running the initiative have sufficient information to determine whether they succeeded…. or not.

Types of Evaluation and the Questions They Can Answer

Impact Evaluation: What Was the Long-Term Impact of Our Program?

Impact refers to the global effects of your program, such as the effects on your patient’s health that are a result of your program. These can include measures like improved health outcomes, lower costs, and better care. Impact evaluations are rigorous, long-term assessments of changes or improvements that reach beyond the scope of your program.

Outcome Monitoring and Evaluation: Did Our Activities Have the Expected Effects?

Outcome refers to the direct results of your project or program efforts on the population served. Outcome measures are defined during project design and represent details that are measurable and may be attributable to program activities.

Outcome monitoring is the tracking of these measures during and after program implementation. Monitoring provides immediate feedback for short-term improvements and contributes to your evaluation dataset. Outcome evaluation is the process of analyzing outcome measures to see whether an effect was observed. Outcome evaluation can include comparison between groups, time series analysis, or pre-post testing.

Process Monitoring and Evaluation: Did We Do What We Said We Would Do? What Worked Well? What Can Be Improved?

Process evaluation focuses on the effectiveness of program implementation and highlights successful strategies or areas for improvement. It describes program elements such as the services delivered, methodology or model used, population of interest, resources required, and the qualifications of those delivering services.

Process monitoring is the act of tracking the inputs and outputs of your project to inform your progress and contribute information for evaluation. These may include who is performing the work, how often, where, and the number of clients or patients in the program. Process evaluation also typically involves accounting for the materials, services, and key resources used.

Formative Evaluation: What Will We Do, Why Will We Do It, and How?

Formative evaluation, which can include a needs assessment or evaluability assessment, is used to inform project planning and design to ensure that a program or activity is feasible, appropriate, and acceptable prior to implementation. Completing a formative evaluation before starting a project helps ensure success by identifying the most likely areas of impact, populations served, evidence-based methodologies, required inputs, and available resources.

Program Evaluation at CIVHC

CIVHC’s Evaluation Team is trained to meet evaluation needs from consultation and project design to data collection, monitoring, analysis, reporting, training, and capacity building. The services and products the team provides include:

Evaluation Services

  • Full-Scale Program Evaluations – CIVHC’s Evaluation Team can evaluate your project or program from start to finish including collecting and analyzing data, creating infographics, and providing final evaluation reports, executive summaries, slide sets, or other materials.
  • Survey Design and Analysis – CIVHC can help you design surveys that ask the right questions in the best ways through use of existing, validated tools or the development of custom survey instruments. Evaluation analysts can also review your data and provide insights.
  • Quantitative Data Collection and Analysis- We will work with you to define a set of meaningful and measurable quantitative metrics to track your program’s achievements, establish a data collection plan, and implement, analyze, and present your results. The CO APCD serves as a unique data source that can be combined with your own program data or other sources to provide comprehensive quantitative information.
  • Qualitative Data Collection and Analysis – Qualitative analysis can provide rich context that may not be achieved through quantitative assessment alone and can provide valuable information for program design or revision. CIVHC can help you design and implement a qualitative evaluation strategy best suited for your program’s needs.
  • Capacity Building – CIVHC staff can help you make the best use of your evaluation data through capacity building, training, program monitoring, and data numeracy education.
  • Evaluation Support for Grant Writing  – A strong evaluation plan in a grant proposal is critical for ensuring you have the capacity and resources for evaluation activities. Evaluation data can also be powerful when proposing a new program or redesign. CIVHC can assist with writing and reviewing these proposal components, as well as coach staff in developing these skills.

Potential Outputs Related to Evaluation Services

Not sure where to start or don't see the services you need listed? Contact us! We can help assess your needs and identify the best approach.