Public and private-sector agencies and organizations often include evaluation as an important component of their programmatic initiatives. But what is program evaluation and why is it important?
Evaluation is the application of defensible criteria to determine the merit or worth of programs (Fitzpatrick, Sanders, & Worthen, 2003). Program evaluation involves the systematic collection of information about a program, or some aspect(s) of a program to reach conclusions about service delivery or effectiveness. The purposes of program evaluation may include 1) judgment of merit or worth; 2) oversight and compliance; 3) program and organizational improvement; or 4) knowledge development.
National Research Center
National Research Center, Inc. (NRC) has provided program evaluation services to an array of clients. NRC has worked with entities like the Fund for Colorado’s Future and The Colorado Trust on a variety of multi-site, multi-year education or health-related initiatives. More often, we work with non-profit, community based organizations—those serving victims of sexual assault and domestic violence, after-school programs, dental aid, child care, African American leaders, Latino parents, elderly, homeless and more. We have published outcome handbooks for older adults, youth-serving programs, community food security projects, and obesity prevention programs as well as a toolkit to help community based organizations measure their effectiveness.
The evaluation team at NRC includes specialists in program evaluation, research design, quantitative and qualitative methods, survey research and cost-benefit analyses. Our staff have education and training in various fields including education, political science, public health, psychology, sociology, social work, economics and criminal justice. The diversity in the educational and professional background of our team, allows NRC professionals to apply their familiarity with program evaluation techniques to the specific content areas that our clients desire.
NRC uses a collaborative approach, teaming with the client to design and implement a comprehensive evaluation. Accordingly, we offer many different types of program evaluation and a variety of activities that include, but are not limited to:
- Formative and Summative Evaluation
- Needs Assessments
- Process Evaluation or Monitoring Studies
- Outcome Evaluation
- Logic Model Development
- Evaluation Handbook Creation
- Outcome Toolkit Production
- Evaluation Capacity-Building
- Technical Assistance
- Evaluation Workshops/Presentations
- Data Collection and Analysis
- Reporting of Results
- Recommendations for Future Planning
- Presentation of Results
NRC’s collaborative perspective allows flexibility in evaluation modeling. We are comfortable with employing management-based or participant-oriented approaches to meet the goals and needs of our clients. One of the new NRC approaches, “insourcing,” is designed to provide evaluation services to community based organizations (CBOs) while placing the least amount of burden on CBO staff and staff time. Whichever evaluation approach is ultimately selected, NRC evaluations are always responsive to the particular needs and interests of our clients.
We generally collect primary data through phone, in-person, mail or Web-based survey research, focus groups or one-on-one interviews. Other data sources that are typically used include: secondary data, information that is kept in local administrative or government files, like attendance records, grades, insurance coverage, vital statistics, Census and economic data. Our evaluations are independent and rigorous, providing our clients with the evidence they need to improve their programs, help convince funders of the value of their investments and draw attention to important findings for scholars or others interested in results that can be generalized.
The NRC Evaluation Process generally includes the following components:
- Understanding the program to be evaluated
- Developing a logic model for the program
- Designing and planning the evaluation
- Developing the instruments and data collection protocols
- Collecting the data
- Analyzing the data
- Writing the report
- Presenting the results
NRC trains and provides technical assistance to build the capacity of organizations to conduct their own assessments of effectiveness by measuring the changes they expect to see through their activities, policies and programs. Our trainings on outcome measurement are interactive, creative and rich with real-life examples. They motivate and offer specific solutions to general problems, taking participants at their level of sophistication to an understanding of how to put results into action.
Fitzpatrick, J. L., Sanders, J. R. & Worthen, B. R. (2003). Program evaluation: Alternative approaches and practical guidelines (3rd ed.). Boston: Allyn & Bacon.