What is the best way to coordinate internal and external evaluation?

All ATE projects are required to allocate funds for external evaluation services. So, when it comes to internal and external evaluation, the one certain thing is that you must have an external evaluator. (In rare cases, alternative arrangements may be approved by a program officer.)

There are two types of external evaluators:

Type 1: Evaluators who are completely external to the institution

Type 2 Evaluators who are external to the project, but internal to the institution (such as an institutional researcher or a faculty member from a different department from where the project is located).

Both types are considered external, as long as the Type 2 external evaluator is truly independent from the project.

An internal evaluator is a member of the project staff, who is directly funded by the project, such as a project manager. More commonly, internal evaluation is a shared responsibility among team members. There are many options for coordinating internal and external evaluation functions. Over the years, I have noted four basic approaches:

(1) External Evaluator as Coach: The external evaluator provides guidance and feedback to the internal project team throughout the life of the grant. This is a good approach when there is already some evaluation competence among team members. The external evaluator’s involvement enhances the credibility of the evaluation and helps the team continue to build their evaluation knowledge and skills.

(2) External Evaluator as Heavy-Lifter: The external evaluator takes the lead in planning the evaluation, designing instruments, analyzing results, and writing reports. The internal team mainly gathers data and provides it to the external evaluator for processing. In this approach, the external evaluator should provide clear-cut data collection protocols to ensure systematic collection and handling of data by the internal team before they turn the information over to the external evaluator.

(3) External Evaluator as Architect: The external evaluator designs the overall evaluation and develops data collection instruments. The project team executes the plan, with technical assistance from the external evaluator as needed—particularly at critical junctures in the evaluation such as analysis and reporting. With this approach, it is important to front-load the evaluation budget in the first year of the project to allow for intensive involvement by the external evaluator.

(4) Divide-and-Conquer: The internal team is responsible for evaluating project implementation and immediate results. The external evaluator handles the evaluation of longer-term outcomes. This is the approach that EvaluATE uses. We carefully track and analyze data about how well we are reaching and engaging our audience. We are responsible for assessing immediate outcomes of our webinars and workshops (i.e., participants’ satisfaction, self-reported learning, and intent to use content). Our external evaluator is responsible for determining and assessing EvaluATE’s impact on the evaluation practices of our users.

A note of caution: Taking on part of an evaluation internally is often seen as a means of conserving project resources, and it can have that effect. But do not make the mistake of thinking internal evaluation is cost-free. At minimum, it takes time, which is sometimes a rarer commodity than money. In short, there is no one best way to coordinate internal and external evaluation. Your approach should make sense for your project in light of available resources (including staff time and expertise) and what you need your evaluation to do for your project.

What to know more? Check out this 5-minute video in which I explain what counts as independent, how to find an external evaluator, and how to divide responsibilities between integrate internal and external evaluators: Evaluation Basics Part 4: Who Can Do It?