Working with an Evaluator: What to Know

Many ATE PIs and project staff new to working with external evaluators may wonder what working with an evaluator even looks like. If that sounds familiar, you’ve come to the right place!

In 2020 interviews, nine ATE PIs reflected on their evaluation experiences. Their insights can shed light on the process and help increase your evaluation’s impact on your project. Here are four takeaways:

Evaluators often wear many hats. All nine PIs shared that their evaluator also acted in at least one other capacity: grant writer, cheerleader, coach, thought partner, project manager, liaison to other ATE grants. So, evaluators often have something to add that you don’t expect. For example, one PI noted that their evaluator was

… looking at that overarching picture of the [ATE] community … talking with other evaluators to identify some of those areas, and then bring that back to help facilitate that conversation across the PIs …

Good relationships with evaluators are vital to project success. In strong PI–evaluator relationships, PIs feel empowered and encouraged in their decisions, see their evaluator as a project team member, and ask questions. Frequent, clear communication can foster those strong relationships, which in turn can help overcome barriers and strengthen projects. One PI said that their evaluator was “always responsive,” while another PI outlined their helpful, frequent communication:

It was an evaluation, but in some ways, it was like a mentorship a little bit as well…. Since we met regularly, you know, the roadblocks that we’re encountering could be addressed as early as possible, and then we’ve kind of worked on a solution together.

Evaluators and PIs should identify ways to establish good rapport. As one PI said:

Early in the process, we had a face-to-face rapport, even though we’re in different states and far away, and that really helped me to get to know them. It helped me to calm down about their role – and to see them more as a partner, which I think is great.

PIs value their evaluations! Many PIs said their evaluations helped them accomplish something for their project, gain insight into a situation, or learn something new. PIs and evaluators can replicate these experiences by discussing their attitudes and expectations about evaluation. As the PIs quoted below indicate, they viewed evaluation as a process whose role was to assess impact and provide a holistic view:

It’s great to get the money from NSF, but the goal is to have an impact, right? So, am I having an impact in my program?… I would never see myself doing a project without doing [an] evaluation.

 They [the evaluators] are not trying to showcase only the good work, but they’re honest in that conversation. I think that’s really key and critical to make the evaluation process something of benefit and value to move projects and centers forward.

Evaluation results can be used in multiple ways. Identifying how an evaluation can be helpful throughout the project is important, as is considering the various ways results might be used. ATE PIs reported having additional insight into their project’s implementation because of their evaluation¾enabling them to revisit components such as outreach and recruitment.

For example, one PI said their evaluation results “helped us drive our communication plan.” Another said that because their evaluation helped them “think outside of the box,” they reconsidered their “focus areas priorities and how we’re executing them.”

Projects also shared evaluation results with audiences inside and outside of their institutions¾in reports published online, conference presentations, and papers.

For more inspiration in using evaluation to benefit your project, check out EvaluATE’s checklist on different ways to use evaluation results.

Acknowledgment: Thank you to Michael Harnar, who planned and conducted the interviews that informed this blog post.