Steps for Communicating Evaluation Design to PIs

As ATE evaluators, our team engages with many new ATE PIs. To equip new clients for success, we initiate a prescriptive process for providing evaluation services. We develop a brief evaluation summary, provide an overview of evaluation roles and responsibilities, and expand the evaluation plan upon grant award. The details of each step are addressed below.

Step 1: Design the Evaluation Plan

We aim to develop the evaluation plan early on in the grant development process. However, the timeline for design varies by client. For example, we engage with Mentor Connect and Mentor Up clients five to six months before grant submission. With non-mentor clients, we start one to two months before grant submission.

With each client, we remain actively engaged during the development of grant goals, so our evaluation plan is adaptive to the project’s direction. Early engagement helps to build rapport and allows more time to learn about the project before designing the evaluation plan.

The initial evaluation plan draft is referred to as the evaluation plan summary. The summary includes a brief one- to one-and-one-half-page evaluation plan addressing objectives, activities, and formative and summative evaluation questions. The questions are derived from a review of the project narrative and initial meeting(s) with the client.

Step 2: Review the Evaluation Process

If the client is awarded the grant, we hold a project kickoff meeting where roles and responsibilities are further discussed. During this time, PIs may have specific questions related to the evaluation plan, process, and reporting. Some common questions are noted below. The goal during this step is to build rapport and reinforce evaluator support throughout the process.

Step 3: The Detailed Evaluation Plan

Once a project team has received notification of funding, our team reconfigures the evaluation summary into a detailed evaluation plan document to address updates to the project since submission. The detailed plan also provides a list of evaluation team deliverables and dates.

We use the template shown below, which aligns with EvaluATE’s data matrix template here.

As the project begins, the detailed evaluation plan is revisited on a regular basis to address any changes that occur as a result of extenuating circumstances (i.e., pandemic, staff changes, supply chain problems). Elements that may require changes include data collection methods/instruments (focus groups/surveys), participant groups, and timelines.

As evaluators, we act as partners in helping PIs maintain a clear, focused perspective on grant activities. We find that open and timely communication helps them meet grant goals with confidence. And beyond the grant period, PIs retain a better understanding of the evaluation process, which strengthens their capacity for evaluation during future projects.