In 2021, EvaluATE started the Outstanding ATE Evaluation Award program. Three exceptional STEM education evaluators served as judges: Cherie Avent, Cynthia Phillips, and Aileen Reid.

Based on their experience selecting last year’s winner, we asked them, “If you could give one piece of advice to ATE grantees and evaluators about evaluation, what would it be?”

As they reflected on their observations of the strengths and weaknesses across the nominated evaluations, a theme emerged: alignment.

Cherie said:

It is important for the intended program / project goals and the evaluation questions to be aligned. The two mutually influence the evaluation conclusions drawn and decision-making within the program.  

In other words, the questions driving an evaluation should support the project’s purpose. If a project aims to enhance teaching, the evaluation questions must go beyond asking about the number of participants and their satisfaction. That is, at least one evaluation question should ask about the project’s influence on changes in teaching practices.

Following up on Cherie’s comments, Cynthia added:

The methods selected to address the evaluation questions also need to be aligned. By “aligned” I mean fit the purpose and support valid and reliable conclusions.  

So the methods should generate data to help answer the evaluation questions and meet technical requirements for evidence-based conclusions. For example, if an evaluation question asks how the project contributes to students’ workforce readiness, a valid skill assessment is needed.

Aileen said:

Attention to stakeholders’ valuesespecially those least well-served by the programis important throughout all aspects of the evaluation process.

To elaborate, the entire evaluation should reflect what people involved in the project care about. Take time to surface and include perspectives of individuals who may not have had a seat at the table during project or evaluation planning.

Evaluations are essentially a service to the clients who commission them. In the ATE context, those clients are ATE principal investigators and their project teams. In serving their clients, evaluators are professionally obligated to draw on their experience and expertise to ensure and strengthen an evaluation’s quality. That may involve guiding clients to embrace evaluation questions they hadn’t thought of; prompting them to engage people they may have overlooked; and balancing their needs and interests with requirements for technical quality.

While attending to the alignment of evaluation purpose, methods, and values, evaluators must also navigate the practical limitations of time and budget. But alignment doesn’t require extra time or money. In fact, it can enhance efficiency by focusing resources on what matters most.

Alignment shines in the 2021 award-winning evaluation. The judges praised its use of mixed methods and clear alignment of conclusions to data. The report’s organization, clear writing, graphic design, and data visualization support the evaluation’s purpose and help ensure that the client and other users clearly understand the results and how they were derived.

Submissions for the 2022 Outstanding ATE Evaluation Award are now open.

About the Authors

Lori Wingate

Lori Wingate box with arrow

Executive Director, The Evaluation Center, Western Michigan University

Lori has a Ph.D. in evaluation and more than 20 years of experience in the field of program evaluation. She is co-principal investigator of EvaluATE and leads a variety of evaluation projects at WMU focused on STEM education, health, and higher education initiatives. Dr. Wingate has led numerous webinars and workshops on evaluation in a variety of contexts, including CDC University and the American Evaluation Association Summer Evaluation Institute. She is an associate member of the graduate faculty at WMU. She, along with Dr. Kelly Robertson, led the development of The Evaluation Center's online training program, Valeo (valeoeval.com)

Creative Commons

Except where noted, all content on this website is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.

Related Blog Posts

Nation Science Foundation Logo EvaluATE is supported by the National Science Foundation under grant number 1841783. Any opinions, findings, and conclusions or recommendations expressed on this site are those of the authors and do not necessarily reflect the views of the National Science Foundation.