What problem does this study address?

For EvaluATE to do its job well, we needed a comprehensive and accurate understanding of what ATE evaluation work really entails. To that end, this study identified the core tasks that are conducted as part typical ATE evaluations.

Why is this study important?

This study built on the work of the American Evaluation Association (AEA) and other organizations that have developed taxonomies of evaluation competencies. The AEA competencies served as the foundation for the ATE evaluation tasks. This study’s findings are helping EvaluATE translate general evaluator competencies to the specific knowledge and skills needed by ATE evaluators and project staff.

How will EvaluATE (or others) use the study findings?

We are using the validated ATE evaluation task list to plan and prioritize EvaluATE’s training activities and resource materials. Our aim is to provide guidance for all core evaluation tasks.

Others engaged in evaluation capacity-building and research on evaluation can use our framework as a starting point for similar validation work in other evaluation contexts.

How did the researchers conduct the study?

We used two methods in this study:

  • Task Tracking: We asked ATE evaluators and principal investigators to track their evaluation-related tasks for one year. We used the task-tracking data to revise the task list to ensure it’s a comprehensive and accurate accounting of what’s involved in a typical ATE evaluation.
  • Delphi Study: We asked people who are recognized as experts in evaluation to review and provide feedback on the tasks and their definitions. We used the experts’ feedback to finalize the task list.
  • Equity, Diversity, and Inclusion (EDI) Review: Prior to finalizing the task list, our researcher partners from EvaluATE’s Measuring Equity, Diversity, and Inclusion in the ATE Program study review the task list from a DEI lens.

These methods generated evidence to answer the research question: What are the key decisions and actions involved in ATE evaluation?

Who participated in this study?

The research team thanks the following evaluators and principal investigators for their participation in this study: Lola Adedokun, Ann-Claire Anderson, David Antol, Jennifer Bellville, George Chitiyo, Phillip Davis, Amy Germuth, David Hata, Sondra LoRe, Candiya Mann, Craig McAtee, Adam McKee, Megan Mullins, Matthias Pleil, Sandra Porter, David Reider, Lisa Shannon, Julie Shattuck, Pamela Silvers, Keith Sturges, Peggie Weeks, Blake Urbach, and Jill Zande. We thank the following members of the EvaluATE team for their input on draft versions of this document: Lyssa Becho, Emma Binder, Ayesha Boyce, and Tiffany Tovey.

Research Team

Lori Wingate

Western Michigan University

Kelly Robertson

Western Michigan University

Study Findings

Findings from this study have already been used to finalize the Essential ATE Evaluation Tasks, plan and prioritize EvaluATE’s training activities and resource materials, and organize EvaluATE’s resources on this website.

READ AbOUT TASK AREAS

VIEW TASK LIsT

Nation Science Foundation Logo EvaluATE is supported by the National Science Foundation under grant number 2332143. Any opinions, findings, and conclusions or recommendations expressed on this site are those of the authors and do not necessarily reflect the views of the National Science Foundation.