Because EvaluATE is involved in advancing evaluation practice, we are intimately involved in the evaluation of our own work through internal evaluation for accountability and improvement purposes. This involves documenting our processes and outputs and answering questions about user engagement, satisfaction, and immediate learning. EvaluATE also recognizes the necessity and value of working with an external evaluation team. As such, The Rucks Group has worked with EvaluATE since 2012 to be outcome-oriented and address questions regarding sustained learning, use, and impact.
Evaluation Purpose
The overarching purpose of EvaluATE’s evaluation is to use a combination of internal and external evaluation efforts, including feedback from our National Visiting Committee, to generate a comprehensive overview of our work. The evaluation of our activities assists in monitoring progress, identifying opportunities for improvement, assessing our impact, and serving as an evaluation model for the ATE community.
The current evaluation of EvaluATE uses the Kirkpatrick Model and is structured around the following evaluation questions:
- To what extent had EvaluATE implemented its activities? What opportunities or successes are being leveraged? What challenges or barriers have emerged? How have they been addressed? (Implementation)
- To what extent has EvaluATE engaged it’s intended and other audiences? (Engagement)
- To what extent are EvaluATE’s users satisfied with EvaluATE’s activities and resources? (Satisfaction)
- To what extent has EvaluATE’s work led to improvements in users’ knowledge of and attitudes toward evaluation? (Learning)
- To what extent has EvaluATE’s work led to improvements in the ATE program’s overall evaluation capacity by prompting evlauators/PIs/project staff to modify their evaluation practices? (Application)
- To what extent has EvaluATE contributed to improvements of evaluation quality in the ATE program? (Impact)
- To what extent are ATE community members increasingly connected with others in the community for advice and collaboration related to evaluation? (Impact)
Evaluation Reports
Current external evaluation reports:
- 2022 Biennial External Evaluation Survey Report
- 2021 Social Network Analysis Report
- 2012-17 Evaluation Report
Past external evaluation reports:
- 2020 Social Network Analysis Report
- 2020 EvaluATE Stakeholder Survey Results
- 2019 Social Network Analysis Report
- 2018 Evaluation Report
- 2016 External Evaluation Snapshot: Frequency of Use of EvaluATE Resources
- 2016 External Evaluation Snapshot: Self-Reported Learning and Behavior Changes Attributed to EvaluATE
- 2016 External Evaluation Snapshot: What Is EvaluATE Doing Especially Well?
- 2016 External Evaluation Snapshot: What Does EvaluATE Need to Improve?
- 2015 EvaluATE External Evaluation Report
- 2013 EvaluATE External Evaluation Report
- 2012 EvaluATE External Evaluation Report
- 2012 EvaluATE stakeholder survey and focus group results
- 2011 EvaluATE stakeholder survey results
External Evaluation Team
The Rucks Group is a research and evaluation firm that gathers, analyzes, and interprets data to enable their clients to measure the impact of their work. The individuals from The Rucks Group who work on this evaluation include:
Michael FitzGerald, Ph.D.
Senior Research and Evaluation Associate
Michael FitzGerald, Ph.D., has extensive experience developing and applying quantitative and qualitative methods to assess and evaluate programs and initiatives. At The Rucks Group, Dr. FitzGerald is responsible for working on several of the firm’s large and complex evaluation initiatives that require innovative solutions and technical assistance to implement successfully. Prior to joining the firm, he spent 15 years at the Cincinnati Children’s Hospital Medical Center as a field service associate professor. Dr. FitzGerald completed his doctorate in social psychology at the University of Cincinnati and earned his bachelor’s degree in psychology at Miami University.
Kathleen Lis Dean, Ph.D.
Senior Research and Evaluation Associate
Kathleen Lis Dean, Ph.D., provides clients with insights from her extensive experience helping organizations connect strategy, evaluation, and learning for program improvement and impact and began at The Rucks Group in January 2021. Before joining The Rucks Group, she spent 20 years in evaluation and strategic leadership roles at higher education, nonprofit, and philanthropic organizations. Dr. Dean earned a Ph.D. in higher education policy and leadership at the University of Maryland, and both a master’s degree in education and a bachelor’s degree in international relations at the University of Delaware.
Lana J. Rucks, Ph.D.
Principal Consultant
Lana Rucks, Ph.D., brings to her work more than two decades of professional history, including over 15 years of research and evaluation experience. She has led dozens of evaluation initiatives funded by various such as the National Science Foundation, U.S. Department of Education, and the U.S. Department of Labor. Additionally, she has taught at Sinclair Community College, the University of Dayton, and Wright State University’s School of Professional Psychology. Dr. Rucks earned a doctorate and master’s degree in social psychology with a concentration in quantitative methods from The Ohio State University. She also earned a master’s degree in experimental psychology from the University of Dayton. At Ohio Wesleyan University she earned a bachelor of arts degree in psychology with a concentration in chemistry.
EvaluATE is supported by the National Science Foundation under grant number 2332143. Any opinions, findings, and conclusions or recommendations expressed on this site are those of the authors and do not necessarily reflect the views of the National Science Foundation.