Celebrating Excellence: Insights from the 2023 Outstanding ATE Evaluation Awards Nominations

EvaluATE’s third annual Outstanding ATE Evaluation Awards saw the largest number of nominations yet, and the high level of quality was exciting to see. The judges unanimously agreed to award two ATE evaluations with the 2023 Outstanding ATE Evaluation Award.

Small, New to ATE Project: Integrating Biotechnology and Applied Engineering to Meet Emerging Advanced Technological Workforce Needs

PI: Brian Worley, Johnston Community College
Evaluators: John and Maggie Cosgrove, Cosgrove & Associates

ATE Project: A Collaborative Approach to Work-Based Learning in Biotechnology: Building Inclusive Lab Environments

PI: James Lewis, City College of San Francisco
Co-PI: Karen Leung, City College of San Francisco
Evaluator: Andrea Goldfien

What makes an outstanding ATE evaluation? The answers can be varied of course, but in the two ATE evaluations honored in 2023, four common themes emerged from the nomination testimonials.

1. Value of Evaluations

The award-winning evaluations were instrumental in shaping the direction of projects, understanding successes, and identifying areas for improvement.

Andrea Goldfien’s evaluation reports were recognized by PI James Lewis for being enlightening. “They stand out for their clarity and organization,” Lewis wrote. “Tables and graphs are both informative and intuitive, while the inclusion of participants’ quotes adds a dimension that vividly captures the project’s impact.”

Goldfien’s work with the project team helped them to recognize the integral role of evaluation in the development of grant projects. “The evaluation is not just a validation tool, but a compass that guides a project’s evolution,” said Lewis. “It is a continuous feedback loop, integral to the project’s lifecycle.”

Recognizing the value of evaluation had the added bonus of providing a positive shift in perspective. Both nominations acknowledged a move from an initial apprehension of evaluation to viewing it as a valuable tool for improvement. “The evaluation process was comprehensive, useful, and even enjoyable,” said project team member Daphne Lewis. “I firmly believe that several members of our team are no longer fearful of evaluation and look at it as a way to improve and promote the program.”

2. Utilization of Tools and Frameworks

Both sets of testimonials highlight the project teams’ use of tools and frameworks provided by the evaluators. These tools contribute to tracking progress, assessing outcomes, and informing future project decisions. Evaluation findings can help solidify a project’s objectives, allowing for agility and adaptation.

Cosgrove & Associates were praised for developing a self-assessment tool, which included the project’s stated outcomes and collected responses up to three times per year. “This tool included questions regarding project accomplishments, what was learned, challenges, actions, taken, and more,” said project team member Julie Griffith. “This information was used for assessment as we checked in on our grant activities and outcome progression. Our project team was also able to use this data to complete the NSF Annual Report, resulting in a more efficient use of our time.”

James Lewis echoed the value of tools and frameworks in data collection when reflecting on Goldfien’s evaluation. “She pursued in-depth interviews, gleaning qualitative insights that spotlighted areas of potential improvement.” Using evaluation findings, Lewis’ project shifted their focus to group participation, embarking on a study of workgroups from specific research labs and biotech companies who attended the workshops together.

3. Continuous Improvement

The importance of continuous improvement is a common theme. Both sets of testimonials stress the idea of learning from evaluations and applying those lessons to enhance project effectiveness over time.

In her nomination testimonial, project team member Naledi Saul acknowledged how the evaluation data helped shift the program’s direction and the team’s thinking. “For years, I had taught variations of mentoring professional skills and always believed that participants primarily appreciated the tailored case studies and high levels of interactivity over the lecture,” she explained. “But the interviews and surveys revealed that what participants truly valued were the tools we had developed and introduced. Participants used these tools when faced with real-world situations and frequently shared them with colleagues.”

Both projects made significant shifts in their approaches based on their evaluation, including:

• Redesigning training modules to focus on evidence-based scenarios.
• Updating the project website to highlight tools that workshop attendees found useful.
• Refining social media activity to reflect audience preferences.
• Extending the length of internships to satisfy industry partners and students.
• Pivoting away from certain software platforms whose cost would be unsustainable.

4. Engagement of Evaluators

Engaging evaluators in regular project meetings is a practice noted in both sets of testimonials. This engagement is valuable for capturing new insights, identifying areas for evaluation, and fostering collaboration.

Frequent communication helped the project’s evaluators understand the unique intricacies involved, adding to the depth of evaluation findings. Approaching the project as collaborators and team members also helped the project team appreciate and trust the evaluation process and findings.

The award-winning evaluation teams stayed connected in different ways, including meetings, site visits, evening check-ins with students, and assessment tools. Both were credited with a holistic approach to evaluation which facilitated successful project completion.

The testimonials emphasize the profound impact of evaluations, underscoring their role not just for accountability or compliance, but as guiding compasses integral to a project’s lifecycle. As we celebrate these outstanding achievements, it is evident that the collaborative efforts between project teams and evaluators have not only enhanced the quality of these ATE projects but have also contributed to a broader shift in perspective – from apprehension to an enthusiastic embrace of evaluation as a catalyst for improvement and program promotion. These award-winning evaluations serve as inspiring examples, demonstrating the transformative power of thoughtful evaluation practices in advancing education.