Beyond Reporting: Getting More Value out of Your Evaluation*

If you’ve been a part of the ATE community for any time at all, you probably already know that ATE projects are required to have their work formally evaluated. NSF program officers want the projects they oversee to include evaluation results in their annual reports.  

What may be less well known is that they also want to hear how projects are making use of their evaluations to learn from and improve their NSF-funded work. Did your evaluation results show that an activity you thought would help you reach your project goals turned out to be a flop? That may be disappointing, but it’s also extremely valuable information.   

There is more to using evaluation results than including findings in your annual reports to NSF or even following your evaluators’ recommendations. Project team members should take time to delve into the evaluation data on their own. For example: 

Read every comment in your qualitative data. Although you should avoid getting caught up in the less favorable remarks, they can be a valuable source of information about ways you might improve your work.  

• Take time to consider the remarks that surprise you—they may reveal a blind spot that needs to be investigated.  
• Don’t forget to pat yourself on the back for the stuff you’re already getting right.  

It’s important to find out whether a project is effective overall, but it can also be very revealing to disaggregate data by participant characteristics such as gender, age, discipline, enrollment status, or other factors. If you find out that some groups are getting more out of their experience with the project than others, you have an opportunity to adjust what you’re doing to better meet your intended audience’s needs. 

The single most important thing you can do to maximize an evaluation’s potential to bring value to your project is to make time to understand and use the results. That means:  

Meet with your evaluator to discuss the results.  

• Review results with your project colleagues and advisors. 
• Make decisions about how to move forward based on the evaluation results.  

• Record those decisions, along with what happens after you take action. That way, you can include this information in your annual reports to NSF. 

ATE grantees are awarded about $66 million annually by the federal government. We have an ethical obligation to be self-critical, use all available information sources to assess progress and opportunities for improvement, and use project evaluations to help us achieve excellence in all aspects of our work.  

*This blog is based on an article from an EvaluATE newsletter published in October 2014.