What problem does this study address?

Many organizations regularly collect evaluation data about their outreach and training events. Survey data about specific events provides useful information, especially for ongoing improvement. As these data accumulate, organizations run the risk of underutilizing the information to identify trends. While this may not be a “problem,” it is certainly a missed opportunity.

Why is this study important?

This study is exploring an innovative methodology to mine evaluation data to inform program design and development. It will provide a framework for other researchers to utilize to further study unstructured evaluation data. As an additional benefit, this study will produce a harmonized database of survey responses from the past 10+ years.

How will EvaluATE (or others) use the study findings?

The study’s findings will give us new insights on the quality and impact of EvaluATE’s training activities. The study will also feed into the development of an optimized feedback survey for evaluating training activities. We’ll be sharing what we learn so evaluators, trainers, and researchers in other contexts can leverage our lessons learned.

How are the researchers conducting the study?

These methods will provide evidence that will enable us to answer these research and evaluation questions:

  1. How do participants in EvaluATE’s training events (webinars and workshops) perceive the value and impact of training?
  2. What content do participants in training events learn from these events?
  3. What level of relationship exists between emotional content in open-ended responses and close-ended quantitative responses?
  4. What types of survey questions are the most sensitive and accurate to gauge participants’ satisfaction and self-assessments of learning?
  5. What terms and topics in open-ended responses are most highly related with close-ended quantitative responses?

Research Team

Nolen Akerman

Western Michigan University Research Partner

Larry Mallak

Western Michigan University Professor of Industrial and Entrepreneurial Engineering and Engineering Management

Study Findings

The research team created a series of videos to walk you through the sentiment analysis process.

Learn More About SA

You can also view an example of a sentiment analysis visualization by Nolen Akerman, co-investigator in this research.

Example of SA

Nation Science Foundation Logo EvaluATE is supported by the National Science Foundation under grant number 2332143. Any opinions, findings, and conclusions or recommendations expressed on this site are those of the authors and do not necessarily reflect the views of the National Science Foundation.