Sentiment Analysis and Topic Modeling of Evaluation Survey Data

Many organizations regularly collect evaluation data about their outreach and training events. Survey data about specific events provides useful information, especially for ongoing improvement. As these data accumulate, organizations run the risk of underutilizing the information to identify trends. While this may not be a “problem,” it is certainly a missed opportunity.

This study is exploring an innovative methodology to mine evaluation data to inform program design and development. It will provide a framework for other researchers to utilize to further study unstructured evaluation data. As an additional benefit, this study will produce a harmonized database of survey responses from the past 10+ years.

The study’s findings will give us new insights on the quality and impact of EvaluATE’s training activities. The study will also feed into the development of an optimized feedback survey for evaluating training activities. We’ll be sharing what we learn so evaluators, trainers, and researchers in other contexts can leverage our lessons learned.

These methods will provide evidence that will enable us to answer these research and evaluation questions:

1. How do participants in EvaluATE’s training events (webinars and workshops) perceive the value and impact of training?
2. What content do participants in training events learn from these events?
3. What level of relationship exists between emotional content in open-ended responses and close-ended quantitative responses?
4. What types of survey questions are the most sensitive and accurate to gauge participants’ satisfaction and self-assessments of learning?
5. What terms and topics in open-ended responses are most highly related with close-ended quantitative responses?