Massive open online course (MOOC) learners vary widely in background and learning objectives. MOOCs are still trying to understand their learners better and provide content that is more suited to their needs. To understand the likes and dislikes of MOOC learners at a general level, we partnered with FutureLearn to study the responses to post-course survey questions from over 800 courses on its popular social learning platform. We analyzed open-ended questions that asked the learners about their most and least favorite parts of the course and ideas for course improvement.

We used Latent Dirichlet Allocation (LDA) topic model, a widely used statistical approach for exploratory analysis of large textual data, to identify underlying topics from the responses to each question. The LDA model represents each topic as a collection of closely related words and each document as a mixture of topics with associated weights. The topics identified were assessed qualitatively for coherence and relevance. Some of the main themes that emerged were:

Lecture videos: Many learners mentioned videos as their favorite part of the course. They said that they could understand the course content better and at a greater speed with videos because the information was presented to them by expert instructors in a clear and structured manner. Some learners suggested using real-life examples in lectures and providing subtitles in different languages.

 Social interaction: A lot of learners greatly enjoyed interacting with peers and instructors on discussion forums and wanted more avenues of interaction, such as live sessions and chat messengers. Some learners, however, did not enjoy online discussions because of a lack of time or the limitation of the medium, which they said led to misinterpretation of thoughts.

 Evaluations: Many learners wanted more quizzes and assignments, as these helped them to consolidate and retain information, and validate their understanding. While many learners were fine with peers evaluating their work, some would have preferred their work be evaluated by teaching assistants.

 Access to learning material: Easy access to learning material for the course and further study was highly valued by many learners. Some learners suggested uploading access-restricted journal articles and providing specific learning material.

 Time commitment: Many learners mentioned that they wanted a reasonable estimate of the time commitment at the beginning of the course, as they felt that they spent more than the expected time. They suggested that the syllabus clearly mention the prerequisites, and the list and depth of topics to be covered in the course so that they could decide and budget their time accordingly.

Lesson Learned

Overall, we gained meaningful insights about likes and dislikes of MOOC learners, which will help MOOC providers to design their courses better. We found topic modeling to be an efficient method for exploratory analysis of large and unseen textual data such as discussion forums and open-ended responses. However, it is important to note that the topics generated by the model may not always be interpretable. Therefore, qualitative assessment of topics is necessary. MALLET is an easy-to-use toolkit that can be used to start exploring topic modeling, with the help of this tutorial.

This work was made possible by the National Science Foundation (NSF) (PRIME #1544259). Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of NSF.


About the Authors

Kerrie Douglas

Kerrie Douglas box with arrow

Assistant Professor, Purdue University

Dr. Kerrie Douglas is an assistant professor of engineering education at Purdue University. She studies how to make inferences about student learning in online environments and how to use assessment to improve the quality of the learning experience in online courses.

Gaurav Nanda

Gaurav Nanda box with arrow

Postdoctoral Research Assistant, Purdue University

Dr. Gaurav Nanda is a postdoctoral research assistant working with Dr. Douglas in engineering education at Purdue University. His research interests include the application of text mining and machine learning methods to analyze unstructured textual data. Currently, he is studying learners’ experiences in massive open online courses by applying text mining approaches to user-generated data such as discussion forums and open-ended feedback.

Creative Commons

Except where noted, all content on this website is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.

Related Blog Posts

Nation Science Foundation Logo EvaluATE is supported by the National Science Foundation under grant numbers 0802245, 1204683, 1600992, and 1841783. Any opinions, findings, and conclusions or recommendations expressed on this site are those of the authors and do not necessarily reflect the views of the National Science Foundation.