Because of the large volume of participants in MOOCs and large on-campus courses, it is difficult for instructors to connect with all students and to receive and manage valuable feedback on how to improve the course. M-CAFE(The MOOC Course Assessment and Feedback Engine) combines the citizen report card method and the interactive elements of Opinion Space to enable timely feedback on course quality between students and instructors.
M-CAFE provides student feedback to instructors periodically during the course, in contrast to existing course evaluations that only request feedback after the course is over. This mobile-friendly platform encourages students to check in weekly to numerically assess their own performance, provide textual ideas about how the course might be improved, and rate ideas suggested by other students. It encourages students to assess and track their own motivation, enthusiasm, and performance over the duration of the course. For instructors, M-CAFE displays ongoing trends and highlights potentially valuable ideas based on collaborative filtering.
M-CAFE differs from standard surveys in that it is online (desktop and mobile) and ongoing (weekly updates during the course) and allows students to consider and rate the ideas of fellow students. The peer-to-peer collaborative filtering approach aims to overcome the scale issue of large size MOOCs and on-campus courses by ranking the ideas.
M-CAFE v2.0 was launched in Aug. 2015 with refined graphics and text, tagging of ideas, a complementary instructor dashboard with real-time data, and a time-value consideration when exposing ideas.
M-CAFE integrates elements of two prior projects: citizen report cards developed by the World Bank and the Opinion Space project (http://opinion.berkeley.edu/) developed by researchers and designers at the CITRIS Data & Democracy Initiative.
For more information, please check out our recent publication.
While MOOCs have numerous educational benefits, low student retention rates point to a systemic problem in meeting the needs of diverse students (Adamapoulos, 2013; Clow, 2013). Interactive online tools such as discussion forums and instructor assessments have been found to increase student engagement and likelihood of course completion (Adamapoulos, 2013; Rovai, 2007). However, these tools are often not directly integrated into the MOOC platform or are employed only at the completion of the course. In order to encourage greater student engagement and retention, we have designed M-CAFE, which will allow ongoing feedback from students to instructors and other students on the quality of the course throughout the duration of the MOOC. The M-CAFE platform can be accessed at http://opinion.berkeley.edu/mooc/mobile/.
1. Monitoring MOOCs: Which information sources do instructors value? Kristin Stephens-Martinez, Marti Hearst, and Armando Fox. Proceedings of the first ACM conference on Learning @ scale conference. Atlanta, GA. March 4-5, 2014.
2. Self-evaluation in advanced power searching and mapping with Google MOOCs. Julia Wilkowski, Daniel Russell, and Amit Deutsch. Proceedings of the first ACM conference on Learning @ Scale conference. Atlanta, GA. March 4-5, 2014.
3. What makes a great MOOC? An interdisciplinary analysis of student retention in online courses. Panagiotis Adamopoulos. Thirty Fourth International Conference on Information Systems. Milan, Italy. Dec. 2013.
4. Using a Social Media Platform to Explore How Social Media Can Enhance Primary and Secondary Learning. Sanjay Krishnan, Yuko Okubo, Kanji Uchino, and Ken Goldberg. Learning International Networks Consortium (LINC) 2013 Conference. MIT, Cambridge, MA. June 2013.
6. MOOCs and the funnel of participation. Doug Clow. Third Conference on Learning and Analytics Knowledge (LAK) 2013 Conference. Leuven, Belgium. April 2013.
M-CAFE 1.0: Motivating and Prioritizing Ongoing Student Feedback During MOOCs and Large on-Campus Courses using Collaborative Filtering. Mo Zhou, Alison Cliff, Sanjay Krishnan, Brandie Nonnecke, Camille Crittenden, Kanji Uchino, Ken Goldberg. Proceedings of the 16th Annual Conference on Information Technology Education, SIGITE ’15, Chicago, September, 2015. ACM. [.pdf]
M-CAFE: Managing MOOC Student Feedback with Collaborative Filtering. Mo Zhou, Alison Cliff, Allen Huang, Sanjay Krishnan, Brandie Nonnecke, Kanji Uchino, Sam Joseph, Armando Fox, Ken Goldberg. Proceedings of the 2nd ACM Conference on Learning At Scale (L@S 2015), Vancouver, BC, Canada, March. 2015. [.pdf]
Opinion Space: A Scalable Tool for Browsing Online Comments. Siamak Faridani, Ephrat Bitton, Kimiko Ryokai, Ken Goldberg. ACM International Conference on Computer Human Interaction (CHI). Atlanta, GA. April 2010.
Distributed Spectral Dimensionality Reduction for Visualizing Textual Data. Sanjay Krishnan and Ken Goldberg. International Conference on Machine Learning (ICML) Workshop on Spectral Learning Methods, Atlanta, GA, June 2013.