1. Q: What’s the motivation for M-CAFE?
A: M-CAFE (The MOOC Course Assessment and Feedback Engine) provides student feedback to instructors at key points during the course, in contrast to existing course evaluations that only request feedback after the course is over. M-CAFE encourages students to check in weekly to numerically assess their own performance, provide textual ideas about how the course might be improved, and rate ideas suggested by other students.
M-CAFE differs from standard surveys in that it is online (desktop and mobile) and allows students to consider and rate the ideas of fellow students to resolve the scale issue of qualitative feedback.
2. Q: Do I have to rate all 5 aspects of the MOOC?
A: No, just click the skip button for aspects where you don’t want to assign a rating.
3. Q: Why do I need to provide answers to the demographic questions?
A: You only need to answer these questions once, and they will allow us to better understand the demographics of this course. We promise to keep your email private, and we will not share it with third parties.
4. Q: In M-CAFE v1.0, after I enter each grade, the Median Grade based on input from all previous participants is revealed. Then I can change my grade: Won’t this bias me and other participants?
A: Revealing the Median values is intended to provide instant feedback in contrast to traditional polls and surveys. We realize this may have a biasing effect: some may adjust their grade to be closer to the average resulting in a form of regression toward the mean. We will include details on the number of such adjustments and the before and after values in all summary reports.
5. Q: In M-CAFE v1.0, how are the mugs arranged on the table?
A: The placement of each participant’s mug is based on how he or she graded the first 5 questions. Think of each issue as one dimension of a 5-dimensional space. The grades you assign define a point in this space. We use dimensionality reduction (currently Principal Component Analysis) to project that space onto the table. If you and another participant assigned very similar grades, your mugs will be close together. Mugs that are distant from yours belong to participants who graded very differently than you.
6. Q: In M-CAFE v1.0, why do I only see 8-10 mugs arranged on the table?
A: To avoid overcrowding, we display only a few mugs at a time for grading. The selection of which participants’ mugs to display is random but biased toward newer mugs, those that have received fewer ratings. Specifically, the likelihood of displaying a mug is based on its standard error = stdev / sqrt(k) where k is the number of times it has been graded so far, and stdev is the variance of those grades. Over time, this ensures that the confidence, or error-bar on the estimated grade for each mug, tends to be the same length for all mugs. Each time you finish grading a set of mugs, the table will be refreshed with more mugs.
7. Q: In M-CAFE v1.0, is there a meaning to the different levels of coffee and tea in the mugs and the orientation of the mug handles?
A: Not yet. We will use those variables in future versions, for example to reflect the score of each participant and the date he or she joined the discussion.
8. Q: In M-CAFE v1.0, why do the the sliders require dragging? If I tap on A, it should go there.
A: We decided to remove the tap feature since as you slide down to see more issues you may accidentally tap the sliders.
9. Q: In M-CAFE v2.0, how do the spheres work?
A: Each sphere represents a participant. Clicking any sphere brings up the corresponding idea/issue that was suggested for the course.
10. Q: In M-CAFE v2.0, how are the spheres arranged in the space?
A: The position of each sphere is based on how the participant rated the five quantitative issues. We use dimensionality reduction to project the space into 3 dimensions. If you and another participant assigned very similar points, your spheres will be close together. Spheres that are distant from yours belong to participants who gave different ratings to the questions.
11. Q: In M-CAFE v2.0, why do I only see 6-7 spheres arranged in the space?
A: To avoid overcrowding, we display only a few spheres at a time, biased toward more recent responses that have not yet received many ratings.
12. Q: In M-CAFE v2.0, what are the hashtags (#) on the spheres?
A: The hashtags (#) on the spheres are the topics of the corresponding ideas/issues. The topic of an idea is specified by the idea owner.
13. Q: What will you do with the grades you collect?
A: The rating data will be analyzed and provided to instructors of the course. We respect student privacy and strive to keep M-CAFE data confidential. Assessments provided through M-CAFE will provide instructors with valuable feedback on how to improve the course.
14. Q: Why are there so few ratings on the issue I suggested?
A: Your rating stats are updated once a day (at midnight). Your suggestion will be displayed and rated as new participants join the discussion. It can take up to a week for enough participants to review and rate your suggestion.
15. Q: Why doesn’t M-CAFE respond as fast as my other phone apps?
A: Technically M-CAFE isn’t an app, you don’t have to download and install it. It’s a website that uses the HTML5 language for compatibility with most desktop, laptop, notepad, and smartphone browsers.
16. Q: How can I learn more about this project, its history, related projects, and the team that built it?
A: Please visit The M-CAFE website for more information.
17. Q: How can I contact the team about a problem or question?
A: For any questions, comments, or concerns please send an email to firstname.lastname@example.org (NOT the course staff) and please do not post your M-CAFE related questions to the course discussion forum.