Example #3: Quiz Case Study

Professor March’s course has a midterm exam that consists of multiple choice questions in Canvas quizzes. She sees that everyone in the course has done very well on this exam.

However, there is a far greater range of scores on the research paper that students submit during Week 12 of the term. 

In New Analytics, she can see a high level snapshot of the classes performance on the midterm and the research paper, which confirms that the class did very well on the midterm but performance varied on the research paper.

Midterm Exam

see more data about individual assignments

This strikes her as odd given that the research paper builds on knowledge assessed in the quiz.

Review

Professor March decides she needs to review quiz statistics, another way to find data about student learning progress. 

As Professor March is looking at the quiz statistics, she notices that the Discrimination Index for 5 out of 8 questions is zero.

Discrimination Index ZERO

Amend

Professor March suspects she needs to:

  1. Redraft some quiz questions
  2. Address the disconnect between the midterm and the research assignment

The learning data is suggestive but she’s not entirely sure what the takeaway is, so she makes an appointment with her college’s academic technologist.

Bloom's Revised TaxomonyProfessor March brought these findings to her college’s academic technologist, who points out that the midterm questions landed in the Understand and Remember sections of Bloom’s Revised Taxonomy, whereas the research paper expects more analytical and critical thinking and writing. They determine that re-writing some of the quiz questions to align quiz questions to the more rigorous learning goals exemplified by the research paper is a goal for the next semester.

A complementary approach is to make sure that students understand the connection between how the concepts in the exam form the foundation of the research paper. To do this, Professor March decides to try an exam wrapper: These short pre and post quizzes ask students to reflect on their own practices and understanding; they also create a feedback loop that gives her insight into her student’s preparation and understanding. Ongoing, this information will help her know what kind of support students need to succeed.

Apply

These changes will be implemented the next time Professor March teaches the course:

  • She will also rewrite some of her midterm exam questions.
  • The “exam wrapper” concept will be introduced to students early in the semester in connection to a low-stakes quiz. She will also use an exam wrapper for the midterm. (Once she gauges how useful this approach is, she might scale these efforts.)

Related Resources