The Monash Critical Thinking Study
5. Peer Instruction
Description
In semester one 2006, we incorporated elements of Eric Mazur's Peer-Instruction methodology into the critical thinking course (Mazur 1997). The technique involves a modification to the way in which lectures are delivered. The lecturer gives a short 10-15 minute mini-lecture on a key concept or topic, then presents one or more multiple choice questions (called ConcepTests) which probe students' understanding of that concept. These questions are often designed to test common student misconceptions about the topic under discussion.
Students are given a minute or two to think about their answer, then every student in the class indicates their answer. This can be done in a number of ways; show of hands, holding up large printed flashcards or using an electronic response system. The lecturer then indicates to the class the percentage of students who have given each answer; for example, 60% have answered 'A', 20% have answered 'B' and so on. (With an electronic response system this is done automatically and the results presented in the form of a bar chart).
If most students respond correctly, the lecturer confirms it and moves on. If most respond incorrectly, the lecturer can explain the idea again and then reassess with another question. If about 40-60% respond correctly, students are given a short time (2-3 minutes) to discuss the question with their neighbour and try to persuade them that their answer is correct. The question is then asked a second time. Typically, more students give the correct answer the second time around; students with the right answer usually convince others of it. The lecturer then moves on, either to another question or to the next topic and the process is repeated (Mazur 1997, 10-11).
The advantages of these interactive lectures over the traditional format are many and varied:
- The ConcepTest questions provide the lecturer with instant feedback about how well students have understood the material, allowing them to adjust the pace and content of their teaching accordingly.
- The questions provide valuable feedback to the student on how well they have actually understood the material and how they are progressing relative to the rest of the class.
- The "convince-your-neighbour" sessions allow for valuable peer interaction between students that promotes active engagement and critical reasoning: students move beyond passive assimilation of the material in the effort to think about and explain it to someone else.
- The convergence on the correct answer following these student-to-student discussions suggests that brief one to one discussion is an invaluable learning tool. Students who have understood the topic are able to explain it effectively to students who have not, perhaps at times more effectively than the lecturer.
- The anonymous nature of the voting system encourages participation by, not just some, but all students.
- The monotony of the traditional lecture is avoided by breaking up the lecture into short segments interspersed with a sequence of questions in which students must actively engage with the material. In this way, student concentration is increased.
Mazur and many other science educators have developed this technique over the last ten years, achieving remarkable results. Studies have consistently recorded impressive gains in students' understanding of physics concepts and significant improvements in quantitative problem solving. Student surveys show that student satisfaction increases as well. (Mazur 1997, Crouch and Mazur 2001, Hake 1998, Meltzer and Manivannan 2002).
Procedure
Approximately 4-6 questions were created for each of the twelve lectures in the critical thinking course. We used large, laminated flashcards as the student response mechanism. Each student was provided with a set of cards labelled from 'A' to 'D'; the students indicate their answer by holding up the appropriate card so that the lecturer can see them. The majority of ConcepTest questions were used in the lectures. Three or four questions were used in tutorials - typically these were questions planned for the lecture but not used due to lack of time.
Students were pre- and post-tested using the CCSTS and GSA. Test forms were distributed randomly at the pre test and students were given the opposite form for the post-test. In addition, a questionnaire was distributed to all students, asking for their assessment of the use of ConcepTest questions in lectures.
Click here for a sample of peer instruction questions.
Results
Students showed a statistically significant increase in critical thinking skills, as measured by the CCTST.
Effect size: 0.4 standard deviations (n = 40). Significant at the 0.01 level.
Students showed no improvement on critical thinking scores on the GSA.
The feedback from the questionnaire survey was very positive. 88% of students said they found the method beneficial. Many students commented on the positive effects of the method on attention, the provision of feedback and improved understanding.
Sample characteristics
|
Semester 1, 2006 sample (PI) | |
| Sample size | 42 |
| Sex | Male: 23 (54.8%) Female: 19 (45.2%) |
|
Age (Years) | Range: 17 (1) - 26 (2) Median: 19 Mode: 18 (28.6%) |
|
Year level |
1st Year: 21 (50%) 2nd Year: 6 (14.3%) 3rd Year: 5 (11.9%) 4th Year: 1 (2.4%) |
| Faculty | Arts: 24 (57.1%) Arts + (Law, Education, Economics, Commerce): 5 (11.9%) Science: 3 (7.1%) Business/Economics: 2 (4.8%) Engineering: 2 (4.8%) IT: 2 (4.8%) Medical science: 1 (2.4%) |
Gains on critical thinking tests
|
CCTST (Max. score = 34) | |||
| N = 40 | Mean | 95% confidence interval | Standard deviation |
|
Pre-test |
21.15 (62.2%) | [19.53, 22.77] | 5.0614 |
| Post-test | 22.95 (67.5%) | [21.04, 24.86] | 5.965 |
| Gain | 1.8 (5.3%) | [0.69, 2.91] | 3.48 |
| Effect size | 0.40 | [0.16, 0.65] | |
| Proportional gain | 17.37% | [8.79, 25.95] | 26.83 |
| GSA (Scaled scores) | |||
|
N = 39 |
Mean | 95% confidence interval | Standard deviation |
|
Pre-test |
460.026 | [431.839, 488.213] | 86.953 |
| Post-test | 485.718 | [450.784, 520.652] | 107.768 |
| Gain | 25.692 | [-3.985, 55.3699] | 91.552 |
| Effect size | 0.25 | [-0.039, 0.539] | |
| Proportional gain | 6.87% | [-0.256, 14.003] | 21.995 |
Effect sizes calculated using pre-test standard deviation estimates of 4.45 CCTST points and 102.76 GSA (scaled) points.
Proportional gain is the gain score score expressed as a percentage of how many points a student could have gained (or lost) relative to the maximum test score (or pre-test score if gain was negative).
Student evaluations
A total of 34 students completed the questionnaire at the end of the semester. Results from four key questions on the questionnaire are shown below.
| The use of the multiple-choice questions and flash cards helped me to understand the material when I attended lectures. |
|
|
| The use of the multiple-choice questions and flash-cards made the lectures more interesting. |
|
|
| The discussions with fellow students helped to improve my understanding of the topic. |
|
|
| What (for you) was the balance of benefit vs. disadvantage from the use of the multiple-choice questions and flash cards in the lectures? |
|
|
Some typical comments
Interaction and engagement
It made lectures interactive and also gave me an opportunity to demonstrate if I had or hadn't grasped the material just taught and then if so, to clarify them … they're fun and interesting!
I enjoyed being able to test my knowledge and ability to apply that knowledge right there in the lecture. It also broke up the lecture and gave us something to actively do instead of just passively listening and taking in the information. ... I wish more lecturers would use this method - well done!
Improved understanding
They consolidated the content of the lecture and helped build confidence that the things I was taking from the lecture were the right ones. ... They should be done in more classes.
They were a good way to reinforce the theory we were learning in the lectures. I wanted to do more of the multiple choice questions I felt they helped me understand the course material and gave me an opportunity to practice my critical thinking skills.
Feedback to students and lecturer
Cleared up misunderstandings. Sometimes I thought I understood and the flash cards showed that I hadn't.
Allowed me to see how I was doing in comparison with others in the class. I was not put down by being wrong, because I could see that at least I was not the only one who answered in a particular way. Also allowed the lecturer to address where and why people went wrong in their reasoning without singling out individuals. .... I hope it continues to be used, perhaps it will spread to other subjects...
Attention
Made me more alert during lectures. Helped sustain my attention and interest during the lecture. … Good, innovative approach.
Broke up lectures a bit, kept me alert and focusing on the material.
Comparison with other studies
 |
| Gains for all studies measured using the CCTST. |
| * Two semester course. |