The Monash Critical Thinking Study
Background
What is Critical Thinking?
Critical thinking is a set of skills and dispositions, essentially involving the ability to analyse and evaluate real arguments in natural language. More specifically, we define critical thinking to include the following set of skills and related dispositions:
- Argument analysis
-
- Clarify the meaning of claims
- Identify arguments
- Identify premises and conclusions in arguments
- Identify implicit premises
- Identify the structure of arguments
- Argument evaluation
-
- Assess the plausibility of statements
- Assess explanations
- Evaluate arguments
- Make valid inferences
Other terms that are often used for critical thinking in this sense are:informal reasoning, informal logic and critical reasoning .
How good are people at critical thinking?
Evidence from a number of different sources suggests that the answer is: not very good.
1. Studies showing that many people find it difficult to provide reasons or arguments to support their beliefs
There is evidence that many people have great difficulty in understanding the concept of an argument - the process of giving reasons for a belief that they hold. (Kuhn 1991). Subjects in Kuhn's study were asked for their opinions on the underlying causes of various social problems, such as:
- What causes prisoners to return to crime after they're released?
- Why do children fail in school?
- What causes unemployment?
Subjects had no difficult in offering opinions on these subjects. However, when asked to give reasons or evidence to support their views, most were unable to do so. Typically, they were able to do little more than simply restate their opinion in different words.
2. Studies that reveal the existence of systematic biases in reasoning
There is a great deal of empirical evidence for systematic errors and biases in human reasoning. (Kahneman et. al. 1982, Baron 1994). These biases include:
- Insensitivity to base-rates and sample size
-
People appear to assess probabilities in terms of conformity to stereotypes, ignoring base rate information, even when it is available.
- Belief in the 'law of small numbers'
-
Overestimation of the representativeness of small samples.
- Confirmation or myside bias
-
The tendency to weigh arguments in support of an accepted belief more strongly than arguments against.
- Outcome/hindsight bias
-
The tendency to evaluate decisions and predictions in terms of their outcomes, rather than in terms of the evidence available at the time.
These biases are robust and widespread and even experts are prone to them.
Can critical thinking be improved through teaching?
Given that people typically are not very good at informal reasoning, the question arise whether there is anything that can be done about it. Can reasoning be improved through teaching? The evidence here is mixed. Since our focus is on university level teaching, we review here some of the data on the how university affects critical thinking skills.
1. A university education is associated with better critical thinking
Kuhn (1991) found that people with a university education were better at informal reasoning. In particular, philosophy graduate students interviewed in her study performed much better than the average, as one might expect. By itself, this is very weak evidence of the effect of university of course, since philosophy graduate schools select for high level skills in reasoning and argument.
However, a more detailed analysis of the available evidence does support the view that a university education has an effect on critical thinking skills over and above that which can be accounted for by maturation and selection or attrition effects. In their review of studies, Pascarella and Terenzini (2005) estimated that the first three years of university provide an improvement of about 0.55 standard deviations, or 20 percentile points in critical thinking, most of the improvement occurring in the first year.
Other studies have shown that university level teaching can lead to improvements in specific reasoning skills, though the effects are not uniform across disciplines. For example, one longitudinal study of U.S college students (Lehman & Nisbett, 1990) found that four years of undergraduate training in social science subjects lead to large improvements in statistical and methodological reasoning (70%) compared to natural science and humanities subjects (25%). (Figure 1).
The same study found that four years of undergraduate training in natural science or humanities subjects leads to improvement in conditional deductive reasoning (60%) compared to social sciences or psychology (0%). (Figure 2).
|
|
|
|
| Figure 2 |
2. Dedicated instruction can improve critical thinking
What about courses that are specifically aimed at improving informal reasoning? Again, the evidence is mixed. A review of 27 studies of college courses designed to enhance informal reasoning found no evidence that specific courses or instructional techniques lead to any improvement. (McMillan 1987, see also Pascarella and Teremzini 1991, McKeachie et. al 1986.) However, more recent studies suggest that some critical thinking courses do work. Figure 3 shows data from a number of university courses aimed at improving critical thinking. Students in all these studies were pre- and post-tested using the a standardized test of critical thinking, the California Critical Thinking Skills Tests (CCTST, Facione 1991, 2002). Effect sizes for gain scores range from 0.32 to 0.89 standard deviations, corresponding to average gains from pre- to post- test of 4.1 to 11.8 percentage points on the CCTST.
|
|
| Figure 3. Comparison of critical thinking courses. Bars show 95% confidence intervals for gain in
standard deviations. |
These effect sizes are comparable to the estimated for the improvement due to a full three years of university education (0.55 standard deviations). Still, even the best courses lead to gains that are not the best one might hope for. Figure 4 shows the improvement of the students in each of the above studies as a percentage of how much the class could have improved, given the average pre-test score for the class. Even the biggest effect (0.89 standard deviations) obtained at Melbourne university, corresponds to an class average improvement of just 24%.
|
|
| Figure 4. Percentage gains for critical thinking courses |
