John Hattie and the controversial meta-analysis synthesis

Mention John Hattie’s name and a variety of responses will come, ranging from disgust, dismissal and anger through to acceptance, interest and resounding praise for the man and his work. These strong reactions (and having more time) have led to me visiting his research myself. 

Recently I began rereading* John Hattie’s Visible learning: A synthesis of over 500 meta-analyses relating to education. I admittedly am only up to chapter two and a bit so far, but it has me very intrigued.

One argument against Hattie’s work is that it addresses the general, but that isn’t what happens on the ground – there are other complexities and moderators involved in day to day practice. I noted that he directly addressed this in his book, arguing that the data revealed that the moderators were indeed minimal and that he found less than he expected.

Two examples have often been touted in general education (both of which are evident in Hattie’s results). One is spoken about favourably: Homework has a minimal impact on primary school (grade school) children, so what is the point in giving it? The second was the second is frequently received with disgust and anger: Class size doesn’t matter (Please note that this actually refers to class size doesn’t matter if the teaching method remains unchanged – a significant difference in meaning). It seems to me that so often teacher’s opinions of Hattie are based on the poorly reported class size finding. Yet, in many ways it is a symptom of a much larger issue.

I hear and read of so many frustrations in the NZ education sector (and okay… experience these too) because of the state of the system in NZ and how it at the whim of the government – a government which has created a rhetoric of crisis for the purposes of forwarding their political agenda**. So many are concerned about the direction of our education and what is being lost in the name of improving our international rankings. But if our focus has become defending the education system that we had, we risk silencing the real issues that exist. Our system worked brilliantly for some and terribly for others – statistically that is. What about the wider social issues that we face?

How do we focus on addressing the existing issues when we feel the need to defend what was? How does ‘transformation’ come about? Conversely, how would change come about if things were left as they were? On the same note, how does one move forward and address the past when one constantly runs to stay in the same place?

Political section over! That wasn’t meant to slip in there, but I think I’ll leave it in anyway. This post seems to have shifted away from Hattie’s work, and I will return there presently. Key points that I have picked up so far include

  • Quality, accurate feedback impacts student learning
  • Hattie’s effect size is slightly different than Cohen’s d (0.2, 0.4 and 0.6 are key points rather than 0.2, 0.5 and 0.8).
  • An effect size of 1.0 is equivalent to a year’s educational growth
  • 0.2 means what was measured would have a positive impact on 20 out of 100 people.
  • The quality of the study was catered for as part of the analysis.
  • Although nearly anything works, this should not become the benchmark for improvement.
  • Hattie was involved in the creation of e-Asttle.


Recently I participated in a debate on the ethical defensibility of Milligram’s experiment investigating power and authority. A point that was raised (and I have unfortunately thrown out my notes so cannot reference it) was whether it would have been controversial if the participants acted the way we expected them to. Indeed, was it so shocking because of what it revealed about human nature?

Perhaps we need to pose the same question when we react to research. Is it so shocking because what it says is contrary to what we expect and ‘know’?

I look forward to reading more of Hattie (although I suspect it will be slow going). I bought the e-book so am determined to actually read it!



*I say rereading but really the first time round I only got 4% through the ebook before giving up – so shout out to Doctor Mira Peter who, in an unrelated lecture, managed to revive and increase my statistical vocabulary enough to meaningfully read and understand the book.

**Isn’t that part of a politician’s job, regardless of political affiliation – create the crisis so you can be the solution and stay in power? Perhaps I am being too bleak, but I would like to note that my position is not based on one political party who happens to be in power,  the impact I see of policy and this rhetoric and where I foresee education in NZ heading in the globo-politcal context.




Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s