BBC’s More or Less programme on Radio 4 aims to explain – and sometime debunk – the numbers used in political debate, the news and everyday life.
On Friday 5th February, their episode entitled “E-cigarettes…” also included a segment on the term time holiday. Having looked at the figures for the past few years, it didn’t occur to us that the programme wouldn’t debunk the government’s misunderstanding and misuse of their stats on attendance and attainment and we were right, as summarised by these quotes:
“Nick Gibbs is saying the absence is the thing which is causing the drop off in grades but the evidence doesn’t show this at all.” said Tom Melly, segment producer.
“There is an association between the proportion of absence and the aggregate level of attainment of students who’ve had that level of absence but it would be wrong to assume that it was necessarily causal. We don’t know that the absences are the reason for the lower attainment. They could both be indicators of something else such as background characteristics and of course it’s also possible that children who aren’t do well at school after a time begin to drift away and perhaps take time off. It could be that the causal mechanism is the other way around” Stephen Gorrard, Professor of Education at Durham University
The programme merely scratches at the surface of the flaws in the Department for Educations evidence base but we’ll avoid the temptation to go an and on about it now. You can find the programme here: http://www.bbc.co.uk/programmes/b06zcg4v or on podcast, with the bit on the absence/attainment stats starting around the 16.16 minute mark.
In case for any reason you can’t access that, here’s a transcript we’ve produced, so any errors are all ours:
Presenter: Tim Harford
TH: In 2013 the Government tightened the rules around parents taking their children out of school during term time, arguing that time away from class damages the child’s education. The policy’s proven to be controversial, with thousands of parents being fined, many arguing that the only way they can afford to take their children on holiday is in term time when it’s cheaper. In October the Local Government Association called for the rules to be loosened but education minister Nick Gibb came out fighting on the Today programme.
[Clip of interview from the Today programme:]
Nick Gibb: Our data shows that just a week off per year as they’re leading up to GCSE courses can reduce the chances of that child getting good GCSEs by about a quarter…
Interviewer (interrupts): Regardless of what of they do?
Nick Gibb: Yes, it does, any absence, even if it’s illness actually, can damage the long term chances of a child achieving good GSCEs if they take just a week off a year. Now, you can’t do anything about illness – if a child is ill of course they can’t come to school – but we can do something about parents who decide to take their child on holiday simply because they want to save money. “
TH: Well, that prompted many listeners to ask “Is it true that a week off school can reduce a child’s chances to get good GCSEs by a quarter?” Well, I’m joined in the studio by More or Less producer James Melly. Hello James.
JM: Hello there.
TH: Where did this statement come from?
JM: Well, last February the Department for Education released some research looking at the relationship between absence and achievement. It looked at kids in the two years they are doing their GCSE courses and compare their results to their absence rates. The kids with a perfect attendance record would have gone to school for 283 days without missing a day and if you look at their results, surprise, surprise, they go for the best results.
TH: OK, so enough about these goodie-goodies, what about the children who took a week’s holiday?
JM: Well, the government doesn’t have any data on these kids. The data they focus on is the kids who miss between 5 and 6 days during their GCSE years.
TH: For any reason. And these children, they do 25% worse than the perfect kids?
JM: It depends on what measurements you use. The normal way people compare scores at GCSE level is 5 GCSEs including English and Maths. They do 8% worse. The governments’ report on this subject uses this measure but Nick Gibb uses a more obscure measurement: kids who got the English Baccalaureate.
TH: I see. He’s gone for whatever with make the figures look the most extreme? Hang on, English Baccalaureate, what is the English Baccalaureate? Is that different exams?
JM: Well, the be honest, that’s what I thought it was but it’s not, it’s a kind of conceptual qualification. If you got GCSEs in English, Maths, a foreign language, two sciences and in history or geography you’ve passed the English Baccalaureate.
TH: And yet economics counts for nothing?
JM: Yes, I’m afraid so Tim, absolutely nothing. So anyway the government takes the EBacc to be the
sign that you’ve got good GCSEs and if you look at these swots dosed up on Calpol who never miss a day well, 43% of them get the EBacc but among the kids who’ve missed a week only 33% got the EBacc which is just about 25% lower.
TH: OK, so if you squint at the numbers in the right light you can get the 25% figure to stand up. But James, there’s a bigger question here, it might not be the precious teaching that’s making the difference, I mean the kind of kid who hardly misses a day is the kind of kid who does well in exams. So, or as the third rule of [ORS?] says, correlation is not necessarily causation.
JM: Well, exactly. Nick Gibbs is saying the absence is the thing which is causing the drop off in grades but the evidence doesn’t show this at all. In fact, even the DfE report acknowledges that factors other than attendance will have an impact on achievement and should be considered to give an accurate picture. Here’s Stephen Gorrard, Professor of Education at Durham University:
[Clip of interview with Professor Gorrard:]
SG: There is an association between the proportion of absence and the aggregate level of attainment of students who’ve had that level of absence but it would be wrong to assume that it was necessarily causal. We don’t know that the absences are the reason for the lower attainment. They could both be indicators of something else such as background characteristics and of course it’s also possible that children who aren’t do well at school after a time begin to drift away and perhaps take time off. It could be that the causal mechanism is the other way around.
JM: I mean it seems to make a certain amount of logical sense that the more school you miss, the less good you’re going to be in exams that require you to have been at school?
SG: Yes, I’m sure that right to some extent. I mean, the first thing is that not all things that young people learn and not all the improvement that they make comes from the school and I think some people would be surprised at how little difference schools make. It’s not that they don’t’ make a difference, but we tend to attribute all of the learning gain to school s whereas when studies have really attempted to assess that they’ve discovered that the schools make a small difference but not that much, so if you were absent for a couple of weeks as Nick Gibb suggests you’re talking about a small proportion of a small proportion in the variation of the outcome and I’m not sure that it would show up in the kind of figures the DfE presented. That’s very different from saying that if a child had a very serious illness that meant that they missed 6 months of school and their life was kind of disturbed by that, that they wouldn’t attain less. I think that’s common sense to suggest they are. But that’s not what this report shows. What the department is trying to say is that taking a child out for a couple of weeks maybe to go on holiday would produce this 25% drop in GCSE and I don’t think the data they have could sustain that.
[End of clip]
TH: And James, have you mentioned this point of view to the Department for Education?
JM: I have and they told me this:
DfE spokeperson: Clearly there are a range of factors that affect achievement but to suggest that missing school does not impact upon children’s life chances would be disingenuous. Our evidence shows there is a link between good attendance and better grades.
JM: there is more research being done, both by the Department for Education and by researchers at Durham University that might be able to establish the extent to which attendance impact son achievement. Until then, I’m afraid Mr Gibb’s not getting an A* from us.