Michael Wilshaw’s Ofsted has made a fool of itself yet again as it publishes a report which says more about its naive approach to statistics than it does about the progress of the most able students.
Ofsted is not happy. Its 2013 report on the progress of those students who achieved level 5 in their Key Stage 2 exams made some recommendations. Apparently, Ofsted were unhappy then that less than a quarter of those achieving the highest level in Maths and English went on to achieve a B grade or above in their GCSEs, and two years later nothing has improved.
Notwithstanding that expecting that two years is enough time to see improvements when the children involved had been through twelve years of education already, is it a reasonable complaint? Read More…
Headteachers are demeaning themselves in their rush to criticise the new English Baccalaureat figures included for the first time in this year’s league tables, and thereby excuse their own schools’ poor rankings.
The English Bac, or EBac, is awarded if a student gains GCSE grade C or above in each of English, Maths, Double Science, a humanities subject and a language.
Complaints have grouped into three main lines:
1. No time to properly game the system.
The main complaint is that the figures are retrospective, with the rules of the game only published after the exam results were out. ‘How can we be expected to do well without the time to change our curriculum policies?’ chant the headteachers.
This exposes the key moral weakness of modern schools, which is that directly manipulating the key indicators to make the school look good is preferred to actually improving the pupils’ education.
Gaming the system, then, is the main occupation of school managers.
2. The EBacc is a return to Academic Snobbery.
Why not allow vocational courses as well?
Schools have a choice to make. Enter children on to the course with the best educational aims (say, French or Science GCSEs) when many will achieve grades D to G passes and so not count in the laegue tables. Or, enter them for vocational courses such as the Btec, that guarantee the ‘equivalent’ of four GCSE grade Cs to any pupil still conscious at the end of the course.
The choice is really this stark, and sadly most schools go for option B with varying degrees of enthusiasm. Those under the cosh from Ofsted especially realise this is the only way out of ‘special measures’. Look at the tables and you can spot such schools: they will have improved their 5 A to C figures at improbably fast rates, have high CVA (value added) scores from the additional ‘equivalent’ courses and LOW EBacc rates.
The ‘most improved’ school in the country, Perry Beaches in Birmingham, has moved its 5 A to C figures from 21% to 74% in four years. CVA is also high, but only 3% got the EBacc. Since it takes four or five years to progress through the school as a pupil, the changes must have been instantaneous to have fed through this quickly.
Certificates of GCSE equivalent passes shouldn’t count if everyone passes them. It misleads prospective parents into thinking the school is academic and improving, when it is only the figures that are going up. The quality of the education may actually be declining in these schools as they move from GCSE to Btec and other similar courses.
3. Independent schools are unfairly penalised.
Independent schools are not restricted to only offer courses approved by the politically directed Qualification Curriculum Authority which only approved courses with sufficient levels of coursework in their assessment schemes.
Independent schools don’t approve of coursework, so many offer alternative courses, such as Classical Civilisation, which don’t count towards the EBacc, damaging their figures.
Now, this is a fair complaint. But Independent schools are not compelled to enter the League Tables manipulation game, or even publish figures at all. They are free to create their own tables if they wish so they can compete on their own manicured level playing fields with their own rules.
Unfair, perhaps, but they can take their ball and play elsewhere if they don’t like it.
Less Gaming, Please.
The arrival of the EBacc has embarrassed lots of schools. They complain of the pressures of league tables and the focus on A to C grade passes which excludes the varying efforts of anyone not near the grade boundaries. But they should welcome anything that makes gaming harder and so less attractive. Less gaming should herald a move back towards professional judgements in schools instead of political ones, where the children come first.
I won’t be holding my breath though. Heads have been manipulating their table positions for a long time, and will be looking for ways to continue the game. It is all many of them know.
Since the Schools Secretary has failed to answer any questions I’ve addressed to him in posts, I submitted a question to Sarah Ebner’s blog over at the Times. She had managed to get Balls to join a live Q&A session and had asked for questions to ask his. Needless to say, the slimy minister read all the questions and proceeded to answer the ones he had prepared earlier, in true Blue Peter style.
The full discussion is here , with my question down at times 14.02 to 14.05. The accompanying photo, captioned http://www.timesonline.co.uk/tol/comment/live_debate/article7012253.ece is reproduced here — does he look shocked at the quality of the question?
The edited sequence goes like this:
14.02 Sarah Ebner:
This is another point which comes up often on the blog. Glen asks: why persist with judgements of schools based on raw percentages of students achieving 5 grade C GCSEs? It penalises schools serving deprived areas – schools which need a hand up, not a kicking. It also pressurises schools to focus on grade D students, and encourages entries into easier ICT online courses and such like.
14.05 Ed Balls:
Sarah, you are completely right and so is Glen, as parents we all have to look at the current league table and try to work out what they really mean on the basis of what we know about the school itself, the catchment area etc. And league tables can sometimes suggest schools are ‘high achieving’ when they actually do a poor job at raising standards and supporting progression. Our new Report Card is designed to give parents much more information – about raw results but also whether all children make progress, discipline, parent satisfaction etc. I think it will be much fairer and more informative – it will be in all schools over the next 2 years.
So, I had asked him about the huge pressure that the government and Ofsted puts, often unfairly, on schools in difficult circumstances, distorting their priorities, and he answers an imaginary question about league tables!
OK, he did that to all of the questions, but then why did he bother to travel to the Times offices, just to act as if he were in the House of Commons (not) answering questions put to him there?
Sixth form students will by now have dragged themselves through the January exam series. They can relax until scores are released in March, when most will be judged according to their college ‘target grades’. And it is likely to be a miserable experience for most.
I used to talk to my students and get to know their individual strengths and weaknesses. I would encourage those who I perceived were studying hard, and chide those who were just attending class without the necessary intellectual engagement. Reports to parents and managers were based on my professional opinion of each child. But not any more.
Teachers still get to know each of their charges, but their professional judgements are now routinely tempered by the knowledge that performance against their grade target trumps all other information.
Target grades are now the ubiquitous tool of comparative assessment in schools: Key Stage 3 results are used to predict GCSE grades, while GCSE grade averages are used to compute the most likely grade a student might achieve at A Level. This is a very good process for working out if the school is doing a good job, since if a year group cohort gains a mean score above the mean predicted grade, then the group has learned more than could reasonable been expected. The school thus has recorded some value added, in the language of education.
Using the same data for individual teachers is only likely to be reliable over a period of several years, since the sample sizes from individual classes are much smaller, leading to more variation from year to year.
Blinded by Numbers
The big problem stems from applying these statistics to individual students. It is very easy to calculate an expected grade from a single child’s previous achievement, but with a sample size of just one, the precision is poor. The reliability stemming from a cohort in the hundreds is lost, and the prediction is routinely in error by a whole grade or so. (See my post Physics Exams Too Easy, Says Ofqual
Now, this would not be a problem if these figures were just another piece of the puzzle to be understood by the teacher, but OFSTED, the government overlord of teaching standards, thinks students should know these rough predictions, and be challenged to achieve them. And leaned on if they don’t come up to scratch.
Once upon a time, I got to know my own students, and made judgements as to their individual abilities and potentials, and assessed their effort accordingly. Not perfect, but at least both teacher and student were in the loop.
Forget the Child – Press the Button and Set That Target
Now, each student is given a grade to achieve by the end of a two year course, during which they will mature and develop. If they are very lucky, they will get several target grades which take into account the historical difficulties of each subject they are studying. If not, as is happening more commonly now, they will get a single grade to span the range from Photography and Media to Chemistry and Maths. And to make the target aspirational, a grade will be added to ensure that only a quarter of students will be able to meet their targets, with poor reports and disciplinary procedures for those souls unlucky enough to keep missing impossible targets.
- Simple and cheap to operate.
- Keeps OFSTED happy.
- No educational merit.
- Can turn keen students into serial target-missers.
An open and shut case for school managers. Shame about the children.
“Even more encouraging is the clear fact that children who have faced the most challenging circumstances over the past 12 years have not been cast adrift and left at the bottom, but thanks to the Academies, National Challenge and City Challenge programmes have actually seen a more rapid improvement in results than those in the least deprived areas.”
The biggest concern with the steady and dramatic increases in the figures, after grade inflation, is the widespread tactic of entering weaker pupils for the laughably easy vocational courses that have ‘GCSE equivalence’. For example, half of all schools enter almost everyone into the vocational ICT course instead of the more challenging GCSE ICT, as it can be completed in one year and can be worth anything up to four GCSEs at grade C or above. The Telegraph reports:
A report last year by Ofsted, the education watchdog, found that qualifications such as the one run by OCR were “less demanding” than other mainstream exams. It said pupils were able to pass “whether or not they had understood what they had done”.
It is true that the number of students nationally that gain 5 A*-C grades is only 3% less if you exclude the ‘equivalent’, vocational courses, but there are categories of schools which have a lot to lose if their pass rated don’t improve sharply.
Schools Minister Vernon Coaker’s press-release said that the new Academies A*-C grades had increased at double the national rate, so I looked at the accompanying data.
The data looks comprehensive, and even breaks down results for different types of school and with/without the vocational courses. The time series do indeed show Academies improving faster then any other major category of schools, but the figures for the Academies without the vocational courses are missing!
Vernon’s minions emphasised the Academy raw percentage increases, but rolled their key data in with that for comprehensive schools, making it impossible to see what effect the dash for vocational courses has had.
Since it is just these schools that are most desperate to bump up their figures quickly to avoid the Wrath of Balls, it is very suspicious that the data, which was released with some fanfare, should have been airbrushed like this. What is the government afraid of? Given the obsession of the government with media manipulation, one can only assume that Academies have achieved their politically essential successes through a choice of courses that benefited the school more than their unfortunate pupils.
The National Union of Teachers (NUT) has voted to scupper the SAT exams for eleven-year-olds this year, but it won’t produce the renaissance in teaching they expect.
It is the teaching union silly season, and time for their AGMs. There is normally a flurry of embarrassing quotes from representatives that are quickly ignored, but this week a substantial motion has been passed by the largest union, the NUT. They have decided to ballot members for industrial action to disrupt the national assessment of seven- and eleven-year-olds (Key Stage 1 and 2 SATs), taking advantage of the recent collapse of the Key Stage 3 assessments and the dithering of the government minister Ed Balls.
Teachers generally dislike these assessments as they are unreliable and used for annual teacher appraisals, and now seems like the best chance in years to force a weak government to abandon them.
The massive expansion of national exams over the last decade or so, fed by the movement to modular exams that can be retaken an unlimited number of times at GCSE and A Level, along with the SATs (the National Curriculum Tests at ages 7, 11 and 14), has overloaded the exam boards’ marking systems. There are simply not enough markers in the country to process all the papers. This caused the collapse of the Key Stage 3 exams last year, and has caused this year’s results to be posted later than ever before, and appeals are expected to flood in shortly afterwards due to quality control problems, especially for the English assessments.
I have posted before that these tests have become too ‘high stakes’ to be useful and national standards should be assessed in other ways, but the NUT is not keen on developing a decent assessment system. They just want to be rid of the SATs.
Their real problem, so they say, is that the pressure placed on pupils in the run-up to the tests is too great. Primary school children spend much of Year 6 preparing, practising the tests and taking test questions away to do for homework. Every child is given targets couched in the assessment language (‘I’m now at level 3B for Writing, and I am aiming for a level 4C’ the pupils will repeat) and the school inspectors check that pupils are aware of them.
True, the pressures are ridiculously high, and teaching to the test is so endemic that to suggest anything different to any teacher younger than 40 will get you a confused look. It is also true that the government has set up the system to be like this, despite their protestations of innocence, but the Union is picking on the wrong target deliberately, hoping that no-one will question their members’ complicity in the whole affair.
The teachers have a great deal of freedom in how they manage their classrooms day-to-day. If they do not want to pressure their charges then they should stop talking up the exams all the time. They should stop teaching to the tests and setting questions from previous years’ papers, and they should certainly stop running the government funded ‘Booster Sessions’ – additional revision work for those poor souls deemed to be close enough to the pass level that they can be artificially pushed over the line with some special attention and pressure.
Instead of declaring the tests ‘harmful’ to pupils, the NUT ought to just tell their members to stop squeezing the last drop of exam performance from their classes simply to gain better pass figures for their annual appraisals and a higher league table position for their school. If the teaching unions want teaching to be treated as a profession, then professional behaviour must be encouraged. All the while teachers put ratings ahead of education their motives will be suspect.
If teaching to the test stopped, whether by scrapping the tests or by teachers taking control of their classrooms, then, the unions say, the curriculum will become broader and children will have a better experience. But they are mistaken. Even if the test went, the skills of teachers to plan their own programmes of study have so withered that most teachers would not know what to do with the extra term of teaching. What would they do with no exam to prepare for? How would they know what skills to develop and knowledge to learn if it is not written down in great detail by the government?
‘It’s not in the test!’
When the SATs were first introduced, teachers had some idea of what schooling was for, some philosophy of education. But over the years those teachers have retired and the younger ones have only known teaching for national assessments. Those few who dare to go beyond the minimum entitlement laid down in the National Curriculum, following their pupil interest or their own enthusiasms, have been slapped down by managers with the immortal lines:
‘What are you teaching that for – it’s not in the test!’
The English teaching ‘profession’ has become so de-skilled that if the tests disappeared nothing would really change. If the prison doors were flung open tomorrow, most teachers would be too frightened of the freedom to go out into the daylight, doomed to pace around the same familiar cell.
The just released 2007 Trends in International Mathematics and Science Study (TIMSS) has seen England rise to fifth position after the four yearly study looked again at the quality of the science education of fourteen-year-olds around the world (BBC report: England’s Pupils in Global Top 10).
English students are beaten only by those from Singapore, Taiwan, Japan and Korea, with all of Europe trailing in their wake. Jim Knight is clearly pleased at this validation of Labour policies on the world stage, but he can’t bring himself to ease the political pressure off over-burdened schools, even if they have done all he asked of them.
Knight has looked and looked, and he managed to find some bad news in the report. That’s right — science teachers up and down the country can stop partying, under the impression that all was well in their subject and a pat on the back was due.
Children are enjoying science less than they used to! There has been a 21% drop in ‘positive attitudes’ reported by the pupils, and Jim is not happy.
Teachers, go and sit on the naughty step.
Must Do Better
Being the best in Europe and the industrialised West is not good enough if a few far-eastern nations with fantastically well drilled children are better.
Knight says in the press release:
This shows we are on the way to being world class but as we move towards this goal we need to make sure every child has fun in the classroom as well as achieving good results.
I am determined to make maths and science more exciting subjects to teach and learn, and I want every school to have access to the most innovative and effective teaching methods. I want more action in the classroom and more problem solving and ‘flash and bang’ to enthuse our pupils.
A new OFSTED target, perhaps? Inspectors could report:Your lesson on nuclear power was well taught and the children learned well, but there wasn’t enough ‘flash and bang’ for the lesson to be rated any good.’
Squeezing the Pips
Other countries to suffer from reduced student positivity included Singapore and Hong Kong — both in the top ten alongside England. Jim Knight seems to think that league table rankings and pupil enjoyment are independent of each other, but teachers have complained for years about the curriculum and targets straitjacket that they have to operate in, and the effect on the enjoyment that classes are able to have.
The government has squeezed children hard so that they achieve their potential, but the pips are squeaking now. If he was serious about restoring awe and wonder to school science lessons, then Knight and Balls would be cutting the testing and accountability burden.
Freeing teachers to impart some of their love for their subjects, though, would risk a slip in the rankings. And that would never do, would it?