Ofsted Fails to Reach a Grade B in Statistics

Michael Wilshaw’s Ofsted has made a fool of itself yet again as it publishes a report which says more about its naive approach to statistics than it does about the progress of the most able students.

Very Disappointed

Ofsted is not happy. Its 2013 report on the progress of those students who achieved level 5 in their Key Stage 2 exams made some recommendations. Apparently, Ofsted were unhappy then that less than a quarter of those achieving the highest level in Maths and English went on to achieve a B grade or above in their GCSEs, and two years later nothing has improved.

Notwithstanding that expecting that two years is enough time to see improvements when the children involved had been through twelve years of education already, is it a reasonable complaint?

Sharp Drop in Uni Applications

UCAS, the body which handles applications for university places, has released 2011 figures. The headline, reported throughout the press, is that applications this year are down by over 12% on last year. The increase in fees, payable after graduation by a form of additional income tax, is being blamed for the drop.

Shadow Education Minister Angela Burns has weighed in, saying that the preliminary data should act as a “harsh wake-up call”.

Toni Pearce, of the National Union of Students, blames the government:
“The confusion caused by the government’s botched reforms is causing young people to at the very least hesitate before applying to university.”
The Guardian misrepresents the data to suggest class differences:

Monday’s figures are just too stark to ignore. When the number of applicants from outside the UK is included, the fall is 9% — greater than it has been for at least six years. The figures show this decline in applicants comes from the pool of students most likely to be badly-off.

Everyone, it seems, thinks that university numbers should be on a one-way escalator to steadily greater proportions of our youngsters studying for degrees and that any drop is, by definition, a bad thing.

Everyone, it seems then, is wrong.

The figures, of course, do not indicate a collapse in student confidence. Although 12% seems a large drop, it should be understood in the context of dramatic rises in recent years, not least last year when many students put off a gap year to apply early. This artificially inflated 2010 figures by taking from what would have been this year’s applicants. Even with the fall, we are still only just below the 2009 rate, with a smaller pool of applicantants. (There has been a 6% drop in 17-year-olds over the last four years of increasing applications and two more years of falls — expect the same stories next year!)

The Real Issue

What the headline writers have failed to address is whether a growing number of graduates is really a good idea. The advantages are supposed to be that the UK needs more graduates for all the new graduate jobs that are being produced in the economy, and that there will be fewer jobs for lower skilled people.
Does the economy need more highly skilled workers? Yes, naturally. Highly skilled people have always been in demand throughout history. But there is a sleight of hand going on here. The problem is that the term graduate is not now synonymous with skilled, and graduate jobs do not often require high levels of skills. Twenty years ago, if you were an employer looking for a reasonably bright, trainable youngster, you advertised for someone with A-levels or good O-levels. Now that anyone who can hold a pen through sixth-form college is encouraged to start a degree, not having a degree is a serious hindrance.

Not because of the skills you did not pick up, but because employers will wonder why you weren’t up to a degree when all and sundry can graduate now.

Reaching Their Limits

When we have students maxing out at GCSE grade C going on to be awarded A-levels and then degrees, you might just wonder what it is they are learning at university. If the top three GCSE grades were beyond them, just what level was the intellectual challenge of the degree? Before you start to panic about the quality of our engineers, doctors and scientists, you can’t get on one of the technical degrees without very good grades: C’s won’t cut the mustard. But having a degree, by itself, is no guarantee of superior skill levels, with many course instructors unable to lift the academic standard to a suitable level without having most of their students failing and dropping out.
For many people, GCSE or A-level is their academic limit. Unfortunately, many of these teenagers are being mislead into thinking that by investing in an expensive three-year course their careers will be appropriately enhanced.

The evidence is against them though, as some school-leavers are starting to realise.

The country does need more skilled workers, but not more low-skilled workers with degree certificates and unrealistic expectations. The solution is not more accessible (read ‘easier’) degrees, but investment in Primary and Secondary education, with higher technical graduate salaries to persuade those bright enough to tackle the harder subjects,
or else encourage talented people from overseas to boost our ailing manufacturing economy.

This brouhaha will blow over soon, regardless. The hugely increased fees charged by middling universities will not last long, as students will expect value for money. Fees will fall to match the desirability or career benefit of the course and student numbers will drop, returning hard working students to the productive economy where they are needed.

Foul Claimed on New School League Tables

Headteachers are demeaning themselves in their rush to criticise the new English Baccalaureat figures included for the first time in this year’s league tables, and thereby excuse their own schools’ poor rankings.

The English Bac, or EBac, is awarded if a student gains GCSE grade C or above in each of English, Maths, Double Science, a humanities subject and a language.

Complaints have grouped into three main lines:

1. No time to properly game the system.

The main complaint is that the figures are retrospective, with the rules of the game only published after the exam results were out. ‘How can we be expected to do well without the time to change our curriculum policies?’ chant the headteachers.

This exposes the key moral weakness of modern schools, which is that directly manipulating the key indicators to make the school look good is preferred to actually improving the pupils’ education.

Gaming the system, then, is the main occupation of school managers.

2. The EBacc is a return to Academic Snobbery.

Why not allow vocational courses as well?

Schools have a choice to make. Enter children on to the course with the best educational aims (say, French or Science GCSEs) when many will achieve grades D to G passes and so not count in the laegue tables. Or, enter them for vocational courses such as the Btec, that guarantee the ‘equivalent’ of four GCSE grade Cs to any pupil still conscious at the end of the course.

The choice is really this stark, and sadly most schools go for option B with varying degrees of enthusiasm. Those under the cosh from Ofsted especially realise this is the only way out of ‘special measures’. Look at the tables and you can spot such schools: they will have improved their 5 A to C figures at improbably fast rates, have high CVA (value added) scores from the additional ‘equivalent’ courses and LOW EBacc rates.

The ‘most improved’ school in the country, Perry Beaches in Birmingham, has moved its 5 A to C figures from 21% to 74% in four years. CVA is also high, but only 3% got the EBacc. Since it takes four or five years to progress through the school as a pupil, the changes must have been instantaneous to have fed through this quickly.

Certificates of GCSE equivalent passes shouldn’t count if everyone passes them. It misleads prospective parents into thinking the school is academic and improving, when it is only the figures that are going up. The quality of the education may actually be declining in these schools as they move from GCSE to Btec and other similar courses.

3. Independent schools are unfairly penalised.

Independent schools are not restricted to only offer courses approved by the politically directed Qualification Curriculum Authority which only approved courses with sufficient levels of coursework in their assessment schemes.

Independent schools don’t approve of coursework, so many offer alternative courses, such as Classical Civilisation, which don’t count towards the EBacc, damaging their figures.

Now, this is a fair complaint. But Independent schools are not compelled to enter the League Tables manipulation game, or even publish figures at all. They are free to create their own tables if they wish so they can compete on their own manicured level playing fields with their own rules.

Unfair, perhaps, but they can take their ball and play elsewhere if they don’t like it.

Less Gaming, Please.

The arrival of the EBacc has embarrassed lots of schools. They complain of the pressures of league tables and the focus on A to C grade passes which excludes the varying efforts of anyone not near the grade boundaries. But they should welcome anything that makes gaming harder and so less attractive. Less gaming should herald a move back towards professional judgements in schools instead of political ones, where the children come first.

I won’t be holding my breath though. Heads have been manipulating their table positions for a long time, and will be looking for ways to continue the game. It is all many of them know.

General Teaching Council 1998-2010 RIP

Michael Gove, the new Education Secretary, has announced that the General Teaching Council for England (GTCE) is to be abolished in the autumn, and not a moment too soon.

It has cost a small fortune to run and was never going to be a rallying point for teacher professionalism, and has failed even to act as a guarantor of teacher quality by disciplining us.

When the GTCE was created in 1998, it had so few teachers paying their subscriptions, even under the threat of de-registration, that it had to arrange for salary deductions to cover its expenses. I, like many other teachers, saw no benefit in the extra layer of bureaucracy. All teachers were already registered with the government Department for Education (and its heirs and successors), many were also members of unions and teacher subject groupings (such as the ASE) and felt we were already quite well represented and regulated.

For my own part, I did not pay any subs until salary deductions started, I responded to no letters, and was pleased that when I moved to a Sixth Form college which didn’t require my registration, the GTCE was unable to take any further money. I had a letter saying that I would be de-registered (struck off) if I didn’t pay up, so I was surprised that two years later they wrote again to say I owed them two years’ payments. They couldn’t even get that right.

The GTCE is, and always has been, a complete irrelevance to teachers. When it finally goes, few will notice and none will care.

So what did the GTCE say on hearing the good news? Did they respond by apologising for wasting everyone’s time and money? Promise to do better? No, they said that they were “seeking legal advice on (their) position“.

Parliament will surely vote to abolish the GTCE later in the year, and it can be finally buried, unloved and unmissed, in the graveyard of the Quangos.

Mass Produced Target Grades

Sixth form students will by now have dragged themselves through the January exam series. They can relax until scores are released in March, when most will be judged according to their college ‘target grades’. And it is likely to be a miserable experience for most.

I used to talk to my students and get to know their individual strengths and weaknesses. I would encourage those who I perceived were studying hard, and chide those who were just attending class without the necessary intellectual engagement. Reports to parents and managers were based on my professional opinion of each child. But not any more.

Teachers still get to know each of their charges, but their professional judgements are now routinely tempered by the knowledge that performance against their grade target trumps all other information.

Value Added

Target grades are now the ubiquitous tool of comparative assessment in schools: Key Stage 3 results are used to predict GCSE grades, while GCSE grade averages are used to compute the most likely grade a student might achieve at A Level. This is a very good process for working out if the school is doing a good job, since if a year group cohort gains a mean score above the mean predicted grade, then the group has learned more than could reasonable been expected. The school thus has recorded some value added, in the language of education.

Using the same data for individual teachers is only likely to be reliable over a period of several years, since the sample sizes from individual classes are much smaller, leading to more variation from year to year.

Blinded by Numbers

The big problem stems from applying these statistics to individual students. It is very easy to calculate an expected grade from a single child’s previous achievement, but with a sample size of just one, the precision is poor. The reliability stemming from a cohort in the hundreds is lost, and the prediction is routinely in error by a whole grade or so. (See my post Physics Exams Too Easy, Says Ofqual

Now, this would not be a problem if these figures were just another piece of the puzzle to be understood by the teacher, but OFSTED, the government overlord of teaching standards, thinks students should know these rough predictions, and be challenged to achieve them. And leaned on if they don’t come up to scratch.

Once upon a time, I got to know my own students, and made judgements as to their individual abilities and potentials, and assessed their effort accordingly. Not perfect, but at least both teacher and student were in the loop.

Forget the Child – Press the Button and Set That Target

Now, each student is given a grade to achieve by the end of a two year course, during which they will mature and develop. If they are very lucky, they will get several target grades which take into account the historical difficulties of each subject they are studying. If not, as is happening more commonly now, they will get a single grade to span the range from Photography and Media to Chemistry and Maths. And to make the target aspirational, a grade will be added to ensure that only a quarter of students will be able to meet their targets, with poor reports and disciplinary procedures for those souls unlucky enough to keep missing impossible targets.

Advantages

• Simple and cheap to operate.
• Keeps OFSTED happy.

Disadvantages

• No educational merit.
• Can turn keen students into serial target-missers.

An open and shut case for school managers. Shame about the children.

Government Buries ‘Good’ News

Even though the new Academies have done so well in last summer’s exam period, the government can still not bring themselves to publish usable comparative data to prove the case.

The recent publication of the GCSE and Equivalent Results in England was accompanied by a government press release which declared:

“Even more encouraging is the clear fact that children who have faced the most challenging circumstances over the past 12 years have not been cast adrift and left at the bottom, but thanks to the Academies, National Challenge and City Challenge programmes have actually seen a more rapid improvement in results than those in the least deprived areas.”

Concerns

The biggest concern with the steady and dramatic increases in the figures, after grade inflation, is the widespread tactic of entering weaker pupils for the laughably easy vocational courses that have ‘GCSE equivalence’. For example, half of all schools enter almost everyone into the vocational ICT course instead of the more challenging GCSE ICT, as it can be completed in one year and can be worth anything up to four GCSEs at grade C or above. The Telegraph reports:

A report last year by Ofsted, the education watchdog, found that qualifications such as the one run by OCR were “less demanding” than other mainstream exams. It said pupils were able to pass “whether or not they had understood what they had done”.

It is true that the number of students nationally that gain 5 A*-C grades is only 3% less if you exclude the ‘equivalent’, vocational courses, but there are categories of schools which have a lot to lose if their pass rated don’t improve sharply.

Schools Minister Vernon Coaker’s press-release said that the new Academies A*-C grades had increased at double the national rate, so I looked at the accompanying data.

Sneaky Tricks

The data looks comprehensive, and even breaks down results for different types of school and with/without the vocational courses. The time series do indeed show Academies improving faster then any other major category of schools, but the figures for the Academies without the vocational courses are missing!

Vernon’s minions emphasised the Academy raw percentage increases, but rolled their key data in with that for comprehensive schools, making it impossible to see what effect the dash for vocational courses has had.

Since it is just these schools that are most desperate to bump up their figures quickly to avoid the Wrath of Balls, it is very suspicious that the data, which was released with some fanfare, should have been airbrushed like this. What is the government afraid of? Given the obsession of the government with media manipulation, one can only assume that Academies have achieved their politically essential successes through a choice of courses that benefited the school more than their unfortunate pupils.

On Keeping Gimmicks Out of the Classroom

Having resisted all manner of education gimmicks and fashions that have been thrust at me by well meaning college managers, it was refreshing to read the latest piece written by renowned undergraduate textbook writer and educator, David Griffiths. Published in the IoP magazine Physics World, Griffiths reminds us that Physics sells itself to students if presented honestly:

“Physics teachers are fortunate (I am among friends, so I can speak frankly): ours is a subject the relevance and importance of which are beyond question, and which is intrinsically fascinating to anyone whose mind has not been corrupted by bad teaching or poisoned by dogma and superstition. I have never felt the need to “sell” physics, and efforts to do so under the banner “physics is fun” seem to me demeaning. Lay out our wares attractively in the marketplace of ideas and eager buyers will flock to us.

“What we have on offer is nothing less than an explanation of how matter behaves on the most fundamental level. It is a story that is magnificent (by good fortune or divine benevolence), coherent (at least that is the goal), plausible (though far from obvious) and true (that is the most remarkable thing about it). It is imperfect and unfinished (of course), but always improving. It is, moreover, amazingly powerful and extraordinarily useful. Our job is to tell this story – even, if we are lucky, to add a sentence or a paragraph to it. And why not tell it with style and grace?”

He goes on to criticise the gimmickry that is supposed to gain better attention from students. He has this to say about the advent of flash cards and electronic clickers:

“They can be powerfully effective in the hands of an inspired expert like Mazur, but I have seen them reduced to distracting gimmicks by less-capable instructors. What concerns me, however, is the unspoken message reliance on such devices may convey: (1) this stuff is boring; and (2) I cannot rely on you to pay attention. Now, point (2) may be valid, but point (1) is so utterly and perniciously false that one should, in my view, avoid anything that is even remotely open to such an interpretation.”

The point is made that any new approach to teaching will produce measurable improvements, but only because of the enthusiasm of the practitioner. Infectious enthusiasm is most likely the key, and not all teachers have that, so maybe the gadgets help these classes. But I’m not convinced.

Griffiths was known as a great lecturer and scorned such fashions. You can watch one of his lectures here.

Boys’ Results Improve Now Maths Coursework is Scrapped

When GCSEs were introduced two decades ago, one of the aims was to help girls catch up with boys in exams. The plan was a classic case of unintended consequences: the requirement for GCSEs to be graded with at least a quarter of the points from coursework has resulted in girls being awarded higher grades across the board.

Although boys and their lack of conformity in the classroom attracted the blame for their deteriorating grades by the feminised teaching profession, the truth is out: boys can doing better than girls. In Mathematics boys are now outperforming girls in all the higher grades.

So what has driven up their scores? Extra relevance of lessons? Better teacher training and school discipline structures? Lessons moved to inner city football clubs or fishing trips for malcontents?

The solution has been obvious for ten years, but has only been implemented because it has become obviouse that work completed at home was open to widespread plagiarism. It has worked for Mathematics GCSE as well as all the International Baccalaureat courses. What is holding the government up from rolling this great innovation to all subjects?

Or the QCA could allow schools to offer the IB and let market forces choose.

AQA GCSE: Physics Without Physics

Last week England’s largest exam board issued a Physics GCSE paper, aimed at our brightest youngsters, that required no mathematical calculations. Last year’s GCSE Physics papers prompted the Qualifications and Curricculum Authority (QCA) to rule that Physics papers were not sufficiently challenging, but AQA has sunk to a new low.

The paper was the P3 Higher Tier one, so a series of conceptual deductions, calculations, simple algebra and graph interpretation would have been expected, but thousands of pupils were surprised by the disappearance from the exam of the bulk of what they had struggled to learn.

The anonymous quote about the three levels of Physics has finally become complete, officially:

There are three levels of Physics courses: Physics with calculus, Physics without calculus and Physics without Physics.

A paper made up from simplistic sequencing and qualitative statement questions is not suitable for bright or even average students, who were disappointed that they were not to be properly tested after all their rigorous preparation.

We need a new generation of scientists and engineers, but they will not be challenged or tempted by the new and ‘accessable’ Physics Without Physics GCSE courses on offer.

Who are these courses now aimed at? The maths-phobic or the future core of a technological society?