Tag Archive | A levels

Mass Produced Target Grades

Sixth form students will by now have dragged themselves through the January exam series. They can relax until scores are released in March, when most will be judged according to their college ‘target grades’. And it is likely to be a miserable experience for most.

I used to talk to my students and get to know their individual strengths and weaknesses. I would encourage those who I perceived were studying hard, and chide those who were just attending class without the necessary intellectual engagement. Reports to parents and managers were based on my professional opinion of each child. But not any more.

Teachers still get to know each of their charges, but their professional judgements are now routinely tempered by the knowledge that performance against their grade target trumps all other information.

Value Added

Target grades are now the ubiquitous tool of comparative assessment in schools: Key Stage 3 results are used to predict GCSE grades, while GCSE grade averages are used to compute the most likely grade a student might achieve at A Level. This is a very good process for working out if the school is doing a good job, since if a year group cohort gains a mean score above the mean predicted grade, then the group has learned more than could reasonable been expected. The school thus has recorded some value added, in the language of education.

Using the same data for individual teachers is only likely to be reliable over a period of several years, since the sample sizes from individual classes are much smaller, leading to more variation from year to year.

Blinded by Numbers

The big problem stems from applying these statistics to individual students. It is very easy to calculate an expected grade from a single child’s previous achievement, but with a sample size of just one, the precision is poor. The reliability stemming from a cohort in the hundreds is lost, and the prediction is routinely in error by a whole grade or so. (See my post Physics Exams Too Easy, Says Ofqual

Now, this would not be a problem if these figures were just another piece of the puzzle to be understood by the teacher, but OFSTED, the government overlord of teaching standards, thinks students should know these rough predictions, and be challenged to achieve them. And leaned on if they don’t come up to scratch.

Once upon a time, I got to know my own students, and made judgements as to their individual abilities and potentials, and assessed their effort accordingly. Not perfect, but at least both teacher and student were in the loop.

Forget the Child – Press the Button and Set That Target

Now, each student is given a grade to achieve by the end of a two year course, during which they will mature and develop. If they are very lucky, they will get several target grades which take into account the historical difficulties of each subject they are studying. If not, as is happening more commonly now, they will get a single grade to span the range from Photography and Media to Chemistry and Maths. And to make the target aspirational, a grade will be added to ensure that only a quarter of students will be able to meet their targets, with poor reports and disciplinary procedures for those souls unlucky enough to keep missing impossible targets.

Advantages

  • Simple and cheap to operate.
  • Keeps OFSTED happy.

Disadvantages

  • No educational merit.
  • Can turn keen students into serial target-missers.

An open and shut case for school managers. Shame about the children.

A Level Results will bring Clearing Heartache

Results day is nearly on us, but those unfortunate students who miss their offer grades will have fewer options than usual this year. There is more competition, and even the standard fallback, clearing, will not have many course places to offer.

For a quick assessment of the regular effects of this silliest part of the silly season on teachers check last year’s post on the matter (the newspapers will just roll out the same stories anyway). This post will focus instead on the students.

This Year is Different

This year is different for anyone biting their nails waiting for Thursday’s results, because of the combined effects of three government policies:

  • One is the often discussed grade inflation, which leads ever larger proportions of the school population to feel they have what it takes to succeed at university, and allows the government to claim standards are improving.
  • The second is the lack of funding to cover the extra costs to institutions of teaching the increasing numbers of undergraduates.
  • And the third is that, for the first time in fifteen years, universities will be financially penalised if they over-recruit.

Traditionally, students’ applications to universities are based on the school predicted grades, which are inflated to improve the chance of an offer. It is not as risky as it sounds, since universities routinely allow students who only miss their offer by one grade to still keep their place. And everyone does it, making it fair, at least. And if they missed out, then last year 44 000 applied for course during Clearing, filling up the remaining university places.

Fewer Places

But now, with 40 000 extra applications and only 3 000 extra places, Clearing will only have around 16 000 places on offer, leaving 65 000 hopeful students without a place. And with a demographic peak reaching college in the next year or two, taking a year out and applying again next year is now looking to be a silly strategy.

One could hope that the main effect of all this is that only those students with poor grades applying to weaker institutions will suffer, but many students are unrealistic when it comes to selecting competitive courses at prestigious universities.

Rejections

Admissions officers are saying they will not allow any ‘softening’ of offers, so missing one A level subject by one grade will lead to a rejection. Even our local ex-teacher-training college-now-university will not soften requirements or offer places for clearing on any but the two most frivolous courses.

The next couple of weeks look likely to deliver heartache on a large scale.

Physics Exams Too Easy, Says Ofqual

Ofqual, the newly formed watchdog for exam standards, has assessed a variety of GCSE and A Level course assessments, and Physics has been found wanting. The grades awarded have been too high for the understanding demonstrated in the exams and there are now students on A Level courses who have an inflated expectation of grades in the summer. Some are in reality so poor they are unlikely to pass.

Despite repeated ministerial assurances that standard have not slipped, it seem that there is less demand in the 2007 Physics GCSE papers than in 2002. Even the head of Ofqual, Ms Tattersall, said in the autumn that she was confident that there had been no dumbing down, but she did what politicians never seem to do — she commissioned research to check. And the first research was published on Friday.

The Findings

It turns out that at the key grades of A and B candidates do not need to perform as well now as they did in the past. The level of challenge is less because the questions require lower order thinking skills, and in many cases can be answered without any Physics knowledge. There is also a greater reliance on objective type (multiple choice) questions.

Of course, this is no surprise to anyone who has been following the constant flow of independent research on the issue, especially from Prof Smithers and co at Buckingham University. Jim Knight, who has consistently ignored this research, claiming that inflating grades were solely due to the fact that the nations students and teachers were the best we’ve ever had, will find it harder to ignore the Ofqual findings. He had a letter on Thursday warning him what had been found, and he has kept his head down since.

Solutions and Problems

The exam boards have been instructed to change the exams to make them more challenging for this summer, stop awarding marks to incorrect answers and retrain their exam writers, but it is quite short notice.

In any case, they can’t change much because there can’t be allowed a sudden change in pass rates. So the easy questions get made more challenging, but the pass marks get lowered to compensate. Ofqual gets a rosy glow of self congratulation. The government takes the credit for a system well managed. And the more things change the more they stay the same.

Especially for the students.

Unrealistic Targets

A Level Physics has been largely consistent in its challenge, so the problem moves onto the Sixth Forms. The difficulty is that entry onto A Levels depends on GCSE grades, for Physics that means GCSE Physics and Maths. If the requirement is a grade C in Physics (or Science, which uses the same Physics papers that were assessed in the report) then we are allowing weaker students onto the course than in the past if those grades are less of a challenge.

And those students are given target grades based on what previous students with the same grades achieved in the past, and if standards are not consistent then those targets will become progressively less achievable. Targets must be reasonable if they are to serve any proper purpose, such as for motivating the students into some action after exams, or for assessing the performance of the teachers.

Already the system is being subverted in colleges. If a student meets the minimum GCSE requirements for a course (in the case of Physics at my college, it is five grade Cs or better, including a B in Maths and C in Science or Physics) then they are enrolled. A grade prediction is generated based on the mean GCSE grade, and a bit is added for encouragement. However, some students would get a prediction of a grade U (a fail), so these are changed to grade E+. It wouldn’t do to let the student know that 90% of students entering with the same GCSE grade failed the course, would it?

The students of course, ignorant of the amount of effort needed just to pass, carry on in the time honoured way of just attending the lessons and doing no study. Until the day the first exam results arrive, they blunder on hoping that the class tests were wrong. When the ‘fail’ slip arrives, there is disbelief. They were let on the course, so they must be clever —how could they fail?

Modular Physics Courses Harm Deep Learning

The change of A Level assessment from terminal exams to modules, examined every term or so, has increased exam success for students, but at the expense of a deep understanding of Physics.

The Curriculum 2000 program introduced Advanced Supplementary (AS) courses in the UK at teh beginning of the decade, to help give breadth to the A Level choices that students made at the end of compulsory schooling. Since they were only committing themselves to nine months, many people who had an interest in Physics, but were uncertain about its reputation for difficulty, felt they could take a chance. The effect on enrollment on to college Physics courses was immediate, with class sizes swelling considerably.

So why is it that Physics and Engineering degree courses have continued their decline in popularity?

Unintended Consequences

Class Size

Larger class sizes affect the time teachers can spend supporting individual students in the first year of the course, and the amount of preparation they can do, due to the increased quantity marking and reporting needed. Many students drop out after one year, but since large class sizes have become the norm, classes are cut to keep the student-staff ratio up. With the old two year courses, the classes shrank for the more challenging second year, allowing students to get one-to-one support more often and teachers to get a better understanding of each student’s learning.

Dropping Courses

Weaker students, facing the prospect of failure after two years on a course, could find the motivation within themselves to work harder as the final exams drew nearer.

Being able to drop one of their three AS courses at the end of the first year encourages students to stick with the easiest courses in the attempt to maximise their haul of grades. This is no bad thing, and has contributed to the higher grades awarded in recent years, but there are serious side-effects.
For example, soon after their introduction I had a potential Oxbridge student, close to the end of the first year, excuse his recent lack of completed homework by saying he was thinking of dropping Physics (expecting grade A or B). Instead, he was considering continuing with Religious Studies (guaranteed, apparently, a grade A) as he wanted the status of a complete sweep of grade As, awarded to 26,000 students in 2008.

Schools are also tempted to encourage students to enroll onto easier subject, limiting science course uptake, but maximising the school’s league table position.

Modules

The modular structure itself, though, is the biggest problem. Instead of a two year coherent course, designed to build concepts and skills progressively, students work towards six, largely independent, work units, called modules. Although the change was supposed to motivate students by keeping up the flow of high stakes examination, most of the effects have been negative and dangerous:

  • Most importantly, modules, with their regular schedule of bite-sized exams, encourage cramming and surface learning from the students and teaching to the test by teachers.
  • Mock exams, as a safe opportunity to test your progress, have lost their power to motivate.
  • The module structure of the exams discourages exam boards from using broad synoptic style questions in the module exams, with deep questioning left until the last summer paper. But of course, the patterns of thought have been set by then, and students often fail to grasp the interconnectedness of the subject.

Drop the Modules

Modular courses act as a disincentive for students to thoroughly learn and understand their chosen subjects. The January exams, at least, should be abandoned by colleges, and the time gained put to goo use teaching the students to understand and love their subjects.

Engineering Diplomas Only Partially Accepted by Universities

With both Oxford and Cambridge reporting that they will accept Advanced Engineering Diplomas for students entering their Engineering degree programmes, the top Russell Group Universities now have a unified response to the Government’s flagship education policy. But it is not accepted without reservations, as

… it is essential that the diploma sufficiently equips candidates with the skills and knowledge they need to flourish on our courses and we want to be fully assured that they are sufficiently robust and challenging academically. Our member universities are in the process of assessing the academic rigour and general suitability of the diploma as a route to higher education.

In fact, although the Advanced Diploma will be considered worth three A Levels, anyone applying for Engineering degrees at a decent university will need to take A Levels alongside it.

Cambridge University says that “Students wishing to apply with this qualification must also have an A-level in physics”, and Bristol University, for example, is equally blunt: while some Faculties will accept Diplomas as full qualification for entry, “Mechanical Engineering [will need an] Engineering Diploma grade A, plus A grades in A level Maths & Physics.

Ed Balls,the UK Schools Secretary, has said the the Diplomas will become “the qualification of choice”, and Schools Minister Jim Knight believes that this

&hellip statement recognises that the diploma is a demanding qualification and that students who work hard and achieve highly in their diploma will be able to study at any university they choose.

I don’t think A Level Physics will be replaced by the Diploma any time soon.

And Another Thing…

Why do the top A Level stories in the newspapers, especially front page photo ones, always exclude successful boys. Or, for that matter, any but the most pretty girls? Or black students?

Yes, I do know the answer, but it hardly matches the equality rhetoric of their editorials.

Defensive Statistics

The results are in, so now is the time for all conscientious Physics teachers to analyse their A Level results.

The newspapers are always first off the mark with ranked tables of gross percentages, showing what proportion of each grammar or independent school gained A or B grades. The tables are then dissected and the top schools pronounced. That they are always selective, either academically or socially, will not be dwelt upon.

Within schools, the Physics teachers will have a couple of weeks to analyse the results of their own students. There are two pressing reasons: to properly assess you own performance, and to have a ready defence against naive assessments by managers.

There will be decisions made using unjustified or unreliable comparisons between very different subjects and with small sample problems. (Why on Earth do class stats get reported to three significant figures?) There will be a need to explain that Johnny did not necessarily underperform in Physics just because he got higher grades in Computing and English.

I have had a cross Head of Sixth Form who presented data showing that half the Physics class had Physics as their worst grade at AS Level. Was I really suggesting that Physics was harder than other subjects? – he asked me. Well, er, yes.

It would be funny if important decisions weren’t being made based on these amateur stats and analysis.