What’s the real reason National are implementing progress tracking via PaCT?

Saturday morning, while all sensible people were eating second breakfast and procrastinating about the weekend chores, Nikki Kaye snuck out a little education policy announcement about National Standards.

That it came out in such an understated way was made even more odd when, on Sunday, National gave us a second three-pronged education policy announcement – and this one was an all-singing, all-dancing affair with hundreds of waving, cheering National supporters in tow.

Leaving Sunday’s announcement to one side for now, I want you to ask yourself  why was one single policy put out separately? Why the day before the bigger announcement? Why not include it in the main announcement? is it that bad that it has to be hidden away? Ponder that as you read on.

The policy announced on Saturday is that National will implement ‘National Standards Plus’. This will require teachers to input National Standards data into the ‘Progress and Consistency Tool’ (PaCT), a computer programme that ostensibly exists to take test results and use them to spit out a child’s attainment level against National Standards. PaCT will then, we are told, use students’ data to calculate their progress so that we can see the ‘value added’ to any student over a given time. It sounds quite sensible on the face of it. Who wouldn’t want to know how a child is progressing?

Input the data and voila!

And it might be good if it weren’t for a couple of pesky details.

First of all, if the data going in is not reliable then the data coming out isn’t either. Or as computer folk like to call it, GIGO – Garbage In, Garbage Out.

Problems with the unreliability of National Standards are well known. Professor Martin Thrupp outlined these issues and how they relate to PaCT in his second Research, Analysis and Insight into National Standards (RAINS) Project report, saying:

“If the Progress and Consistency Tool [PaCT] to be made mandatory by the Government is mainly intended as a form of national moderation for [Overall Teacher Judgement] -making, then it can be expected to be an expensive failure. This is because it will not be able to address many of the various influences and pressures schools and teachers face, illustrated by this report, that will lead schools to take different ‘readings’ of the National Standards and of OTJs. “

So, issues with the reliability of National Standards data relating to students are the first key problem: GIGO.

The other elephant in the room, glaring over from the sidelines, is PaCT’s role in teacher evaluation.

The announced change in how PaCT is used will see students’ data being recorded against their teachers. Again, this seems useful at first glance. Surely, people say, that would help evaluate which teachers are doing the best job? But it’s not that simple.

One issue is that students often have a burst of learning after work by many teachers over a number of years, and to attribute that only to the teacher they are currently with would be incorrect. For example, for year 0-2 teachers, it can be quite some time before the fruits of their labours come to fruition, and to attribute all gains made, say, in Year 3 to just the Year 3 teacher would be erroneous.

So GIGO problems apply as much to PaCT data relating to teachers as to students, rendering it far too unreliable to accurately judge a teacher’s impact on a student’s learning.

Nikki Kaye assured me today via Twitter that PaCT will not be used to implement performance pay, but as one of the software engineers that built PaCT warned me almost a decade ago that the capacity for this has been built into the system, this remains a concern.

All in all, this new policy seems to be a poorly thought out move. While National Standards continue to be anything but standard, PaCT will only ever be the lipstick on the National Standards pig. In other words, you can pretty National Standards up any way you want, they are still just plain shonky.

So the question remains, what’s the real reason for National implementing progress tracking via PaCT?

~ Dianne

Further reading:

Research, Analysis and Insight into National Standards (RAINS) Project – Final Report: National Standards and the Damage Done, by Martin Thrupp & Michelle White, November 2013

The Search for Better Educational Standards – A Cautionary Tale, by Martin Thrupp, (ISBN 978-3-319-61959-0)