Yes, it’s that time again, when the OECD releases the PISA test results and Education Ministers everywhere frantically start to spin the information to justify whatever plans they already had. Statisticians in government departments everywhere lurch across desks in darkened rooms, poring over the data, eagerly cherry picking the bits that serve their Minister’s purpose. Such fun!
Then there are those dedicated researchers who put out articles quick-smart explaining why PISA is flawed and unreliable. They explain in great detail the ins and outs of data collection and test setting and statistical analysis and, despite our best efforts, maybe one in a thousand of us can follow what they are saying. But we read anyway and nod sagely. Because there are graphs and there is data, so it must be good stuff.
The media, of course, enter into some kind of Nirvana, gleefully whipping up a hoohah about countries “slipping down” or “surging up” the tables. Heaven forbid a country has the temerity to stay in the same place – how’s a journo meant to get a headline out of that kind of carry on?
Of course, in all of this madness, we could take the Yong Zhao route and denounce PISA altogether – say no to the sausage factory. But that doesn’t sell papers or make for rousing Ministerial pronouncements, or even attract blog readers, so, yeah nah.
Instead, yet again, we will be treated to the PISA circus, like it or not, so please remember to engage your critical thinking skills.
It is worrying that in today’s NZ Herald Hekia Parata again conflates poverty and socio-economic status, and to further confuse matters throws in decile ratings as if the three things are the same. They are not.
Either she doesn’t know the differences or she chooses to ignore them, and I’m not sure which. Either way, she continues to mislead to public.
The Difference Between Socio-Economic Status and Poverty
Socio-economic status is far more complex than poverty. SES takes into consideration a far wider set of factors such as parents’ education achievements, occupation, social status, neighbourhood and so on.
Researchers looking at the impact of SES on student achievement will look at such things as how many books a home has in it, what art work it has, whether there is a desk to work at, how many parents there are, even considering matters such as mental health, birth weight and drug habits.
SES is not merely about income. SES is not the same as poverty.
Expert Opinion on the Impact of Socio Economic Status on a Student’s Educational Success
The 18% in the early part of the PISA report that Parata likes to quote actually refers to the effects of poverty alone – not socio-economic status. Remember, they are not the same thing. The same report she misquotes goes on to say that the impact of socio-economic status is around 75%. That is in stark contrast to Parata’s assertions, is it not?
Stephen Machin, in his 2006 OECD report on Social Disadvantage and Educational Experiences, notes:
“The evidence from empirical research is that education and social disadvantage are closely connected and that people from less advantaged family backgrounds acquire significantly less education than their more advantaged counterparts.
This translates into significantly reduced life chances as individuals’ economic and social outcomes as adults are significantly hampered by lower education levels owing to social disadvantage.”
The fact is, whilst teacher quality is a big *in-school* factor for student success, the out-of-school factors – the socio-economic factors impacting the student every single day – have by far the biggest impact overall.
And if we are not addressing those adequately, we are merely tinkering at the edges.
~ Dianne Khan, SOSNZ
Sources and further reading:
Hekia Parata: Socio-economic factors are often overstated, NZ Herald, 6/11/15
“The Minister uses the figure that 18% of the achievement gap is caused by socio-economic background”, says Dr Gordon.
“That figure came from a wrongly calculated OECD report, and is significantly out of kilter with the overwhelming evidence by the OECD itself that social factors are the key determinant of educational outcomes, across nations, across cultures, across schooling systems, public or private, large or small.”
Dr Gordon says that she does not know why the Minister continues to use a discredited figure.
“What does the research say? It says that children from high-education homes with more than 500 books, a bedroom for every child, a computer for learning and a range of other factors start school around two years ahead of those in the poorest, education-poor areas. Not only that, but the kids who are ahead in the race have all their ducks in a row to spring ahead even further.
“By age 15, the average literacy and numeracy gap between the 500- plus book group, and the fewer-than-10 book families, is over three years of learning using the OECD’s own index of learning.
“Those at the lower end have more barriers to learning than those at the top, and this is made worse by harder lives, worse conditions and fewer resources.
“In an NCEA system, where there are multiple routes and a number of pathways to achieving qualifications, the numbers of children from poor families achieving NCEA at levels 1 and 2 has expanded. This is because the changed system allows people with different abilities to turn these into qualifications. It does not mean that the wealth and resource gap has closed”.
“External factors such as high levels of child poverty (nearly every child in each decile 1-3 school, plus others, now lives in a family where there are never enough resources to meet all the family needs) and the flight from low decile schools (making those schools smaller and removing economies of scale) make these gaps worse.
“School resources and programmes, such as health-promoting schools, social workers, PB4L and other schemes work the other way, to close the gap.
QPEC wishes the Minister to accept the evidence for the huge socio-economic barriers to learning and work to design a system that will properly overcome these.
As we look into the evidence on this one, let’s be clear on one point right from the start: let’s understand the difference between “destiny” and “probability”. And, if we don’t want decile to be destiny, then what are we doing about it!
QPEC firmly holds the view that every student should get the greatest opportunity possible to succeed to the fullest extent of their abilities and their willingness to work hard and achieve.
Neither does QPEC accept that students from disadvantaged backgrounds cannot succeed.
But, the evidence on this one is clear.
A major study of the teaching profession, carried out by the OECD in 2005, made this statement in their summary paper:
“Student learning is influenced by many factors, including: students’ skills, expectations, motivation and behaviour; family resources, attitudes and support; peer group skills, attitudes and behaviour; school organisation, resources and climate; curriculum structure and content; and teacher skills, knowledge, attitudes and practices. Schools and classrooms are complex, dynamic environments, and identifying the effects of these varied factors, and how they influence and relate with each other – for different types of students and different types of learning — has been, and continues to be, a major focus of educational research.
Three broad conclusions emerge from research on student learning. The first and most solidly based finding is that the largest source of variation in student learning is attributable to differences in what students bring to school – their abilities and attitudes, and family and community background. Such factors are difficult for policy makers to influence, at least in the short-run. The second broad conclusion is that of those variables which are potentially open to policy influence, factors to do with teachers and teaching are the most important influences on student learning. In particular, the broad consensus is that “teacher quality” is the single most important school variable influencing student achievement.” [Emphasis added]
Source: OECD, “Teachers Matter: Attracting, Developing and Retaining Effective Teachers
The problem with the OECD approach – we can’t change the kids, so let’s focus on the teachers – is that it does not deal head on with what the OECD itself calls, the first and most solidly based finding:
Factors associated with the student are the largest source of variation in student achievement.
It is important to go beyond ideology and examine the hard evidence of the strong links between student background and student achievement. Failure to diagnose this correctly leads to two major problems.
– First, we miss the main goal, which is how do we improve children’s lives;
– and second, education policy initiatives are misdirected.
Teachers and schools are part of the solution; they are not the cause of the problem.
Table 1: Percentage of school leavers with NCEA Level 2 or above, by ethnic group and school quintile (2012 data)
KEY to Ethic Groups: M=Maori, P = Pasifika, A=Asian, M = MELAA, O=Other, E=European
Quintile 1 = deciles 1 & 2, etc; MELAA = Middle Eastern, Latin American & African.
The table above reports NCEA Level 2 school leaver achievement levels by school quintile, gender and ethnicity. Of students from quintile 5 (deciles 9 & 10) schools, 89.6% of them left school with at least NCEA Level 2, compared with only 58.1% for those in quintile 1 (deciles 1 & 2) schools.
Socio-economic advantage is clearly a major predictor of educational achievement.
Table 2: PISA Reading Literacy, ranked by the student’s socio-economic status, across the 10 highest performing school systems (PISA 2009 Reading Literacy):
In this table, the 5th percentile means the lowest 5% and the 95th percentile is the highest 95% of students, measured on the OECD’s own index of economic, cultural and social indicators.
So, this table is slightly different from our NCEA L2 table, because it shows the student’s own status, rather than where they go to school.
But the pattern is indisputable:
Student achievement rises lockstep with socio-economic status in every school system.
QPES Press Release
“PISA shock” is the term that has been coined for the sense of political crisis and knee-jerk policy reaction that typically occurs when a country drops in the Organisation for Economic Co-operation and Development’s international education rankings.
Almost 100 education experts, including several of us from New Zealand, recently sent an open letter to the OECD’s chief education spokesperson, Andreas Schleicher, pointing out that the Programme of International Student Assessment (PISA) rankings have become more of a problem than a solution.
In many ways the concern of New Zealand educators is also around the wider influence of the OECD education programme on New Zealand educational politics and policy – perhaps as much “OECD hangover” as “PISA shock”.
New Zealand’s government led by prime minister John Key often draws on the authority of the OECD to endorse its policy direction, using both PISA findings and the related arguments of Schleicher. New Zealand’s minister of education Hekia Parata regularly quotes Schleicher, saying, “Without data, you are just another person with an opinion.”
But Schleicher is not close enough to the New Zealand context to correct any misuse of the OECD’s results by the Key government, and this causes problems. One example has involved the impact of poverty on student achievement.
Shortly after the latest PISA results came out in 2013, Parata started to say that New Zealand’s PISA results showed that socio-economic status accounted for only 18% of student achievement. This was surprising, to say the least, when a powerful relationship between social class and student achievement has been a theme of international research in the sociology of education for more than 50 years.
Further investigation revealed that the 18% claim was based only on PISA’s narrow definition of family socio-economic influence. Using PISA’s wider criteria that include neighbourhood and school socio-economic factors, about 78% of New Zealand’s latest results became explained by socio-economic conditions. But it is worth noting that some academics, such as Harvey Goldstein at Bristol, point out that PISA is not a long-term study, and so can’t estimate factors like this.
But downplaying the impact of poverty in order to emphasise the responsibility of teachers to raise achievement has been a regular strategy of the Key government. And when faced with the corrected figure by opposition parties in parliament in January, the prime minister still fell back on the authority of Schleicher to argue for the greater importance of teachers and schools.
Adding insult to injury, Schleicher himself started arguing that “poverty isn’t destiny” and arrived in New Zealand for the International Summit on the Teaching Profession in March stressing the power of high expectations in the face of social contexts.
Some of the points Schleicher has been making might be useful if the arguments were employed carefully. Unfortunately, in the national politics of New Zealand – and probably in many other countries – any such subtleties are quickly lost. Instead the OECD/Schleicher arguments become fertile ground for the politics of blaming teachers for the underperformance of students from poor backgrounds.
The OECD hangover in New Zealand goes far beyond PISA. In January, the Key government’s latest school policy proposal called “Investing in Educational Success” was announced. This is intended to introduce a number of new teaching and leadership roles into New Zealand’s schools, providing extra payments for carrying out the required roles as part of a NZ$359m (£184.4m) investment plan.
By February, a four-minute video of Schleicher endorsing the policy had appeared on Parata’s National Party website. This was concerning as although the OECD tries to be non-partisan, here was Schleicher, featuring on a party-political website and endorsing the governing coalition’s announcement of new education spending in an election year.
Watching the highly scripted video clip it also becomes apparent that Schleicher was willing to endorse the new policy without entering into the controversies it would cause in New Zealand.
He leaves out how the policy was announced by the Key government after a cabinet decision, without prior consultation and only subsequent input into the detail rather than the thrust of the policy. Not mentioned are worrying shifts in the power relations between the New Zealand government and teachers and between New Zealand teachers themselves that are likely to be caused by the policy.
Also not mentioned is the involvement and reinforcement of other New Zealand education policies that have been causing concern such as the new National Standards for primary schools, as well as many practical considerations. Instead, Schleicher discusses the policy only in an abstract, non-contextualised way. As the Quality Public Education Coalition pressure group said, his endorsement evokes the “best of all worlds”.
With Schleicher’s endorsement of the policy there can be no claim of misinterpretation by the Key government. It is more that Schleicher is not being careful enough about how the OECD’s support would be used in a local setting.
Our open letter concluded by suggesting the OECD had become the “global arbiter of the means and ends of education around the world”. In New Zealand, the OECD risks becoming known as a stick to beat educators with. Its reputation is unlikely to improve until it starts genuinely listening and acting on local concerns.
Over 100 academics last week wrote to Andreas Schleicher at the OECD asking that PISA tests be halted. The Guardian, along with many others, ran articles on this – and the Guardian’s article elicited a response from Schleicher, in which he says
“There is nothing that suggests that Pisa, or other educational comparisons, have caused a “shift to short-term fixes” in education policy. On the contrary, by opening up a perspective to a wider range of policy options that arise from international comparisons, Pisa has provided many opportunities for more strategic policy design. It has also created important opportunities for policy-makers and other stakeholders to collaborate across borders. The annual International Summit of the Teaching Profession, where ministers meet with union leaders to discuss ways to raise the status of the teaching profession, is an example. Not least, while it is undoubtedly true that some reforms take time to bear fruit, a number of countries have in fact shown that rapid progress can be made in the short term, eg Poland, Germany and others making observable steady progress every three years…”
Harvey Goldstein responded to that letter and, as The Guardian didn’t print it, he has given me permission to share it here:
To: Editor, The Guardian
Andreas Schleicher (letters May 8) claims that, as a result of educational policy changes induced by PISA comparisons, ” a number of countries have in fact shown that rapid progress can be made in the short term”. What he means, of course, is that by concentrating efforts on performing well on the PISA tests these countries have managed to climb up the PISA rankings. This is, however, precisely the point made in the letter to him from a number of academics, including myself, to which he is responding. What we were objecting to was the way in which the relentless cycle of global testing impoverishes educational systems by promoting educational uniformity via concentration on performing well on globally standardised tests.
In fact, as Dr Schleicher well knows but refuses to acknowledge, PISA results in themselves are unable to tell us why particular countries do well or badly, and the results are typically interpreted by policymakers in order to justify their own existing predilections for curriculum reform. As we suggested in our letter, this is a good time for OECD to reflect on its PISA (and similar) programmes by suspending the next round of testing and instituting a global debate that involves all stakeholders.
Read more on PISA here:
“We are deeply concerned that measuring a great diversity of educational traditions and cultures using a single, narrow, biased yardstick could, in the end, do irreparable harm to our schools and our students.“
Dear Dr Schleicher,
We write to you in your capacity as OECD’s (Organisation for Economic Co-operation and Development) director of the Programme of International Student Assessment (Pisa). Now in its 13th year, Pisa is known around the world as an instrument to rank OECD and non-OECD countries (60-plus at last count) according to a measure of academic achievement of 15-year-old students in mathematics, science, and reading. Administered every three years, Pisa results are anxiously awaited by governments, education ministers, and the editorial boards of newspapers, and are cited authoritatively in countless policy reports. They have begun to deeply influence educational practices in many countries. As a result of Pisa, countries are overhauling their education systems in the hopes of improving their rankings. Lack of progress on Pisa has led to declarations of crisis and “Pisa shock” in many countries, followed by calls for resignations, and far-reaching reforms according to Pisa precepts.
We are frankly concerned about the negative consequences of the Pisa rankings. These are some of our concerns:
While standardised testing has been used in many nations for decades (despite serious reservations about its validity and reliability), Pisa has contributed to an escalation in such testing and a dramatically increased reliance on quantitative measures. For example, in the US, Pisa has been invoked as a major justification for the recent “Race to the Top” programme, which has increased the use of standardised testing for student-, teacher-, and administrator evaluations, which rank and label students, as well as teachers and administrators according to the results of tests widely known to be imperfect (see, for example, Finland’s unexplained decline from the top of the Pisa table).
In education policy, Pisa, with its three-year assessment cycle, has caused a shift of attention to short-term fixes designed to help a country quickly climb the rankings, despite research showing that enduring changes in education practice take decades, not a few years, to come to fruition. For example, we know that the status of teachers and the prestige of teaching as a profession have a strong influence on the quality of instruction, but that status varies strongly across cultures and is not easily influenced by short-term policy.
By emphasising a narrow range of measurable aspects of education, Pisa takes attention away from the less measurable or immeasurable educational objectives like physical, moral, civic and artistic development, thereby dangerously narrowing our collective imagination regarding what education is and ought to be about.
As an organisation of economic development, OECD is naturally biased in favour of the economic role of public [state] schools. But preparing young men and women for gainful employment is not the only, and not even the main goal of public education, which has to prepare students for participation in democratic self-government, moral action and a life of personal development, growth and wellbeing.
Unlike United Nations (UN) organisations such as UNESCO or UNICEF that have clear and legitimate mandates to improve education and the lives of children around the world, OECD has no such mandate. Nor are there, at present, mechanisms of effective democratic participation in its education decision-making process.
To carry out Pisa and a host of follow-up services, OECD has embraced “public-private partnerships” and entered into alliances with multi-national for-profit companies, which stand to gain financially from any deficits—real or perceived—unearthed by Pisa. Some of these companies provide educational services to American schools and school districts on a massive, for-profit basis, while also pursuing plans to develop for-profit elementary education in Africa, where OECD is now planning to introduce the Pisa programme.
Finally, and most importantly: the new Pisa regime, with its continuous cycle of global testing, harms our children and impoverishes our classrooms, as it inevitably involves more and longer batteries of multiple-choice testing, more scripted “vendor”-made lessons, and less autonomy for teachers. In this way Pisa has further increased the already high stress level in schools, which endangers the wellbeing of students and teachers.
These developments are in overt conflict with widely accepted principles of good educational and democratic practice:
No reform of any consequence should be based on a single narrow measure of quality.
No reform of any consequence should ignore the important role of non-educational factors, among which a nation’s socio-economic inequality is paramount. In many countries, including the US, inequality has dramatically increased over the past 15 years, explaining the widening educational gap between rich and poor which education reforms, no matter how sophisticated, are unlikely to redress.
An organisation like OECD, as any organisation that deeply affects the life of our communities, should be open to democratic accountability by members of those communities.
We are writing not only to point out deficits and problems. We would also like to offer constructive ideas and suggestions that may help to alleviate the above mentioned concerns. While in no way complete, they illustrate how learning could be improved without the above mentioned negative effects:
1 Develop alternatives to league tables: explore more meaningful and less easily sensationalised ways of reporting assessment outcomes. For example, comparing developing countries, where 15-year-olds are regularly drafted into child labour, with first-world countries makes neither educational nor political sense and opens OECD up for charges of educational colonialism.
2 Make room for participation by the full range of relevant constituents and scholarship: to date, the groups with greatest influence on what and how international learning is assessed are psychometricians, statisticians, and economists. They certainly deserve a seat at the table, but so do many other groups: parents, educators, administrators, community leaders, students, as well as scholars from disciplines like anthropology, sociology, history, philosophy, linguistics, as well as the arts and humanities. What and how we assess the education of 15-year-old students should be subject to discussions involving all these groups at local, national, and international levels.
3 Include national and international organisations in the formulation of assessment methods and standards whose mission goes beyond the economic aspect of public education and which are concerned with the health, human development, wellbeing and happiness of students and teachers. This would include the above mentioned United Nations organisations, as well as teacher, parent, and administrator associations, to name a few.
4 Publish the direct and indirect costs of administering Pisa so that taxpayers in member countries can gauge alternative uses of the millions of dollars spent on these tests and determine if they want to continue their participation in it.
5 Welcome oversight by independent international monitoring teams which can observe the administration of Pisa from the conception to the execution, so that questions about test format and statistical and scoring procedures can be weighed fairly against charges of bias or unfair comparisons.
6 Provide detailed accounts regarding the role of private, for-profit companies in the preparation, execution, and follow-up to the tri-annual Pisa assessments to avoid the appearance or reality of conflicts of interest.
7 Slow down the testing juggernaut. To gain time to discuss the issues mentioned here at local, national, and international levels, consider skipping the next Pisa cycle. This would give time to incorporate the collective learning that will result from the suggested deliberations in a new and improved assessment model.
We assume that OECD’s Pisa experts are motivated by a sincere desire to improve education. But we fail to understand how your organisation has become the global arbiter of the means and ends of education around the world. OECD’s narrow focus on standardised testing risks turning learning into drudgery and killing the joy of learning. As Pisa has led many governments into an international competition for higher test scores, OECD has assumed the power to shape education policy around the world, with no debate about the necessity or limitations of OECD’s goals. We are deeply concerned that measuring a great diversity of educational traditions and cultures using a single, narrow, biased yardstick could, in the end, do irreparable harm to our schools and our students.
Andrews, Paul Professor of Mathematics Education, Stockholm University
Atkinson, Lori New York State Allies for Public Education
Ball, Stephen J Karl Mannheim Professor of Sociology of Education, Institute of Education, University of London
Barber, Melissa Parents Against High Stakes Testing
Beckett, Lori Winifred Mercier Professor of Teacher Education, Leeds Metropolitan University
Berardi, Jillaine Linden Avenue Middle School, Assistant Principal
Berliner, David Regents Professor of Education at Arizona State University
Bloom, Elizabeth EdD Associate Professor of Education, Hartwick College
Boudet, Danielle Oneonta Area for Public Education
Boland, Neil Senior lecturer, AUT University, Auckland, New Zealand
Burris, Carol Principal and former Teacher of the Year
Cauthen, Nancy PhD Change the Stakes, NYS Allies for Public Education
Cerrone, Chris Testing Hurts Kids; NYS Allies for Public Education
Ciaran, Sugrue Professor, Head of School, School of Education, University College Dublin
Deutermann, Jeanette Founder Long Island Opt Out, Co-founder NYS Allies for Public Education
Devine, Nesta Associate Professor, Auckland University of Technology, New Zealand
Dodge, Arnie Chair, Department of Educational Leadership, Long Island University
Dodge, Judith Author, Educational Consultant
Farley, Tim Principal, Ichabod Crane School; New York State Allies for Public Education
Fellicello, Stacia Principal, Chambers Elementary School
Fleming, Mary Lecturer, School of Education, National University of Ireland, Galway
Fransson, Göran Associate Professor of Education, University of Gävle, Sweden
Giroux, Henry Professor of English and Cultural Studies, McMaster University
Glass, Gene Senior Researcher, National Education Policy Center, Santa Fe, New Mexico
Glynn, Kevin Educator, co-founder of Lace to the Top
Goldstein, Harvey Professor of Social Statistics, University of Bristol
Gorlewski, David Director, Educational Leadership Doctoral Program, D’Youville College
Gorlewski, Julie PhD, Assistant Professor, State University of New York at New Paltz
Gowie, Cheryl Professor of Education, Siena College
Greene, Kiersten Assistant Professor of Literacy, State University of New York at New Paltz
Haimson, Leonie Parent Advocate and Director of “Class Size Matters”
Heinz, Manuela Director of Teaching Practice, School of Education, National University of Ireland Galway
Hughes, Michelle Principal, High Meadows Independent School
Jury, Mark Chair, Education Department, Siena College
Kahn, Hudson Valley Against Common Core
Kayden, Michelle Linden Avenue Middle School Red Hook, New York
Kempf, Arlo Program Coordinator of School and Society, OISE, University of Toronto
Kilfoyle, Marla NBCT, General Manager of BATs
Labaree, David Professor of Education, Stanford University
Leonardatos, Harry Principal, high school, Clarkstown, New York
MacBeath, John Professor Emeritus, Director of Leadership for Learning, University of Cambridge
McLaren, Peter Distinguished Professor, Chapman University
McNair, Jessica Co-founder Opt-Out CNY, parent member NYS Allies for Public Education
Meyer, Heinz-Dieter Associate Professor, Education Governance & Policy, State University of New York (Albany)
Meyer, Tom Associate Professor of Secondary Education, State University of New York at New Paltz
Millham, Rosemary PhD Science Coordinator, Master Teacher Campus Director, SUNY New Paltz
Millham, Rosemary Science Coordinator/Assistant Professor, Master Teacher Campus Director, State University of New York, New Paltz
Oliveira Andreotti Vanessa Canada Research Chair in Race, Inequality, and Global Change, University of British Columbia
Sperry, Carol Emerita, Millersville University, Pennsylvania
Mitchell, Ken Lower Hudson Valley Superintendents Council
Mucher, Stephen Director, Bard Master of Arts in Teaching Program, Los Angeles
Tuck, Eve Assistant Professor, Coordinator of Native American Studies, State University of New York at New Paltz
Naison, Mark Professor of African American Studies and History, Fordham University; Co-Founder, Badass Teachers Association
Nielsen, Kris Author, Children of the Core
Noddings, Nel Professor (emerita) Philosophy of Education, Stanford University
Noguera, Pedro Peter L. Agnew Professor of Education, New York University
Nunez, Isabel Associate Professor, Concordia University, Chicago
Pallas, Aaron Arthur I Gates Professor of Sociology and Education, Columbia University
Peters, Michael Professor, University of Waikato, Honorary Fellow, Royal Society New Zealand
Pugh, Nigel Principal, Richard R Green High School of Teaching, New York City
Ravitch, Diane Research Professor, New York University
Rivera-Wilson Jerusalem Senior Faculty Associate and Director of Clinical Training and Field Experiences, University at Albany
Roberts, Peter Professor, School of Educational Studies and Leadership, University of Canterbury, New Zealand
Rougle, Eija Instructor, State University of New York, Albany
Rudley, Lisa Director: Education Policy-Autism Action Network
Saltzman, Janet Science Chair, Physics Teacher, Red Hook High School
Schniedewind, Nancy Professor of Education, State University of New York, New Paltz
Silverberg, Ruth Associate Professor, College of Staten Island, City University of New York
Sperry, Carol Professor of Education, Emerita, Millersville University
St. John, Edward Algo D. Henderson Collegiate Professor, University of Michigan
Suzuki, Daiyu Teachers College at Columbia University
Swaffield, Sue Senior Lecturer, Educational Leadership and School Improvement, University of Cambridge
Tanis, Bianca Parent Member: ReThinking Testing
Thomas, Paul Associate Professor of Education, Furman University
Thrupp, Martin Professor of Education, University of Waikato, New Zealand
Tobin, KT Founding member, ReThinking Testing
Tomlinson, Sally Emeritus Professor, Goldsmiths College, University of London; Senior Research Fellow, Department of Education, Oxford University
Tuck, Eve Coordinator of Native American Studies, State University of New York at New Paltz
VanSlyke-Briggs Kjersti Associate Professor, State University of New York, Oneonta
Wilson, Elaine Faculty of Education, University of Cambridge
Wrigley, Terry Honorary senior research fellow, University of Ballarat, Australia
Zahedi, Katie Principal, Linden Ave Middle School, Red Hook, New York
Zhao, Yong Professor of Education, Presidential Chair, University of Oregon
Diane Ravitch notes:
In my recent book, Reign of Error, I quote extensively from a brilliant article by Keith Baker, called “Are International Tests Worth Anything?,” which was published by Phi Delta Kappan in October 2007. Baker, who worked for many years as a researcher at the U.S. Department of Education, had the ingenious idea to investigate what happened to the 12 nations that took the First International Mathematics test in 1964.
He looked at the per capita gross domestic product of those nations and found that “the higher a nation’s test score 40 years ago, the worse its economic performance on this measure of national wealth–the opposite of what the Chicken Littles raising the alarm over the poor test scores of U.S. children claimed would happen.”
He found no relationship between a nation’s economic productivity and its test scores.
Nor did the test scores bear any relationship to quality of life or democratic institutions.
Read the rest here: My View of the PISA Scores.
Fair enough, I understand that – I am passionate too, and immediately want to know what the results do or do not tell us about how NZ is doing, educationally.
But surely it’s a time to read, reflect, research, and discuss the findings and the study itself, and try to eke out something meaningful form it, rather than just jump in and score Brownie points?
The goal is to see where we can improve things for our learners, after all.
Below are the Ministry’s main take away points from the study, copied verbatim from their web site.
I am going to refrain from commenting or adding my own observations or thoughts for now, as I would rather people read them with an open mind and ask questions of them.
Here goes – get your thinking caps on:
In New Zealand, over 5,000 students (4,291 for core PISA subjects, 958 for financial literacy) from 177 schools took part in the study, in July 2012.
I’d love to hear others’ observations, in the comments below or on the Facebook page.
The big drop in New Zealand’s student achievement in recent years is a direct result of a failure of this government’s education, economic and social policies, says NZEI Te Riu Roa National President Judith Nowotarski.
Mrs Nowotarski says the results are a clear wake up call to the government.
She says the Minister of Education has overseen one of the biggest drops in our student ranking in recent years.
The OECD PISA results, measuring 15 year old student achievement in science, maths and reading, shows that since 2008 New Zealand has had one of the fastest-growing rates of inequity as well as a corresponding decrease in student performance.
“Across the board New Zealand’s performance has dropped on all the scores and this is something that the government should be ashamed of. It shows its policies are nothing short of disastrous.”
“For five years the government has been obsessed with collecting unnecessary and irrelevant data when it should have been focussed on making a difference for students.
“The Minister is being misleading when she claims that this decline is a long-running trend. By far the biggest drop in achievement has occurred since 2009.
“This government’s obsession with data combined with no solutions for failing education policies has been a disaster for many New Zealand children. Instead of working with teachers and schools to improve education, the government has been hell-bent on dismantling our public education system.
“Countries that have a higher level of equity also have better achievement outcomes for all students. Eighteen percent of New Zealand children now live in poverty and the student achievement gap reflects the impact that poverty has on students’ learning.
“All the findings are saying the same thing. It’s now time for the government to start to look at what really works in education.
“We need the government to work with teachers and schools to restore our education system to its previous top performing level instead of having one of the fastest levels of decline in the OECD.”
Like a dog with a bone, Hekia is at it again:
Ms Parata said socio-economic status or decile was “not destiny”. There were many examples of schools and students from low-decile areas achieving strong results. (1)
“Educationally, the evidence is that students can make good progress based on the quality of teaching they get, not on their socio-economic background.” (1)
Yes, Mrs Parata, some do achieve well despite their socio-economic background, but many more don’t. Why are you not addressing that issue?
I get that some teachers are better than others, that some schools are better than others, but it is galling that this government keep making out like poverty has no part in student achievement and opportunity.
To refuse to consider that part of the problem is that schools are not funded equally. To deny that students’ home lives have a huge influence on their education.
Nope, again Hekia trots out a platitude or two that shore up the belief that we need do nothing to rectify the poverty of our people and, for bonus points, blames any problem on teachers.
It’s hogwash and gets us nowhere.
And it’s shameful for a Minister to act this way.
Sources and further reading:
(1) Schools divided along wealth lines, By Nicholas Jones, The New Zealand Herald, retrieved 11.23AM Tuesday Jul 9, 2013
Who achieves what in secondary schooling? Executive summary (162.67KB, 7 pages)
The results, presented by researchers Liz Gordon and Brian Easton today, reveal the simplistic nature of the claim and the complex issues being ignored every time it is made.
PPTA president Angela Roberts said the overlapping issues of ethnicity, gender and socio-economic status were ignored when simplistic figures such ‘1 in 5’ or ‘20% of students are failing’ were bandied about.
“The message of there being a crisis in schooling is being used to drive through radical policies, but there is not a crisis. There are challenges and we need to deal with these by recognising the complexity of the issues,” she said.
The government’s practice of separating out a single factor – such as ethnicity – and comparing one sub-group to other whole populations was “statistically grossly misleading” and failed to recognise many of the factors contributing to underachievement, Roberts said.
The closest to the politically popular 20% figure the researchers were able to find was that 14.3% of students failed to achieve proficiency level 2 on PISA reading – and a closer examination of this group showed that 74% were male and that socio-economic factors such as parental income and the number of books in the home were clearly contributing issues.
“Constantly focussing on ethnicity as a single factor fails to recognise these overlapping issues,” Roberts said.
A companion report by Easton also contains data that suggests the constant labelling of ‘underachiever’ has had an impact on how students identify themselves ethnically.
Roberts hoped the research would enable the government to take a more sophisticated and nuanced approach to educational achievement and recognise the dangers of over-simplification.
“We hope that politicians and editorial writers will stop throwing around figures like ‘1 in 5’ and ‘national disgrace’ when in reality the issues are much more complicated.”
For links to the full reports and summaries, go here.
or “Who is behind us and why?”
or “Who is slipping backwards in the ratings, and, well what are they doing that we must avoid like the plague?”
Okay now we’ve got the title sorted, let’s see how the 15 year olds are doing around the world…
And more to the point – how the Kiwis are doing… *
|2009 Programme for International Student Assessment — test scores|
|#||Reading – Overall||Mathematics||Science|
|1||China: Shanghai||556||China: Shanghai||600||China: Shanghai||575|
|3||Finland||536||Hong Kong||555||Hong Kong||549|
|7||New Zealand||521||Liechtenstein||536||New Zealand||532|
|12||Norway||503||China: Macao||525||Chinese Taipei||520|
|23||Chinese taipei||495||Slovak Republic||497||United States||502|
|PISA average :||501|
|PISA average :||496|
|PISA average :||493|
|Reading – Overall||Mathematics||Science|
|28||China: Macao||487||United Kingdom||492||Iceland||496|
So, Out of 65 countries worldwide, New Zealand is 7th for Reading and Science, and 13th for Maths.
Now, where are the UK and USA – because we are following in their footsteps with Charter Schools, so they must be good, right?
Why would we want to copy the faulty ideology of countries we are thrashing hands down?
If we want to improve even further, what about maybe paying attention to one of the countries consistently above us, instead?
* Sorry the table was so big, but I had to make it that long to keep the USA and UK visible there near the bottom…
Reviewers of the data noted that “New Zealand’s education system has won major praise with it nearing the top in literacy, mathematics and science according to a highly recognised international assessment system. But the data points to some alarming gaps in New Zealand – especially socio-economic.”  So we are doing well despite the shocking gaps between those with much and those with little. Go figure.
So just how well did we do?
Overall, in 2009 New Zealand was ranked 5th out of 34 OECD countries for mean PISA scores across reading, mathematics and science.
Fifth. Fifth out of thirty four. FIFTH!
Where was the USA? The UK? Aus? Below NZ, not above. So next time a politician stands up and talks about education here in the God Zone, just remember – 5th in the world.
Are the 2012 statistics a fluke?
No, they’re not. NZ consistently performs well, as shown in the 2000, 2002 and 2003 information below: 
Over three-quarters (76 percent) of New Zealanders aged 25–64 years have achieved secondary or tertiary educational qualifications.
This is at the upper end of the OECD scale, placing New Zealand twelfth among 30 nations, slightly behind Austria and ahead of Finland, and well above the OECD average of 65 percent.
There is considerable variation in the proportion of people holding qualifications, from 13 percent in Mexico to 88 percent in the Czech Republic.
|Educational Attainment(percentage of 25–64 year olds attaining at least upper secondary education), 2002
High rates of early childhood education
New Zealand also has higher rates of participation in early childhood education than most other OECD countries.
Ninety-three percent of New Zealand four year olds were involved in early childhood education in 2000, compared with an OECD average of just 73 percent. New Zealand ranked ninth in the proportion of four year olds in education.
|Education(proportion of 4 year olds in primary or pre-primary education), 2000
New Zealand children rank relatively highly on international literacy scales.
The OECD Programme for International Student Assessment measures performance levels of students near the end of compulsory education in reading, mathematical and scientific literacy.
The data shows that New Zealand children rank seventh among OECD countries, with comparable data in terms of the average score across the three scales, behind Finland, Korea, Japan, Canada, the Netherlands and Australia. New Zealand rates above the OECD average on each of the scales – fifth in reading, ninth in mathematics and seventh in science.
| Student Literacy(student performance on the combined reading, scientific and mathematical literacy scales), 2003
So just how well did we do in the latest statistics?
The science results for 15 year olds were topped by Shanghai (575), Finland (554), Singapore (542) and New Zealand (532). That means we are the 4th best in science out of 65 countries. The OECD average was 501.
The top reading literacy scores for 15-year-olds showed Korea (with a score of 539), Finland (536), Hong Kong-China (533), Singapore (526), Canada (524) and then New Zealand (521). New Zealand was well above the OECD average of 493. It’s worth noting that all but one of the countries out-performing NZ there have education systems based on equitable education for all, rather than competition. Singapore is the only exception to that. None have charter schools.
We did very well in maths, too.
It’s worth remembering all of this and celebrating how well we do as a country.
That’s not to say we don’t have areas in need of careful focus and improvement – of course we do. All teachers know that – we all want that. We want to be able to easily get access to professional development so we can enhance the skills we have. We want to lift achievement in immigrants, Pacific Islanders and Maori students so that they stand a better chance of achieving the same as other groups. We want to address the huge and worrying disparity in achievement between the haves and the have-nots.
It’s a great system we have, and a wonderful one to build on and improve further – it isn’t in need of a complete overhaul, just the trust and respect of those in charge, and a willingness to listen to our advice, suggestions and ideas.
Then we can all get on with more of what we do well – teaching.