A critique of the Competent Children Study (Competent Learners Research Project)
By Sarah Alexander and Gillian Croad.
The Competent Children Study.
The Competent Children/ Learners Study was cited by Government as providing evidence of long-lasting effects of early childhood education on children’s academic achievement and competencies, and used to justify the expansion of government influence over service provision. The research received much political and professional acclaim in New Zealand for its demonstration of the positive effects of early childhood education on children’s competencies and educational achievement.
This paper grew from our initial interest in reading the evidence that supported what the Minister and Ministry of Education were saying about the Competent Children study. What we found when we took the time to read the New Zealand Council for Educational Research’s reports on the Competent Learners research was quite different to what we had been led to believe about it.
Our paper is concerned not so much with the quality of the research because every piece of research has strengths and limitations, but with the political importance it attained in the absence of much critical review, discussion and understanding of the full research reports.
The assessment reported here suggests that the findings have been over-generalised beyond that warranted by the data. The use of the research to legitimate public policy and spending should be questioned.
Given the political significance Competent Children has attained there would appear to be a strong case for comprehensive critical analysis of this research by policy people, Ministry of Education officials, and early childhood researchers and professional leaders.
NZ Journal of Teachers’ Work
Here is what we wrote in the NZ Journal of Teachers’ Work, Vol. 2, No. 1, 17-19, 2005 “Being a Critical Consumer of Research is Important: Consider the Political Spin and Shortcomings of NZCER’s Competent Children Study, for example”
A Ministry of Education (2001) paper to the OECD states that ‘the Competent Children Study has been and is a significant resource … the study is the only one of its kind in the New Zealand context, and has, over an extended period, influenced policy and practice’ (pp. 10-11).
The Minister of Education Trevor Mallard has for a number of years cited the research to back stated policy directions and funding decisions in early childhood education. In a media statement on 1 June 2005, the Minister of Education stated that:
As I have repeatedly said, that research shows that regular, quality and intensive early childhood education makes a positive impact on children’s learning later in life … The research includes a New Zealand study – the Competent Children research series, undertaken by the independent research organisation New Zealand Council for Educational Research. It has so far followed a group of around 500 Wellington region children from age 5 to age 12, and measured their key competencies and what affects their learning.
We had become increasingly concerned about how the findings were being reported widely and rather superficially with very little critical concern about it expressed within the education sector. Given the political and professional prominence given to the research we were aware that a review of the research in relation to the claims made about it would not be well received by everyone. However, we believe that publicly funded policy-driven research which is used to legitimatise education policy and spending should not be exempted from scrutiny and critique.
The research has relied on statistical analysis to tease out associations between the variables selected for study. As the researchers have noted, any significant associations between variables cannot provide ‘proof’ of relations between individual factors. Yet, some of the resulting discourse on the implications for teaching practice flowing from the reported analyses appears to ignore this distinction, reflecting a rather unsophisticated understanding of how such research should inform practice.
For example, a small number of benefits of ‘quality’ early childhood education were found in the research, but in the absence of a good understanding of the research methodology and the subsequent limitations, meaningful discussions of these are not possible. Given
- the sheer number of different aspects of children’s lives,
- the fact that the study sampled children at the end of their early childhood education experience only, and
- that most of the study since then has been of their experience within the compulsory school system,
it seems possible that even these benefits may be due to uncontrolled factors rather than the benefits of early childhood education alone.
Our review, points to a number of shortcomings in the research, including the sampling technique and the composition of the sample, the research design, data analysis, and interpretations of the data. Assumptions made in the research about children’s competencies, quality in early childhood education and the measurement of quality therefore need to be questioned.
The research was reviewed in relation to eight claims about its importance for early childhood education.
- It is a study of some 500 children
- It drew on a random selection of centres
- It is nationally and internationally recognised
- It provides proof of the educational benefits of early childhood education
- It is a flagship study of early childhood education in New Zealand
- The study relates to early childhood targeting and is of important strategic value
- The study relates to early childhood resourcing and is of important strategic value
- It confirms the Government’s intention to ‘beef up’ children’s participation in quality early childhood education.
The above claims were found to be either simply incorrect or a distortion of the research in the form of putting the best ‘spin’ on it.
For example, in relation to the size of the sample, what is not often stated in media releases is that the study actually started with 307 nearly five-year-old children who were about to begin school. When the children were age 8 the researchers brought in more children from a separate study. To say now that it is a study of 500 children is misleading and masks the problems of bringing together two different sample groups with data obtained from different sources using different methodological approaches.
Our review then provides an example of, and a lesson in, why publicly funded policy-driven research such as the Competent Children study should be more critically examined and openly discussed.
We hope that readers will take the time, too, to read the research, examine it and understand it better.
Thank you.
