Adjudicator's decision on VRTS

[Etienne]

Verbal reasoning test scores

The objection was that the methods of selection being used do not comply with the Code, and in particular the requirements to ensure fair access to educational opportunity in paragraphs 1.70, 1.71 and 1.72. The objector has claimed that the tests in use do not comply with section 104 (2) of the Act (which defines selective admission arrangements as ‘selection by reference to general ability, with a view to admitting only pupils with high ability’). He has suggested that tests should be used that ‘address the unrepresentatively low percentage of children from disadvantaged backgrounds reaching the qualification standard in the 11+’. The objector believes that using VRT as the means of selection is ‘(a) not a culture-fair test’ and ‘(b) highly susceptible to out-of-school coaching’.
8. In support of his objection, the objector has drawn attention to two parts of the National Foundation for Educational Research (“NFER”) website, and has quoted from the Guide, which itself refers readers to the website, which, the objector says, ‘finds no score gain from familiarisation/practice such as provided by the Council, but concludes that “The effect of sustained coaching over a period of 9 months is shown to be substantial”‘(his italics). ‘Coaching programmes,’ the objector says, ‘are expensive, and are therefore not available to all parents and their children’.

I conclude there is uncertainty as to the benefits and contra-benefits offered by VRT in contrast with other forms of testing., and that it is by no means clear that any other testing method that the website has suggested to me would have a clear advantage. That is not to assume that other testing methods may be available that I have not considered, but it is to note that no such other methods have been proposed to me as alternatives.
16. The objector has sent me a copy of the Council's letter to him of 17th February 2010, in which, while agreeing that 'this form of test has limitations', the Council argues that 'to this date verbal reasoning tests have provided the best methodology for selection available at a reasonable cost'. It further explains that it 'run a robust appeals process to enable those children who have not been able to fully demonstrate their abilities in the test to have a wider and independent consideration of their potential'. The process for such appeals is set out clearly in the Guide. In an earlier letter to the objector of 12th June 2008, the Council said that it shared the objector's concern 'about the percentage of children entitled to free school meals or from disadvantaged backgrounds meeting the qualification standard in the 11+ tests' (an issue to which I will return in the next paragraph). The Council saw its appeals system as the remedy for this, since it was possible for appeal panels 'to take into account contextual information and personal circumstances'. Data provided for the Council's Community, Cohesion and Equalities Forum for its meeting on 30th June 2009 indicated that of, 859 selection appeals heard in January and February 2008, 331 were successful, indicating that appeals play an important part in the selection process.

I agree with the Council that it is not clear whether the NFER supports the conclusions of the researchers cited on its website, but nevertheless I must consider the possibility that there may be total or partial validity in the conclusions.

A Council officer then expressed an equal concern with that of the headteachers, and reported 'that an educational psychologist ha[d] been commissioned to look at the format of the 11+ verbal reasoning tests and consider alternative methods of testing', and that the Overview and Scrutiny Committee would be 'looking at the entire 11+ process, including the tests'. The Council has supplied me with a copy of the psychologist's report. Despite some discrepancy between the results of the various studies surveyed in the report, what seemed to be emerging was some concern that discernible, even significant, positive effects were possibly occurring in the test results of children who were given sustained coaching, and that further local research might be advisable in order to quantify this risk. I asked the Council for a copy of the relevant Overview and Scrutiny Committee minutes, but it seems that serious consideration has not been afforded to the psychologist's report or any implications it might have for the Buckinghamshire testing regime.

It is doubtful, however, whether such additional practice and coaching could be eliminated, and it is probably an inevitable concomitant of the testing regime. It is likely that there is some unfairness in the system. However, I have not been able to discover ways in which it could be eliminated, other than be abolishing the tests. Since selection by ability at established grammar schools is permitted by law, I believe it is impossible for me to remove those elements of the selection system that appear to breach the Code. I therefore view these unfair elements as inevitable and unavoidable. It is thus for this practical reason alone that I am not upholding the objection on these grounds.

Conclusion
38. It is, in my view, undeniable that opportunities for unfairness are likely to exist in Buckinghamshire's selection system, just as they will do in any selective system, whatever the method of selection testing that is used. If selective schools exist, then selection must be carried out. If selection is carried out, then it is natural for parents, and even schools, to want children to maximise their evident ability and what they perceive as their latent ability. Some groups of children are likely to fare better than others in the Buckinghamshire tests. However, I am not persuaded that the differences are significantly and consistently radically different from what is observable for those groups when performance is measured by other means. I believe, from the reports and committee minutes that I have seen, that the Council is concerned to monitor these factors, and to work towards eliminating differences in general educational performance and attainment that accompany variety in ethnicity and degrees of affluence and social health.
39. I have been concerned at the advantage that may benefit some children for whom additional practice and coaching are available, and at the disadvantage that may be suffered by children who do not attend Buckinghamshire's independent and maintained primary schools. However, it is difficult to envisage means by which these causes of advantage and disadvantage could be totally eliminated. In any case, it is likely that these causes of advantage and disadvantage, provided they are reduced to an absolute minimum by the Council's diligent observation and persuasion, are an inevitable and inescapable by-product of any selection regime. So I must balance any advantage and disadvantage against the undisputable entitlement of the Council, its population and schools to retain selective secondary schools.
40. Although I am not upholding the objection, I encourage the Council to continue its mission to eliminate whatever unfairnesses linger with its selective system.
41. I am also making changes to the admission arrangements of schools in Buckinghamshire, as set out above.

The full decision is available here:
http://webarchive.nationalarchives.gov. ... &order=lea" onclick="window.open(this.href);return false;
(scroll down to Buckinghamshire, October 2010)

[Tree]

So in summary the VRTS is a bit c*** and probably unfair but no one has told me about anything better so hay ho not much we can do about it so we might as well keep it. It's all a bit depressing really my overall sense is that of CBA on the part of the adjudicator.

[Tree]

'run a robust appeals process to enable those children who have not been able to fully demonstrate their abilities in the test to have a wider and independent consideration of their potential'

Etienne et al. does this comment have an impact on the appeal system it seems to be implying that if you felt that the test ie a vr test was the wrong type of test to demonstrate your dc's abilities (eg if they were better at NVR for example) then this could be a basis for your appeal has this been allowed before ??

[Etienne]

Yes, it has certainly happened.

The alternative evidence for NVR/Quantitative Reasoning/Maths/Science would have to be persuasive, and some panels might be more receptive than others to the argument.

[bucksdaddy]

To quote a politician, I agree with Tree.

[Tree]

Etienne i just wondered if this adjudication could give more weight to this argument and seems to undermine the cc's stance at the start of the appeal process which is 'we believe that your child is not suitable to grammar school cos they didn't get 121 on the vtr' but in this adjudication they seem to be implying that they know the test is unfair and the appeal process is their way of compensating for this

[pippi]

So in summary the VRTS is a bit c*** and probably unfair but no one has told me about anything better so hay ho not much we can do about it so we might as well keep it. It's all a bit depressing really my overall sense is that of CBA on the part of the adjudicator.

Or as Captain Barbosa put it:

And secondly, you must be a pirate for the Admissions code to apply and you're not. And thirdly, the code is more what you'd call "guidelines" than actual rules.

The Council said that it saw its appeals system as the remedy for 'the percentage of children entitled to free school meals or from disadvantaged backgrounds meeting the qualification standard in the 11+ tests'. That seems a bit dubious on a number of grounds.

[Etienne]

Etienne i just wondered if this adjudication could give more weight to this argument and seems to undermine the cc's stance at the start of the appeal process which is 'we believe that your child is not suitable to grammar school cos they didn't get 121 on the vtr' but in this adjudication they seem to be implying that they know the test is unfair and the appeal process is their way of compensating for this

I don't really think it would undermine the LA's case at the start. The only issue for the panel at that point is whether the published arrangements are lawful, and whether the LA has implemented them. If an appeal panel believes the arrangements to be unlawful, all it can do is refer them to ....... the Adjudicator!

However, I do think it could strengthen the position of parents who want to put forward a case based on very strong alternative evidence (NVR/Quantitative Reasoning/Maths/Science).

[LoobyLu]

Para 41: " I am also making changes to ..admissions arrangements - see above" ???? Etienne, can you shed any further light??

[Etienne]

I think these are the ones Sally-Anne mentioned in the other thread:
http://www.elevenplusexams.co.uk/forum/ ... 09#p203409