Bucks VR test poor resolution
Moderators: Section Moderators, Forum Moderators
Bucks VR test poor resolution
I have been distilling in my mind the true problem with the current pass rate of 90% for the bucks VR test and why this is a poblem and how it makes the test both unfair and not fit for purpose.
There are 80 questions in the VR test and the test questions are chosen to produce a sort of normal distribution of results and are then used to choose around 25% of children with another 5% or so chosen on appeal. If we look at the population of children who get over 65/80 which roughly equates to about 20% of kids and over the 2 tests for kids getting over 65 on either test about 30% or so of kids then it is from this population that children who pass will be selected.
So the decision for each one of these children who have got at least 65 questions right in 1 paper as to whether that child goes to a grammar school comes down to how they perform in these 15 questions. These questions will not be the same 15 questions for every child but alot of them will be the same and i suspect the pool of questions in the paper that make up the vast majority on these questions will be about 20-25 questions in the whole paper. This means that effectively only about a quarter or a third of the test is relevant.
So in bucks the decision on selection to grammar school is not only on one measurement of ability ie verbal reasoning but this is actually only tested using about 20-25 questions and as anyone knows who has been through the coaching process that it ends up being maybe 5 or 6 types of questions out of the 21 or whatever that make up the majority of these 25, my experience whith our kids is that it ends up being mostly a test of vocabulary as most of the codes and maths based questions are amenable to practice.
So how could it be better well for a start you could widen the range of difficulty of question if the pass mark was brought down to around 55-60/80 then using the same logic as above the decision about grammars chools would be based on 40 or 50 questions per test or 100 in total thus improving the resolution.
I was very interested to see pippi's results for kent where they have 3 tests VR NVR and maths and the pass mark(120) for each equates to a raw score of around 50% thus using most of the questions in the tests to make the decision ie about 180 questions are relevant for a result with a huge increase in resolution.
I really do think it is outrageous and this test really needs modifying .
There are 80 questions in the VR test and the test questions are chosen to produce a sort of normal distribution of results and are then used to choose around 25% of children with another 5% or so chosen on appeal. If we look at the population of children who get over 65/80 which roughly equates to about 20% of kids and over the 2 tests for kids getting over 65 on either test about 30% or so of kids then it is from this population that children who pass will be selected.
So the decision for each one of these children who have got at least 65 questions right in 1 paper as to whether that child goes to a grammar school comes down to how they perform in these 15 questions. These questions will not be the same 15 questions for every child but alot of them will be the same and i suspect the pool of questions in the paper that make up the vast majority on these questions will be about 20-25 questions in the whole paper. This means that effectively only about a quarter or a third of the test is relevant.
So in bucks the decision on selection to grammar school is not only on one measurement of ability ie verbal reasoning but this is actually only tested using about 20-25 questions and as anyone knows who has been through the coaching process that it ends up being maybe 5 or 6 types of questions out of the 21 or whatever that make up the majority of these 25, my experience whith our kids is that it ends up being mostly a test of vocabulary as most of the codes and maths based questions are amenable to practice.
So how could it be better well for a start you could widen the range of difficulty of question if the pass mark was brought down to around 55-60/80 then using the same logic as above the decision about grammars chools would be based on 40 or 50 questions per test or 100 in total thus improving the resolution.
I was very interested to see pippi's results for kent where they have 3 tests VR NVR and maths and the pass mark(120) for each equates to a raw score of around 50% thus using most of the questions in the tests to make the decision ie about 180 questions are relevant for a result with a huge increase in resolution.
I really do think it is outrageous and this test really needs modifying .
Re: Bucks VR test poor resolution
This is exactly what my thoughts were when I saw that table..a much more rounded picture of ability I'd say. Not only is the Bucks test 'vehicle' squeezed into a mere 15 questions, the format is so predictable, and the results are so proportionate to the level of hot-housing the child is subjected to, that there is no way the test can be relied on as an indicator of a rounded academic ability. It is this flaw that concerned me the most when DD was sitting the test. On the one hand I don't believe in tutoring for a test like this, and on the other, I know if I don't, my child will be at a serious disadvantage relative to most of her cohort. So our personal belief gives way to 'failure', and then we are left with the hard task of 'proving' our childrens ability to a panel of three strangers..who in spite of a mountain of academic evidence and glowing recommendation...will still be cast in the glow of doubt, because they didn't quite make that magic, but seriously manipulated mark.
There are many children in the sets below my DD who are undoubtedly going to GS, purely because their hot-housing programmes extended well beyond the summer, well beyond Year 5..and in some cases..as early as Year 4!! If it were not for the fact that I had planned an alternative should the system let us down..I would probably have joined them..but in the weeks that followed her result, I have to admit, I started to feel that it was perhaps my own stubborn view that had let her down. I have reconciled myself with that one now..and I am happier for the fact that wherever she goes, she will be there on much broader, much fairer merit.
There are many children in the sets below my DD who are undoubtedly going to GS, purely because their hot-housing programmes extended well beyond the summer, well beyond Year 5..and in some cases..as early as Year 4!! If it were not for the fact that I had planned an alternative should the system let us down..I would probably have joined them..but in the weeks that followed her result, I have to admit, I started to feel that it was perhaps my own stubborn view that had let her down. I have reconciled myself with that one now..and I am happier for the fact that wherever she goes, she will be there on much broader, much fairer merit.
Re: Bucks VR test poor resolution
I agree with Tree on how unfair the whole process is and I cannot believe my children will be subject to a system which was abolished in most other parts of the country in the 1960s! So unless we move house, we are stuck with it. My first child will be sitting the test this year, and I wondered if I could ask Tree if he has any opinions on which are the critical question types he mentions? As I say, it is such an unfair and narrow test of ability, not to mention the fact that children develop at different rates any way. Many thanks.
Re: Bucks VR test poor resolution
Yep, it's unfair.
We've known this since the 1960's.
When I was a young civil servant, around 1980, I was invited to take the 'fast track' board. As the tests were devised by Cyril Burt, I questioned their fairness. 'Not the point lad', I was told firmly. 'It gives us the right result'.
When, 10 years ago, I sat before an Appeals panel, the reply to the same query was similar, 'It's not a test of ability, it's a test of suitability'.
We've known this since the 1960's.
When I was a young civil servant, around 1980, I was invited to take the 'fast track' board. As the tests were devised by Cyril Burt, I questioned their fairness. 'Not the point lad', I was told firmly. 'It gives us the right result'.
When, 10 years ago, I sat before an Appeals panel, the reply to the same query was similar, 'It's not a test of ability, it's a test of suitability'.
-
rubyrubyruby
- Posts: 58
- Joined: Sun Nov 28, 2010 7:35 pm
Re: Bucks VR test poor resolution
Good post tree- I have similarly been pondering about how a normal distibution can be said to work well when the pass mark is so high.
I also cannot see how bucks can justify having only a VR test when there are a large number of children fro whom VR is a very poor metric of their overall ability yet have very high NVR scores. The appeals process should pick up some of these, but bucks is effectively saying that VR is intrinsically a better measure of raw intellect- I have not seen the dta that supports that anywhere.
Seems to be a particularly high rate of failing to get through on appeal at present- I wonder why?
RRR
I also cannot see how bucks can justify having only a VR test when there are a large number of children fro whom VR is a very poor metric of their overall ability yet have very high NVR scores. The appeals process should pick up some of these, but bucks is effectively saying that VR is intrinsically a better measure of raw intellect- I have not seen the dta that supports that anywhere.
Seems to be a particularly high rate of failing to get through on appeal at present- I wonder why?
RRR
