11+ test error
Moderators: Section Moderators, Forum Moderators
-
Sally-Anne
- Posts: 9235
- Joined: Wed Jan 11, 2006 8:10 pm
- Location: Buckinghamshire
)Just read chinups excellent posts again and i think i missed part of the point if it's true that the 1700 or so kids who take the paper later (and who are now taking a new paper) tend to have a higher raw scores and they are standardised together with no reference to earlier testers then they will obviously be at a disadvantage. If their scores are standardised with the results of the previous year who took the test then ok but their scores should not be added to this pool because this would shift the proportion of early-testers and late-testers.
The real problem is the standardisation of the of kids who have taken the first test because their pool now contains no late testers (who would normally raise the raw score pass mark) and so would therefore have an advantage i'm not sure how you can deal with this anomaly.
I think we should all take out shares in educational lawyer firms and statistical expert witness firms as they will be quids in come appeal time.
going to bed now have headache
The real problem is the standardisation of the of kids who have taken the first test because their pool now contains no late testers (who would normally raise the raw score pass mark) and so would therefore have an advantage i'm not sure how you can deal with this anomaly.
I think we should all take out shares in educational lawyer firms and statistical expert witness firms as they will be quids in come appeal time.
going to bed now have headache
-
Sally-Anne
- Posts: 9235
- Joined: Wed Jan 11, 2006 8:10 pm
- Location: Buckinghamshire
I have a feeling that the lawyers and expert witnesses shouldn't be booking their Caribbean holidays quite yet.
GL Assessment is a very professional operation, and I think that if they have confidence in their standardisation process, the rest of us are likely to end up agreeing with them in the long run. (Always assuming that the information is published, but I honestly can't see it not being put in the public domain in this situation.)
Developing early signs of a winter-long headache on all this!
S-A
GL Assessment is a very professional operation, and I think that if they have confidence in their standardisation process, the rest of us are likely to end up agreeing with them in the long run. (Always assuming that the information is published, but I honestly can't see it not being put in the public domain in this situation.)
Developing early signs of a winter-long headache on all this!
S-A
Ok reading the statement from the council again closely i think it is possible to see how the stndardisation is going to happen:
Because it is a test used in the grammar school selection procedure in a previous year and has already been robustly standardised on those 7000 children’s results.
I think this means that the existing standardisation curves from the previous year when this test was taken will be used and the scores from children taking the test this year will be "read off" the existing standardisation rather than included in it. (this seems reasonable unless you can demonstrate a reason why the two cohorts (late testers now and previous years entrants) will be different )
The original test was sat by all children tested in Buckinghamshire primary schools and most partner schools. It will be standardised in the normal way.
The original test is going to be standardised using the results of all the children taking it. Now here there may be a problem as if it is correct that the late testers on the whole get higher scores that the early testers and their scores are not included in the standardisation then the pass mark will be lower and so would advantage the early testers over previous years early testers. However i wonder if the late testers are ever included in the standardisation due to the timing of the tests and when the results come out etc (does anyone now if this is the case??).
If I am reading the situation right then I think their approach is as fair as it can be, particularly if late testers are not usually included in the standardisation, if they are then it may advantage original testers this year.
Because it is a test used in the grammar school selection procedure in a previous year and has already been robustly standardised on those 7000 children’s results.
I think this means that the existing standardisation curves from the previous year when this test was taken will be used and the scores from children taking the test this year will be "read off" the existing standardisation rather than included in it. (this seems reasonable unless you can demonstrate a reason why the two cohorts (late testers now and previous years entrants) will be different )
The original test was sat by all children tested in Buckinghamshire primary schools and most partner schools. It will be standardised in the normal way.
The original test is going to be standardised using the results of all the children taking it. Now here there may be a problem as if it is correct that the late testers on the whole get higher scores that the early testers and their scores are not included in the standardisation then the pass mark will be lower and so would advantage the early testers over previous years early testers. However i wonder if the late testers are ever included in the standardisation due to the timing of the tests and when the results come out etc (does anyone now if this is the case??).
If I am reading the situation right then I think their approach is as fair as it can be, particularly if late testers are not usually included in the standardisation, if they are then it may advantage original testers this year.
11 plus error
I think that further information in the fullness of time would go a long way towards reassuring everyone. And I should reiterate that, whilst I have studied statistics, I certainly don't always come in the top percentile (not nearly!), and welcome all criticism of any comments I have made.
On another point. I accept that in most years that there has not been a problem with a two stage exam process. I still think it carries risks, however, and the news that a small percentage of parents will exploit the slightest error doesn't exactly reassure me. But does anybody know whether the 11 plus tests held in Kent, or the outer boroughs of London (Kingston etc) allow for some children taking the exam on one date and a significant percentage taking the same papers later? If the system is the same everywhere then so be it, but I still think that, for an exam affecting 1000s, there are inherent risks.
On another point. I accept that in most years that there has not been a problem with a two stage exam process. I still think it carries risks, however, and the news that a small percentage of parents will exploit the slightest error doesn't exactly reassure me. But does anybody know whether the 11 plus tests held in Kent, or the outer boroughs of London (Kingston etc) allow for some children taking the exam on one date and a significant percentage taking the same papers later? If the system is the same everywhere then so be it, but I still think that, for an exam affecting 1000s, there are inherent risks.
I think NFER still do the standardisation - and will also have set the replacement test unless it was very recent.Sally-Anne wrote:GL Assessment is a very professional operation, and I think that if they have confidence in their standardisation process, the rest of us are likely to end up agreeing with them in the long run.
-
Sally-Anne
- Posts: 9235
- Joined: Wed Jan 11, 2006 8:10 pm
- Location: Buckinghamshire
Courtesy of one of our members we now have some factual information about the impact of the replacement test.
It is clear that a child taking the replacement test would need to achieve a higher raw score - perhaps two marks higher - than a child taking the original 1st Bucks paper.
(Lights blue touchpaper and retreats to a safe place!)
It is clear that a child taking the replacement test would need to achieve a higher raw score - perhaps two marks higher - than a child taking the original 1st Bucks paper.
(Lights blue touchpaper and retreats to a safe place!)
FREEDOM OF INFORMATION ACT 2000 - INFORMATION REQUEST
Thank you for contacting Buckinghamshire County Council.
Your request for information has now been considered and the information you have requested is as follows:
We have assumed for the purposes of this comparison that the tests were all taken on 27 October either last year or this year. This is because the 11+ standardisation does not necessarily equate to month of birth as the standardisation 'slot' a child will be in depends on their personal age in years and months on the date they sat each test.
What raw score did a child born in January 30 need to achieve to pass with 121 on the original test 1 paper. = 68
What raw score did a child born in January 30 need to achieve to pass with 121 on the replacement test 1 paper. = 70
What raw score did a child born in January 30 need to achieve to pass with 121 on the test 1 paper last year. = 69
We cannot comment on the example score you give from the website, as we do not know the date of the test and the exact date of birth to enable us to ascertain which 'slot' would apply.
With regards to the extra marks given to younger children; the standardisation looks at the amount by which each age band's raw score marks vary from the norm and corrects it accordingly so it varies according to each test, and by only as much as that group's performance has varied. We do know that younger children tend not to perform as strongly as older children so it tends to be the younger groups that are adjusted to the norm in this way.
11 plus error
I have, for what it's worth, made a few comments under the 'Raw Scores' thread in Appeals, suggesting that this outcome is not quite as incendiary as a first reading suggests.
As ever, all comments and corrections welcome.
I think you would need to know what the pass mark was (in terms of raw score) for the cohort which originally took the 'replacement test' (2008 someone said?). If that is pretty much the same as the pass mark for the replacement test this year then it might just be that the original test this year was actually slightly more difficult and hence after standardisation had a lower pass mark in terms of raw score. But as usual I stand to be corrected.
As ever, all comments and corrections welcome.
I think you would need to know what the pass mark was (in terms of raw score) for the cohort which originally took the 'replacement test' (2008 someone said?). If that is pretty much the same as the pass mark for the replacement test this year then it might just be that the original test this year was actually slightly more difficult and hence after standardisation had a lower pass mark in terms of raw score. But as usual I stand to be corrected.
11 plus error
Apologies - didn't read carefully enough. Since last years pass mark given then some of the explanation is that this year's original paper was more difficult - and the rest - well yes maybe a stronger cohort taking the replacement. Only I still can't believe that they only standardised amongst that (very small, not even all OCC) cohort for the replacement paper. Maybe I'm naive but I honestly think they must have had a more spohisticated method than that - for starters I think the pass mark on the replacement paper would have been even higher for such a small, strong cohort. I still think we would need further adjustment on (a) how they compared the takers of this year's test 1 mark 2 with the original takers and (b) what are their assumptions (and adjustments) on differences between the two cohorts taking the same test.
