re:raw scores and appeals
Moderators: Section Moderators, Forum Moderators
raw scores and appeals
I've been watching this thread because I have some interest in stats and standardisation (although I could not possibly have carried out the exercise that Tree has admirably done). As I have said before, I think that what happened this year ie two 'test 1s' has been very unsatisfactory - and is the fault of no one at the Council I should add. However, I would be wary of jumping to conclusions about who exactly has been placed at a 'definite disadvantage'. Tree is obviously correct in that different 'pass marks' (when raw scores looked at) on the two 'first papers' reflect different cohorts and also probably a difference in the difficulty of the paper - although the council says that papers are 'broadly' the same (not absolutely the same as it claimed in the Q&As on here after the paper replacement, but 'broadly' if you look in the handbook). It would be impossible to create a paper that had absolutely the same level of difficulty and standardisation is meant to correct for that, as well as age. Thus even if exactly the same cohort took two different papers the pass mark would probably be different for each. If exactly the same cohort had taken the two 'first papers' it would suggest that this years first paper was a little more difficult than the previous years which was used as the replacement - and in fact that is not necessarily untrue from anecdotal reports (which I know might be disputed).
The second argument for 'disadvantage' for those taking the replacement paper is that the standardisation was only done amongst the 'late takers' who are, historically (and indeed this year, going on results) a stronger bunch. But we don't know that the standardisation was done only amongst this group and indeed, given it's small size (remember, quite a few OCCs, including those at the alleged 'miscreant' school, took the first paper) I suspect that it was not. Bucks or rather NFER might well have simply compared the marks for the replacement paper against the cohort which originally took it (?the previous year), and 'passed' anyone who gained the pass mark in that year and failed those who didn't. If they used that (admittedly not perfect, but nothing is in this case) method then takers of the second paper would have been 'standardised', or at least directly compared, to a large normal (in county and OCC) cohort. To say that it's not fair that they didn't pass with a mark of X would mean that anyone who took the same paper in the previous year should have passed with a mark of X. Since roughly the same percentage of OCCs has passed this year (I think) then, unless that cohort has 'improved' markedly it would suggest that maybe this kind of standardisation was used.
In other words, without more information I would be extremely wary of rushing to declare disadvantage.
The second argument for 'disadvantage' for those taking the replacement paper is that the standardisation was only done amongst the 'late takers' who are, historically (and indeed this year, going on results) a stronger bunch. But we don't know that the standardisation was done only amongst this group and indeed, given it's small size (remember, quite a few OCCs, including those at the alleged 'miscreant' school, took the first paper) I suspect that it was not. Bucks or rather NFER might well have simply compared the marks for the replacement paper against the cohort which originally took it (?the previous year), and 'passed' anyone who gained the pass mark in that year and failed those who didn't. If they used that (admittedly not perfect, but nothing is in this case) method then takers of the second paper would have been 'standardised', or at least directly compared, to a large normal (in county and OCC) cohort. To say that it's not fair that they didn't pass with a mark of X would mean that anyone who took the same paper in the previous year should have passed with a mark of X. Since roughly the same percentage of OCCs has passed this year (I think) then, unless that cohort has 'improved' markedly it would suggest that maybe this kind of standardisation was used.
In other words, without more information I would be extremely wary of rushing to declare disadvantage.
Hi chinup you are right 'definate disadvantage' is probably too strong. But if it is the case (which i can't remember) that the replacement test this year was the same as the 1st paper last year then as the pass mark was 1 mark different between last years full cohort and this years replacement cohort then it was not just that this years cohort was read off last years standardisation tables (because then she would have passed) but were standardised in some way as a group and that in flossie's particular case this process seemed to lead to a 120 mark rather than a 121 had she taken the same test with last years cohort. So i agree that they have done the sensible thing (some combination of standardisation of the new and old cohorts presumably) but that for flossie this seems to produced an individual significance (ie fail) which may be worth mentioning?
raw scores and appeals
I agree that some of the difference (half, to put it far too bluntly!) seems to be a difference in difficulty and the rest seems to be as yet unexplained. Although (see other thread) I would still be wary of rushing in to argue that this year's replacement was only standardised amongst a very small, strong cohort - I feel that there must have been some 'marker' against the original cohort to take the test. Obviously the sophisticated chaps and chappesses at the exam board could probably explain, only in their absence yes, worth a quick mention but I wouldn't hang an entire appeal on it. The overall percentage of OCCs passing seems to be roughly the same (that of course includes takers of the original and replacement test) and it is unlikely for cohorts to show sudden marked improvements over the course of a year.
Personally - and I note the refusal of all the GCSE, AS, Alevel boards to reschedule modules and allow for late takings because of the current weather - I still feel that Bucks allowance of later sittings of the same paper should be reviewed.
Personally - and I note the refusal of all the GCSE, AS, Alevel boards to reschedule modules and allow for late takings because of the current weather - I still feel that Bucks allowance of later sittings of the same paper should be reviewed.
Thanks tree and chinup for giving me your opinions about this. I think I will mention it at appeal but I am going to mainly focus on proving dd's suitability for Grammar.
I think the first paper 1 possibly has a lower pass score due to the absence of the generally stronger OOC's and the second paper 1 had its pass mark driven higher due to the stronger OOC's scores being added in with last year's scores i.e two lots of OOC scores plus Bucks children's scores as opposed to first paper 1's mainly Bucks children taking the test.
I think the first paper 1 possibly has a lower pass score due to the absence of the generally stronger OOC's and the second paper 1 had its pass mark driven higher due to the stronger OOC's scores being added in with last year's scores i.e two lots of OOC scores plus Bucks children's scores as opposed to first paper 1's mainly Bucks children taking the test.
Hi, do you have figures on how many OOC took the 2nd paper this year and how many last? My DC is OOC and took 1st paper and I know many OOC who took 1st and only 1 who took the second 1st paper. Not sure argument stands up re marks being skewed by too many OOCs in the standardisation pot otherwise... also think if your DC is only 1 mark off and has evidence of strong academic ability you don't need to argue the OOC point
Hi Cats12,
Unfortunately I don't have any figures for how many OOC's took the second test 1. Thanks for the extra info, perhaps I'll just mention in that case that if she had taken the test last year she would have passed?
We hope the academic evidence that we provide will be enough anyway, to prove suitability. It's the not knowing that's doing my head in!!!
Unfortunately I don't have any figures for how many OOC's took the second test 1. Thanks for the extra info, perhaps I'll just mention in that case that if she had taken the test last year she would have passed?
We hope the academic evidence that we provide will be enough anyway, to prove suitability. It's the not knowing that's doing my head in!!!
Hi Flossie, I think the fact that if DC took test last year and would have passed is a very strong point. I'm no expert and no statistician so I always base my thoughts on clarity of evidence. Your DCs high scores, academic ability and above point re last year's test make a strong case - then I think as with most things in life, a positive, friendly attitude in the appeal with lots of eye contact and smiles, listening to all their comments too will go a long way. Best of luck 