I have been doing some work on the above figures as I did for last year. I have come up with a formula which fits the data and so you can work out a raw score(RS) from a standardised score(SS) and a SS from a RS for any score and apply an age correction. If you want to avoid all the detail and are just wanting to find out what you probably needed to pass for 2010 11 plus then skip to the conclusion at the end or the table in the next post.
Ok so if we look at the above data the most useful data we have is for December where we have a range of SS’s from 110 to 140 with the corresponding RS’s. Studying this data we can see that the original formulae:
SS = 15(RS — meanscore)/sd + 100 ……………(1)
quoted by NFER (or whoever it is now) for converting RS to SS does not fit this data at all. This formula is linear which means that 1 Standard deviation SD of raw score should be the same for all raw scores. However if we look at the table above we can see that 1 standard deviation of scores ie 125-140 equates to 5 raw score points (74-79) at the high end of the scores and equates to (62-74) or 12 raw score points for ss’s of 110 to 125. This means that the questions become proportionally harder as they get more difficult. Or quoting this (even though I say it myself) amusing post :viewtopic.php?f=12&t=18108
the ruler stretches out, and the resolution is less. I expect this is a result of bucks kids getting very good at VR and clustering around the top 10% of the test which was originally based on a national population and so they have had to make the most difficult 10 or 15 question even more difficult in order to stop everyone getting near full marks.
So how does this non linearity come about, I have no particular experience of this but I assume that 11 plus tests are made by building up a paper containing question from a large database. This database contains thousands of questions that have been tested on children and has an indication of difficulty e.g. 20% of kids get this question right of 80 % get this right etc. and the paper is then built up so as to generate a distribution of results to match the proposed distribution of VR ability which is considered normally distributed. If this is done correctly then the rs results should fit formula (1) above. However the bucks results are skewed to the right because with more than the national average getting high scores so the exam makers have increased the difficulty of the top 15 or so question to stretch the exam further along the normal distribution curve but not by adding more questions but by making the questions much harder and this has therefore added non-linearity.
I have tried a number of different expressions to fit the data but decided on an exponential form as this would be the most natural to use for human measurements and also seemed to fit best the formulation which worked best was this :
SS = (exp(RS/c1))/c3)+c2 …………………………………………. (2)
RS = c1xln((SS-c2)c3)
Where c1 c2 and c3 are constants that are used to fit the expression to the data but are not directly related to SD or mean scores as in data fitting I used with formulae (1) last year and exp(x) is exponential e^x and ln(x) is the natural logarithm of x.
So fitting the data using numerical method and varying the constants to get the best fit for December born children above for paper 1 we have :
RS = 11.74xln((SS-102.48)x20.28) …………. (4)
SS = (exp(RS/11.74))/20.28)+102.48) ……..(5)
And for paper 2 we have
RS = 8.44xln((SS-105.35)x331.54) …………. (6)
SS = (exp(RS/8.44))/331.54)+105.35) ……..(7)
So by using the above equations you can calculate a RS from a SS and visa versa for December born children. This gives a projected pass mark for December born children of 70 for paper 1 and 72 for paper 2.
I didn’t have enough data for the other months to calculate new expressions for each month but I think it is reasonable to assume that the curve for each paper is going to be similar for each month as it is a function of the paper make up and should be independent of the age of the child.
So we have to produce a correction for age of child based on comparing the data expected for a certain score for a December born child with the actual data received for a particular month. For example if we look at a march born child with a RS or 65 giving a SS of 117 for paper 1 using equation (4) for a December born child a SS of 117 would give a RS of 67 therefore suggesting that a march born child with 2 marks less than a December born child would get the same SS of 117.
However as WP pointed out there are some discrepancies if we assume the data is correct it seem that although the trend is for the SS to get lower for the same RS as you get older this may not always be the case the main discrepancy is that a September born child for paper 1 and a jan born child both get the same SS of 119 for RS 69 even though they are 9 month different in age.
If you average all the differences and plot a graph of the difference in RS for the same SS across the months it is variable but follows a trend leading to an average difference of about 5 RS marks over the year which is similar to what we found last year. So until we can get more data I think the simplest think is to add 0.4 to the RS for the same SS for every month a child is older than a december born child and subtract 0.4 per month for every month a child is younger than a December child.
IN CONCLUSION then using the curve fitting methods above we can estimate a childs RS from their SS and visa versa by using equations (4) and (5) for paper 1 and (6) and (7) for paper 2. We can then estimate similar scores for children born in other months by adding 0.4 per month to their RS for every month older than December they are and 0.4 less for every month younger than December they are.
Eg if we wanted to know the RS's for an April born child with SS's of 118 in paper 1 and 116 in paper 2 we would first of all calculate the RS for these SS for a December born child using equation (4) for paper 1 and (6) for paper 2 so for paper 1 we have :
RS = 11.74xln((118-102.48)x20.28) = 67.52
And paper 2 we have
RS= 8.44xln((116-105.35)x331.54) =68.9
We then correct for age by subtracting 0.4 for every month the child is younger than December or 4x0.4 = 1.6 giving us a RS of 67.52 -1.6 = 65.92 or 66 for paper 1 and 68.9-1.6 = 67.3 or 67 for paper 2.
Using this method we can estimate the scores needed to pass for each paper by month for 2010 as
month paper 1 paper 2
sept 70.8 73.4
oct 70.4 73.0
nov 70.0 72.6
dec 69.6 72.2
jan 69.2 71.8
feb 68.8 71.4
mar 68.4 71.0
apr 68.0 70.6
may 67.6 70.2
jun 67.2 69.8
jul 66.8 69.4
aug 66.4 69.0
Some qualifying remarks assumptions
This assumes the distribution of score is similar for all age children as it is for december age children
The equations are only fitted across the range 110 -140 i don't think it works well under 110
The use of 0.4 rs correction per month as can bee seen from the original data is pretty crude and there are obviously alot of variations in this and it should only be used as a rough guide.
I hope this all makes sense please let me know if you want any explanation.
All the best and good luck to everyone in the upcoming tests and appeals