Raw scores relative to standardised scores 2010 tests
Moderators: Section Moderators, Forum Moderators
-
- Posts: 9235
- Joined: Wed Jan 11, 2006 8:10 pm
- Location: Buckinghamshire
Raw scores relative to standardised scores 2010 tests
I am posting below the accumulated scores that I have been informed of for this year's testers.
Please bear in mind two essential points:
1. Raw scores are really not relevant at an appeal. One missed question might be worth mentioning, but more than that and you must prove GS ability in every way.
2. Raw scores do not equate to the same standardised scores each year. The standardised score depends upon the variations in the total cohort.
The only deduction that it is faintly safe to make from this information is the score that a child might need to achieve in practice tests for future 11+ exams in Bucks.
Any further information will be added here, and an update will be posted.
If the discussion of these scores becomes too cumbersome I will split posts into new threads as appropriate.
Sally-Anne
Please bear in mind two essential points:
1. Raw scores are really not relevant at an appeal. One missed question might be worth mentioning, but more than that and you must prove GS ability in every way.
2. Raw scores do not equate to the same standardised scores each year. The standardised score depends upon the variations in the total cohort.
The only deduction that it is faintly safe to make from this information is the score that a child might need to achieve in practice tests for future 11+ exams in Bucks.
Code: Select all
Paper 1 Paper 2
RS SS RS SS
Sep 69 119 73 120
Sep 71 121 74 121
Oct 70 120 70 115
Oct 70 121 74 121
Oct 72 124 72 119
Oct 73 126 79 141
Nov 67 117 71 117
Nov 70 121 73 121
Dec 59 110 62 110
Dec 65 115 68 115
Dec 69 121 73 121
Dec 72 125 74 125
Dec 74 130 76 130
Dec 78 140 79 140
Jan 69 121 73 121
Jan 73 128 76 131
Feb 69 121 72 121
Mar 65 117 69 118
Mar 68 121 72 121
Apr 68 121 72 121
May 67 121 71 121
Jun 67 121 71 121
Jul 66 121 70 121
Aug 64 119 66 116
Aug 65 121 66 121
If the discussion of these scores becomes too cumbersome I will split posts into new threads as appropriate.
Sally-Anne
Re: Raw scores converted to standardised scores: 2010 tests
I wonder about a couple of these.
In test 1, Jan 69 119 seems unlikely given Dec 69 120 and Sep 69 119.
In test 2, there are two 79s, but the older one gets the higher score.
The mappings of 68 and 69 in test 2 also seem to be off the curve.
In test 1, Jan 69 119 seems unlikely given Dec 69 120 and Sep 69 119.
In test 2, there are two 79s, but the older one gets the higher score.
The mappings of 68 and 69 in test 2 also seem to be off the curve.
-
- Posts: 9235
- Joined: Wed Jan 11, 2006 8:10 pm
- Location: Buckinghamshire
Re: Raw scores converted to standardised scores: 2010 tests
Hi WP
I have been back and checked my sources on all of these, bar 3 that I can't recall the source for at present. I will continue to hunt them down - they are the Oct 73 & 79, Nov 67 & 71 and Jan 73 & 76. Otherwise, barring the one report that came secondhand (Harry M's daughter's best friend) I can only assume that there may have been typos made by nervous appeal parents!
I agree that there are one or two apparent discrepancies, but as this is the first time we have attempted this exercise on any scale, the objective is not really statistical accuracy at present. It is more for the information of future parents who want to know roughly what scores their child might need in practice.
I will post any corrections if I find them, and if anyone can help on that, please post here.
S-A
I have been back and checked my sources on all of these, bar 3 that I can't recall the source for at present. I will continue to hunt them down - they are the Oct 73 & 79, Nov 67 & 71 and Jan 73 & 76. Otherwise, barring the one report that came secondhand (Harry M's daughter's best friend) I can only assume that there may have been typos made by nervous appeal parents!
I agree that there are one or two apparent discrepancies, but as this is the first time we have attempted this exercise on any scale, the objective is not really statistical accuracy at present. It is more for the information of future parents who want to know roughly what scores their child might need in practice.
I will post any corrections if I find them, and if anyone can help on that, please post here.
S-A
Re: Raw scores converted to standardised scores: 2010 tests
Hi all
I have been doing some work on the above figures as I did for last year. I have come up with a formula which fits the data and so you can work out a raw score(RS) from a standardised score(SS) and a SS from a RS for any score and apply an age correction. If you want to avoid all the detail and are just wanting to find out what you probably needed to pass for 2010 11 plus then skip to the conclusion at the end or the table in the next post.
Ok so if we look at the above data the most useful data we have is for December where we have a range of SS’s from 110 to 140 with the corresponding RS’s. Studying this data we can see that the original formulae:
SS = 15(RS — meanscore)/sd + 100 ……………(1)
quoted by NFER (or whoever it is now) for converting RS to SS does not fit this data at all. This formula is linear which means that 1 Standard deviation SD of raw score should be the same for all raw scores. However if we look at the table above we can see that 1 standard deviation of scores ie 125-140 equates to 5 raw score points (74-79) at the high end of the scores and equates to (62-74) or 12 raw score points for ss’s of 110 to 125. This means that the questions become proportionally harder as they get more difficult. Or quoting this (even though I say it myself) amusing post :
http://www.elevenplusexams.co.uk/forum/ ... 12&t=18108" onclick="window.open(this.href);return false;
the ruler stretches out, and the resolution is less. I expect this is a result of bucks kids getting very good at VR and clustering around the top 10% of the test which was originally based on a national population and so they have had to make the most difficult 10 or 15 question even more difficult in order to stop everyone getting near full marks.
So how does this non linearity come about, I have no particular experience of this but I assume that 11 plus tests are made by building up a paper containing question from a large database. This database contains thousands of questions that have been tested on children and has an indication of difficulty e.g. 20% of kids get this question right of 80 % get this right etc. and the paper is then built up so as to generate a distribution of results to match the proposed distribution of VR ability which is considered normally distributed. If this is done correctly then the rs results should fit formula (1) above. However the bucks results are skewed to the right because with more than the national average getting high scores so the exam makers have increased the difficulty of the top 15 or so question to stretch the exam further along the normal distribution curve but not by adding more questions but by making the questions much harder and this has therefore added non-linearity.
I have tried a number of different expressions to fit the data but decided on an exponential form as this would be the most natural to use for human measurements and also seemed to fit best the formulation which worked best was this :
SS = (exp(RS/c1))/c3)+c2 …………………………………………. (2)
And rearranging
RS = c1xln((SS-c2)c3)
Where c1 c2 and c3 are constants that are used to fit the expression to the data but are not directly related to SD or mean scores as in data fitting I used with formulae (1) last year and exp(x) is exponential e^x and ln(x) is the natural logarithm of x.
So fitting the data using numerical method and varying the constants to get the best fit for December born children above for paper 1 we have :
RS = 11.74xln((SS-102.48)x20.28) …………. (4)
SS = (exp(RS/11.74))/20.28)+102.48) ……..(5)
And for paper 2 we have
RS = 8.44xln((SS-105.35)x331.54) …………. (6)
SS = (exp(RS/8.44))/331.54)+105.35) ……..(7)
So by using the above equations you can calculate a RS from a SS and visa versa for December born children. This gives a projected pass mark for December born children of 70 for paper 1 and 72 for paper 2.
I didn’t have enough data for the other months to calculate new expressions for each month but I think it is reasonable to assume that the curve for each paper is going to be similar for each month as it is a function of the paper make up and should be independent of the age of the child.
So we have to produce a correction for age of child based on comparing the data expected for a certain score for a December born child with the actual data received for a particular month. For example if we look at a march born child with a RS or 65 giving a SS of 117 for paper 1 using equation (4) for a December born child a SS of 117 would give a RS of 67 therefore suggesting that a march born child with 2 marks less than a December born child would get the same SS of 117.
However as WP pointed out there are some discrepancies if we assume the data is correct it seem that although the trend is for the SS to get lower for the same RS as you get older this may not always be the case the main discrepancy is that a September born child for paper 1 and a jan born child both get the same SS of 119 for RS 69 even though they are 9 month different in age.
If you average all the differences and plot a graph of the difference in RS for the same SS across the months it is variable but follows a trend leading to an average difference of about 5 RS marks over the year which is similar to what we found last year. So until we can get more data I think the simplest think is to add 0.4 to the RS for the same SS for every month a child is older than a december born child and subtract 0.4 per month for every month a child is younger than a December child.
IN CONCLUSION then using the curve fitting methods above we can estimate a childs RS from their SS and visa versa by using equations (4) and (5) for paper 1 and (6) and (7) for paper 2. We can then estimate similar scores for children born in other months by adding 0.4 per month to their RS for every month older than December they are and 0.4 less for every month younger than December they are.
Eg if we wanted to know the RS's for an April born child with SS's of 118 in paper 1 and 116 in paper 2 we would first of all calculate the RS for these SS for a December born child using equation (4) for paper 1 and (6) for paper 2 so for paper 1 we have :
RS = 11.74xln((118-102.48)x20.28) = 67.52
And paper 2 we have
RS= 8.44xln((116-105.35)x331.54) =68.9
We then correct for age by subtracting 0.4 for every month the child is younger than December or 4x0.4 = 1.6 giving us a RS of 67.52 -1.6 = 65.92 or 66 for paper 1 and 68.9-1.6 = 67.3 or 67 for paper 2.
Using this method we can estimate the scores needed to pass for each paper by month for 2010 as
month paper 1 paper 2
sept 70.8 73.4
oct 70.4 73.0
nov 70.0 72.6
dec 69.6 72.2
jan 69.2 71.8
feb 68.8 71.4
mar 68.4 71.0
apr 68.0 70.6
may 67.6 70.2
jun 67.2 69.8
jul 66.8 69.4
aug 66.4 69.0
Some qualifying remarks assumptions
This assumes the distribution of score is similar for all age children as it is for december age children
The equations are only fitted across the range 110 -140 i don't think it works well under 110
The use of 0.4 rs correction per month as can bee seen from the original data is pretty crude and there are obviously alot of variations in this and it should only be used as a rough guide.
I hope this all makes sense please let me know if you want any explanation.
All the best and good luck to everyone in the upcoming tests and appeals
Tree
I have been doing some work on the above figures as I did for last year. I have come up with a formula which fits the data and so you can work out a raw score(RS) from a standardised score(SS) and a SS from a RS for any score and apply an age correction. If you want to avoid all the detail and are just wanting to find out what you probably needed to pass for 2010 11 plus then skip to the conclusion at the end or the table in the next post.
Ok so if we look at the above data the most useful data we have is for December where we have a range of SS’s from 110 to 140 with the corresponding RS’s. Studying this data we can see that the original formulae:
SS = 15(RS — meanscore)/sd + 100 ……………(1)
quoted by NFER (or whoever it is now) for converting RS to SS does not fit this data at all. This formula is linear which means that 1 Standard deviation SD of raw score should be the same for all raw scores. However if we look at the table above we can see that 1 standard deviation of scores ie 125-140 equates to 5 raw score points (74-79) at the high end of the scores and equates to (62-74) or 12 raw score points for ss’s of 110 to 125. This means that the questions become proportionally harder as they get more difficult. Or quoting this (even though I say it myself) amusing post :
http://www.elevenplusexams.co.uk/forum/ ... 12&t=18108" onclick="window.open(this.href);return false;
the ruler stretches out, and the resolution is less. I expect this is a result of bucks kids getting very good at VR and clustering around the top 10% of the test which was originally based on a national population and so they have had to make the most difficult 10 or 15 question even more difficult in order to stop everyone getting near full marks.
So how does this non linearity come about, I have no particular experience of this but I assume that 11 plus tests are made by building up a paper containing question from a large database. This database contains thousands of questions that have been tested on children and has an indication of difficulty e.g. 20% of kids get this question right of 80 % get this right etc. and the paper is then built up so as to generate a distribution of results to match the proposed distribution of VR ability which is considered normally distributed. If this is done correctly then the rs results should fit formula (1) above. However the bucks results are skewed to the right because with more than the national average getting high scores so the exam makers have increased the difficulty of the top 15 or so question to stretch the exam further along the normal distribution curve but not by adding more questions but by making the questions much harder and this has therefore added non-linearity.
I have tried a number of different expressions to fit the data but decided on an exponential form as this would be the most natural to use for human measurements and also seemed to fit best the formulation which worked best was this :
SS = (exp(RS/c1))/c3)+c2 …………………………………………. (2)
And rearranging
RS = c1xln((SS-c2)c3)
Where c1 c2 and c3 are constants that are used to fit the expression to the data but are not directly related to SD or mean scores as in data fitting I used with formulae (1) last year and exp(x) is exponential e^x and ln(x) is the natural logarithm of x.
So fitting the data using numerical method and varying the constants to get the best fit for December born children above for paper 1 we have :
RS = 11.74xln((SS-102.48)x20.28) …………. (4)
SS = (exp(RS/11.74))/20.28)+102.48) ……..(5)
And for paper 2 we have
RS = 8.44xln((SS-105.35)x331.54) …………. (6)
SS = (exp(RS/8.44))/331.54)+105.35) ……..(7)
So by using the above equations you can calculate a RS from a SS and visa versa for December born children. This gives a projected pass mark for December born children of 70 for paper 1 and 72 for paper 2.
I didn’t have enough data for the other months to calculate new expressions for each month but I think it is reasonable to assume that the curve for each paper is going to be similar for each month as it is a function of the paper make up and should be independent of the age of the child.
So we have to produce a correction for age of child based on comparing the data expected for a certain score for a December born child with the actual data received for a particular month. For example if we look at a march born child with a RS or 65 giving a SS of 117 for paper 1 using equation (4) for a December born child a SS of 117 would give a RS of 67 therefore suggesting that a march born child with 2 marks less than a December born child would get the same SS of 117.
However as WP pointed out there are some discrepancies if we assume the data is correct it seem that although the trend is for the SS to get lower for the same RS as you get older this may not always be the case the main discrepancy is that a September born child for paper 1 and a jan born child both get the same SS of 119 for RS 69 even though they are 9 month different in age.
If you average all the differences and plot a graph of the difference in RS for the same SS across the months it is variable but follows a trend leading to an average difference of about 5 RS marks over the year which is similar to what we found last year. So until we can get more data I think the simplest think is to add 0.4 to the RS for the same SS for every month a child is older than a december born child and subtract 0.4 per month for every month a child is younger than a December child.
IN CONCLUSION then using the curve fitting methods above we can estimate a childs RS from their SS and visa versa by using equations (4) and (5) for paper 1 and (6) and (7) for paper 2. We can then estimate similar scores for children born in other months by adding 0.4 per month to their RS for every month older than December they are and 0.4 less for every month younger than December they are.
Eg if we wanted to know the RS's for an April born child with SS's of 118 in paper 1 and 116 in paper 2 we would first of all calculate the RS for these SS for a December born child using equation (4) for paper 1 and (6) for paper 2 so for paper 1 we have :
RS = 11.74xln((118-102.48)x20.28) = 67.52
And paper 2 we have
RS= 8.44xln((116-105.35)x331.54) =68.9
We then correct for age by subtracting 0.4 for every month the child is younger than December or 4x0.4 = 1.6 giving us a RS of 67.52 -1.6 = 65.92 or 66 for paper 1 and 68.9-1.6 = 67.3 or 67 for paper 2.
Using this method we can estimate the scores needed to pass for each paper by month for 2010 as
month paper 1 paper 2
sept 70.8 73.4
oct 70.4 73.0
nov 70.0 72.6
dec 69.6 72.2
jan 69.2 71.8
feb 68.8 71.4
mar 68.4 71.0
apr 68.0 70.6
may 67.6 70.2
jun 67.2 69.8
jul 66.8 69.4
aug 66.4 69.0
Some qualifying remarks assumptions
This assumes the distribution of score is similar for all age children as it is for december age children
The equations are only fitted across the range 110 -140 i don't think it works well under 110
The use of 0.4 rs correction per month as can bee seen from the original data is pretty crude and there are obviously alot of variations in this and it should only be used as a rough guide.
I hope this all makes sense please let me know if you want any explanation.
All the best and good luck to everyone in the upcoming tests and appeals
Tree
Last edited by Tree on Mon Jan 24, 2011 8:26 am, edited 1 time in total.
Re: Raw scores converted to standardised scores: 2010 tests
Have managed to enter data from above analysis as a table, the only way to get it lined up was by using 2 digits for months the same cautions and assumptions apply
Code: Select all
Paper 1 Birth month
SS 09 10 11 12 01 02 03 04 05 06 07 08
110 60 60 59 59 59 58 58 57 57 57 56 56
111 62 61 61 60 60 60 59 59 58 58 58 57
112 63 63 62 62 61 61 61 60 60 59 59 59
113 64 64 63 63 63 62 62 61 61 61 60 60
114 65 65 64 64 64 63 63 62 62 62 61 61
115 66 66 65 65 65 64 64 63 63 63 62 62
116 67 67 66 66 66 65 65 64 64 64 63 63
117 68 68 67 67 66 66 66 65 65 64 64 64
118 69 68 68 68 67 67 66 66 66 65 65 64
119 69 69 69 68 68 67 67 67 66 66 65 65
120 70 70 69 69 69 68 68 67 67 67 66 66
121 71 70 70 70 69 69 68 68 68 67 67 66
Paper 2 Birth month
SS 09 10 11 12 01 02 03 04 05 06 07 08
110 63 63 62 62 62 61 61 60 60 60 59 59
111 65 64 64 64 63 63 62 62 62 61 61 60
112 66 66 65 65 65 64 64 63 63 63 62 62
113 67 67 67 66 66 65 65 65 64 64 63 63
114 68 68 68 67 67 66 66 66 65 65 64 64
115 69 69 69 68 68 67 67 67 66 66 65 65
116 70 70 69 69 69 68 68 67 67 67 66 66
117 71 71 70 70 69 69 69 68 68 67 67 67
118 72 71 71 70 70 70 69 69 68 68 68 67
119 72 72 71 71 71 70 70 69 69 69 68 68
120 73 72 72 72 71 71 70 70 70 69 69 68
121 73 73 73 72 72 71 71 71 70 70 69 69
Re: Raw scores converted to standardised scores: 2010 tests
For comparison, this table comes from the Kent section via an FoI request: http://www.elevenplusexams.co.uk/school ... nt-11-plus. If I understand correctly an aggregate score of 360 or more is required to qualify, and there is only one paper of each type (so not best of two papers as in Bucks).
Code: Select all
Age at time of testing: 10 yrs old 10 yrs 6 mths 11 yrs old 10 yrs old 10 yrs 6 mths 11 yrs old
To achieve a score of: 120 120 120 140 140 140
Verbal Reasoning 36/80 (45%) 39/80 (49%) 42/80 (53%) 58/80 (73%) 61/80 (76%) 63/80 (79%)
Non-Verbal Reasoning 35/72 (49%) 37/72 (51%) 38/72 (53%) 48/72 (67%) 49/72 (68%) 50/72 (69%)
Maths 24/50 (48%) 26/50 (52%) 27/50 (54%) 38/50 (76%) 39/50 (78%) 40/50 (80%)
-
- Posts: 126
- Joined: Mon Jul 19, 2010 1:30 pm
Re: Raw scores converted to standardised scores: 2010 tests
It is sometimes normal with age standardisation. The most probable cause with be the mean score for October born might have been lower than mean score for the December born. It is only a prediction though.WP wrote:I wonder about a couple of these.
In test 1, Jan 69 119 seems unlikely given Dec 69 120 and Sep 69 119.
In test 2, there are two 79s, but the older one gets the higher score.
The mappings of 68 and 69 in test 2 also seem to be off the curve.
SS= 15(rs-ms)/sd +100
stressedparent
Re: Raw scores converted to standardised scores: 2010 tests
Ahhh Tree! our DS is May born (31st of May at that!!!) and scored 120 on his second paper (having scored 118 on the first paper) ... it is 'interesting' to see an analysis that shows a RS of 70 would have given him either 120 or 121 ????? what a difference that would have made!!!! I did tell him he was very close
Re: Raw scores converted to standardised scores: 2010 tests
Err does this mean, at least for VR, that Kent have harder Qs than Bucks or Bucks DCs are so tutored that pass rates are very high? (As more like 90%+ is required to achieve a pass in Bucks for the older end of the age group). Or is there some other reason?For comparison, this table comes from the Kent section via an FoI request: http://www.elevenplusexams.co.uk/school ... nt-11-plus" onclick="window.open(this.href);return false;. If I understand correctly an aggregate score of 360 or more is required to qualify, and there is only one paper of each type (so not best of two papers as in Bucks).
Code:
Age at time of testing: 10 yrs old 10 yrs 6 mths 11 yrs old 10 yrs old 10 yrs 6 mths 11 yrs old
To achieve a score of: 120 120 120 140 140 140
Verbal Reasoning 36/80 (45%) 39/80 (49%) 42/80 (53%) 58/80 (73%) 61/80 (76%) 63/80 (79%)
Non-Verbal Reasoning 35/72 (49%) 37/72 (51%) 38/72 (53%) 48/72 (67%) 49/72 (68%) 50/72 (69%)
Maths 24/50 (48%) 26/50 (52%) 27/50 (54%) 38/50 (76%) 39/50 (78%) 40/50 (80%)
By the way thanks so much for compiling and analysing all the Bucks data Sally-Anne and Tree - this is very useful for the next batch of parents!
Re: Raw scores converted to standardised scores: 2010 tests
Mrsmum that is a good point my analysis produces continuous data which is the rounded up or down whereas th 11 plus is discrete ie you can't get 76.5 only 76 ie everything is rounded down so the real tables don't allow ambiguous results.
Stressedparent i think the ambiguities between ages can occur in either of 2 ways if you imagine the data for each month as defined by 2 variables one that describes the distribution which could be strait line ie (SS= 15(rs-ms)/sd +100) but which actually seems to be a curve reasonably modelled by an exponential and this function is probably related to the structure of the paper and a second variable which has the effect of shifting the whole curve, this variable is more related to the overall performance of children in that birth month which as you say would impact on the mean score. So an ambiguity could arise between different birth months either because a particular birth month produces a different curve or over or under performs as a group affecting the whole distribution or a combination of both, however it appears that the general trend in improving results by about 0.4 rs per month of age is consistent across both papers as it was for last year
The other thing to remember is that the higher end of these curves is fixed ie they all have to end with 79/80 giving 141 this means that the difference between ages must become less as the rs tends towards 80 so the age correction i quoted as 0.4 per month does not work as well for the higher scores infact i chose this value as the best fit for around the 121 score which is what most people want to know
Jules7 i think we need to get some kent 11plus VR papers and give then to bucks children and see how they do !
Stressedparent i think the ambiguities between ages can occur in either of 2 ways if you imagine the data for each month as defined by 2 variables one that describes the distribution which could be strait line ie (SS= 15(rs-ms)/sd +100) but which actually seems to be a curve reasonably modelled by an exponential and this function is probably related to the structure of the paper and a second variable which has the effect of shifting the whole curve, this variable is more related to the overall performance of children in that birth month which as you say would impact on the mean score. So an ambiguity could arise between different birth months either because a particular birth month produces a different curve or over or under performs as a group affecting the whole distribution or a combination of both, however it appears that the general trend in improving results by about 0.4 rs per month of age is consistent across both papers as it was for last year
The other thing to remember is that the higher end of these curves is fixed ie they all have to end with 79/80 giving 141 this means that the difference between ages must become less as the rs tends towards 80 so the age correction i quoted as 0.4 per month does not work as well for the higher scores infact i chose this value as the best fit for around the 121 score which is what most people want to know
Jules7 i think we need to get some kent 11plus VR papers and give then to bucks children and see how they do !