Raw scores relative to standardised scores 2010 tests

Eleven Plus (11+) in Buckinghamshire (Bucks)

Moderators: Section Moderators, Forum Moderators

11 Plus Platform - Online Practice Makes Perfect - Try Now
Sally-Anne
Posts: 9235
Joined: Wed Jan 11, 2006 8:10 pm
Location: Buckinghamshire

Raw scores relative to standardised scores 2010 tests

Post by Sally-Anne »

I am posting below the accumulated scores that I have been informed of for this year's testers.

Please bear in mind two essential points:

1. Raw scores are really not relevant at an appeal. One missed question might be worth mentioning, but more than that and you must prove GS ability in every way.

2. Raw scores do not equate to the same standardised scores each year. The standardised score depends upon the variations in the total cohort.

The only deduction that it is faintly safe to make from this information is the score that a child might need to achieve in practice tests for future 11+ exams in Bucks.

Code: Select all

                  Paper 1                    Paper 2
               RS       SS               RS           SS  
Sep            69       119              73           120
Sep            71       121              74           121
Oct            70       120              70           115
Oct            70       121              74           121
Oct            72       124              72           119  
Oct            73       126              79           141
Nov            67       117              71           117
Nov            70       121              73           121 
Dec            59       110              62           110
Dec            65       115              68           115
Dec            69       121              73           121
Dec            72       125              74           125
Dec            74       130              76           130
Dec            78       140              79           140
Jan            69       121              73           121    
Jan            73       128              76           131
Feb            69       121              72           121   
Mar            65       117              69           118
Mar            68       121              72           121
Apr            68       121              72           121
May            67       121              71           121
Jun            67       121              71           121
Jul            66       121              70           121
Aug            64       119              66           116
Aug            65       121              66           121         
Any further information will be added here, and an update will be posted.

If the discussion of these scores becomes too cumbersome I will split posts into new threads as appropriate.

Sally-Anne
WP
Posts: 1331
Joined: Thu Jan 03, 2008 9:26 am
Location: Watford, Herts

Re: Raw scores converted to standardised scores: 2010 tests

Post by WP »

I wonder about a couple of these.

In test 1, Jan 69 119 seems unlikely given Dec 69 120 and Sep 69 119.

In test 2, there are two 79s, but the older one gets the higher score.

The mappings of 68 and 69 in test 2 also seem to be off the curve.
Sally-Anne
Posts: 9235
Joined: Wed Jan 11, 2006 8:10 pm
Location: Buckinghamshire

Re: Raw scores converted to standardised scores: 2010 tests

Post by Sally-Anne »

Hi WP

I have been back and checked my sources on all of these, bar 3 that I can't recall the source for at present. I will continue to hunt them down - they are the Oct 73 & 79, Nov 67 & 71 and Jan 73 & 76. Otherwise, barring the one report that came secondhand (Harry M's daughter's best friend) I can only assume that there may have been typos made by nervous appeal parents!

I agree that there are one or two apparent discrepancies, but as this is the first time we have attempted this exercise on any scale, the objective is not really statistical accuracy at present. It is more for the information of future parents who want to know roughly what scores their child might need in practice.

I will post any corrections if I find them, and if anyone can help on that, please post here.

S-A
Tree
Posts: 536
Joined: Mon Aug 31, 2009 7:19 pm
Location: bucks

Re: Raw scores converted to standardised scores: 2010 tests

Post by Tree »

Hi all

I have been doing some work on the above figures as I did for last year. I have come up with a formula which fits the data and so you can work out a raw score(RS) from a standardised score(SS) and a SS from a RS for any score and apply an age correction. If you want to avoid all the detail and are just wanting to find out what you probably needed to pass for 2010 11 plus then skip to the conclusion at the end or the table in the next post.

Ok so if we look at the above data the most useful data we have is for December where we have a range of SS’s from 110 to 140 with the corresponding RS’s. Studying this data we can see that the original formulae:

SS = 15(RS — meanscore)/sd + 100 ……………(1)

quoted by NFER (or whoever it is now) for converting RS to SS does not fit this data at all. This formula is linear which means that 1 Standard deviation SD of raw score should be the same for all raw scores. However if we look at the table above we can see that 1 standard deviation of scores ie 125-140 equates to 5 raw score points (74-79) at the high end of the scores and equates to (62-74) or 12 raw score points for ss’s of 110 to 125. This means that the questions become proportionally harder as they get more difficult. Or quoting this (even though I say it myself) amusing post :

http://www.elevenplusexams.co.uk/forum/ ... 12&t=18108" onclick="window.open(this.href);return false;

the ruler stretches out, and the resolution is less. I expect this is a result of bucks kids getting very good at VR and clustering around the top 10% of the test which was originally based on a national population and so they have had to make the most difficult 10 or 15 question even more difficult in order to stop everyone getting near full marks.

So how does this non linearity come about, I have no particular experience of this but I assume that 11 plus tests are made by building up a paper containing question from a large database. This database contains thousands of questions that have been tested on children and has an indication of difficulty e.g. 20% of kids get this question right of 80 % get this right etc. and the paper is then built up so as to generate a distribution of results to match the proposed distribution of VR ability which is considered normally distributed. If this is done correctly then the rs results should fit formula (1) above. However the bucks results are skewed to the right because with more than the national average getting high scores so the exam makers have increased the difficulty of the top 15 or so question to stretch the exam further along the normal distribution curve but not by adding more questions but by making the questions much harder and this has therefore added non-linearity.

I have tried a number of different expressions to fit the data but decided on an exponential form as this would be the most natural to use for human measurements and also seemed to fit best the formulation which worked best was this :

SS = (exp(RS/c1))/c3)+c2 …………………………………………. (2)

And rearranging

RS = c1xln((SS-c2)c3)

Where c1 c2 and c3 are constants that are used to fit the expression to the data but are not directly related to SD or mean scores as in data fitting I used with formulae (1) last year and exp(x) is exponential e^x and ln(x) is the natural logarithm of x.

So fitting the data using numerical method and varying the constants to get the best fit for December born children above for paper 1 we have :

RS = 11.74xln((SS-102.48)x20.28) …………. (4)

SS = (exp(RS/11.74))/20.28)+102.48) ……..(5)

And for paper 2 we have

RS = 8.44xln((SS-105.35)x331.54) …………. (6)

SS = (exp(RS/8.44))/331.54)+105.35) ……..(7)

So by using the above equations you can calculate a RS from a SS and visa versa for December born children. This gives a projected pass mark for December born children of 70 for paper 1 and 72 for paper 2.

I didn’t have enough data for the other months to calculate new expressions for each month but I think it is reasonable to assume that the curve for each paper is going to be similar for each month as it is a function of the paper make up and should be independent of the age of the child.

So we have to produce a correction for age of child based on comparing the data expected for a certain score for a December born child with the actual data received for a particular month. For example if we look at a march born child with a RS or 65 giving a SS of 117 for paper 1 using equation (4) for a December born child a SS of 117 would give a RS of 67 therefore suggesting that a march born child with 2 marks less than a December born child would get the same SS of 117.

However as WP pointed out there are some discrepancies if we assume the data is correct it seem that although the trend is for the SS to get lower for the same RS as you get older this may not always be the case the main discrepancy is that a September born child for paper 1 and a jan born child both get the same SS of 119 for RS 69 even though they are 9 month different in age.

If you average all the differences and plot a graph of the difference in RS for the same SS across the months it is variable but follows a trend leading to an average difference of about 5 RS marks over the year which is similar to what we found last year. So until we can get more data I think the simplest think is to add 0.4 to the RS for the same SS for every month a child is older than a december born child and subtract 0.4 per month for every month a child is younger than a December child.

IN CONCLUSION then using the curve fitting methods above we can estimate a childs RS from their SS and visa versa by using equations (4) and (5) for paper 1 and (6) and (7) for paper 2. We can then estimate similar scores for children born in other months by adding 0.4 per month to their RS for every month older than December they are and 0.4 less for every month younger than December they are.

Eg if we wanted to know the RS's for an April born child with SS's of 118 in paper 1 and 116 in paper 2 we would first of all calculate the RS for these SS for a December born child using equation (4) for paper 1 and (6) for paper 2 so for paper 1 we have :

RS = 11.74xln((118-102.48)x20.28) = 67.52

And paper 2 we have

RS= 8.44xln((116-105.35)x331.54) =68.9

We then correct for age by subtracting 0.4 for every month the child is younger than December or 4x0.4 = 1.6 giving us a RS of 67.52 -1.6 = 65.92 or 66 for paper 1 and 68.9-1.6 = 67.3 or 67 for paper 2.

Using this method we can estimate the scores needed to pass for each paper by month for 2010 as

month paper 1 paper 2

sept 70.8 73.4
oct 70.4 73.0
nov 70.0 72.6
dec 69.6 72.2
jan 69.2 71.8
feb 68.8 71.4
mar 68.4 71.0
apr 68.0 70.6
may 67.6 70.2
jun 67.2 69.8
jul 66.8 69.4
aug 66.4 69.0

Some qualifying remarks assumptions

This assumes the distribution of score is similar for all age children as it is for december age children

The equations are only fitted across the range 110 -140 i don't think it works well under 110

The use of 0.4 rs correction per month as can bee seen from the original data is pretty crude and there are obviously alot of variations in this and it should only be used as a rough guide.

I hope this all makes sense please let me know if you want any explanation.

All the best and good luck to everyone in the upcoming tests and appeals

Tree
Last edited by Tree on Mon Jan 24, 2011 8:26 am, edited 1 time in total.
Tree
Posts: 536
Joined: Mon Aug 31, 2009 7:19 pm
Location: bucks

Re: Raw scores converted to standardised scores: 2010 tests

Post by Tree »

Have managed to enter data from above analysis as a table, the only way to get it lined up was by using 2 digits for months the same cautions and assumptions apply

Code: Select all

Paper 1				Birth month								
SS 	09	10	11	12	01	02	03	04	05	06	07	08
110	60	60	59	59	59	58	58	57	57	57	56	56
111	62	61	61	60	60	60	59	59	58	58	58	57
112	63	63	62	62	61	61	61	60	60	59	59	59
113	64	64	63	63	63	62	62	61	61	61	60	60
114	65	65	64	64	64	63	63	62	62	62	61	61
115	66	66	65	65	65	64	64	63	63	63	62	62
116	67	67	66	66	66	65	65	64	64	64	63	63
117	68	68	67	67	66	66	66	65	65	64	64	64
118	69	68	68	68	67	67	66	66	66	65	65	64
119	69	69	69	68	68	67	67	67	66	66	65	65
120	70	70	69	69	69	68	68	67	67	67	66	66
121	71	70	70	70	69	69	68	68	68	67	67	66
												
Paper 2				Birth month								
SS 	09	10	11	12	01	02	03	04	05	06	07	08
110	63	63	62	62	62	61	61	60	60	60	59	59
111	65	64	64	64	63	63	62	62	62	61	61	60
112	66	66	65	65	65	64	64	63	63	63	62	62
113	67	67	67	66	66	65	65	65	64	64	63	63
114	68	68	68	67	67	66	66	66	65	65	64	64
115	69	69	69	68	68	67	67	67	66	66	65	65
116	70	70	69	69	69	68	68	67	67	67	66	66
117	71	71	70	70	69	69	69	68	68	67	67	67
118	72	71	71	70	70	70	69	69	68	68	68	67
119	72	72	71	71	71	70	70	69	69	69	68	68
120	73	72	72	72	71	71	70	70	70	69	69	68
121	73	73	73	72	72	71	71	71	70	70	69	69
pippi
Posts: 320
Joined: Wed Jul 09, 2008 12:33 am
Location: Bucks

Re: Raw scores converted to standardised scores: 2010 tests

Post by pippi »

For comparison, this table comes from the Kent section via an FoI request: http://www.elevenplusexams.co.uk/school ... nt-11-plus. If I understand correctly an aggregate score of 360 or more is required to qualify, and there is only one paper of each type (so not best of two papers as in Bucks).

Code: Select all

Age at time of testing:  10 yrs old  10 yrs 6 mths  11 yrs old  10 yrs old  10 yrs 6 mths  11 yrs old
To achieve a score of:   120         120            120         140         140 	        140
Verbal Reasoning         36/80 (45%) 39/80 (49%) 	42/80 (53%) 58/80 (73%) 61/80 (76%) 	63/80 (79%)
Non-Verbal Reasoning     35/72 (49%) 37/72 (51%)    38/72 (53%) 48/72 (67%) 49/72 (68%)    50/72 (69%)
Maths                    24/50 (48%) 26/50 (52%) 	27/50 (54%) 38/50 (76%) 39/50 (78%) 	40/50 (80%)
stressedparent
Posts: 126
Joined: Mon Jul 19, 2010 1:30 pm

Re: Raw scores converted to standardised scores: 2010 tests

Post by stressedparent »

WP wrote:I wonder about a couple of these.

In test 1, Jan 69 119 seems unlikely given Dec 69 120 and Sep 69 119.

In test 2, there are two 79s, but the older one gets the higher score.

The mappings of 68 and 69 in test 2 also seem to be off the curve.
It is sometimes normal with age standardisation. The most probable cause with be the mean score for October born might have been lower than mean score for the December born. It is only a prediction though.

SS= 15(rs-ms)/sd +100
stressedparent
mrsmum
Posts: 77
Joined: Mon Nov 29, 2010 8:25 pm

Re: Raw scores converted to standardised scores: 2010 tests

Post by mrsmum »

Ahhh Tree! our DS is May born (31st of May at that!!!) and scored 120 on his second paper (having scored 118 on the first paper) ... it is 'interesting' to see an analysis that shows a RS of 70 would have given him either 120 or 121 ????? what a difference that would have made!!!! I did tell him he was very close :D
Jules7
Posts: 248
Joined: Mon Oct 18, 2010 7:55 pm

Re: Raw scores converted to standardised scores: 2010 tests

Post by Jules7 »

For comparison, this table comes from the Kent section via an FoI request: http://www.elevenplusexams.co.uk/school ... nt-11-plus" onclick="window.open(this.href);return false;. If I understand correctly an aggregate score of 360 or more is required to qualify, and there is only one paper of each type (so not best of two papers as in Bucks).

Code:
Age at time of testing: 10 yrs old 10 yrs 6 mths 11 yrs old 10 yrs old 10 yrs 6 mths 11 yrs old
To achieve a score of: 120 120 120 140 140 140
Verbal Reasoning 36/80 (45%) 39/80 (49%) 42/80 (53%) 58/80 (73%) 61/80 (76%) 63/80 (79%)
Non-Verbal Reasoning 35/72 (49%) 37/72 (51%) 38/72 (53%) 48/72 (67%) 49/72 (68%) 50/72 (69%)
Maths 24/50 (48%) 26/50 (52%) 27/50 (54%) 38/50 (76%) 39/50 (78%) 40/50 (80%)

Err does this mean, at least for VR, that Kent have harder Qs than Bucks or Bucks DCs are so tutored that pass rates are very high? (As more like 90%+ is required to achieve a pass in Bucks for the older end of the age group). Or is there some other reason?

By the way thanks so much for compiling and analysing all the Bucks data Sally-Anne and Tree - this is very useful for the next batch of parents!
Tree
Posts: 536
Joined: Mon Aug 31, 2009 7:19 pm
Location: bucks

Re: Raw scores converted to standardised scores: 2010 tests

Post by Tree »

Mrsmum that is a good point my analysis produces continuous data which is the rounded up or down whereas th 11 plus is discrete ie you can't get 76.5 only 76 ie everything is rounded down so the real tables don't allow ambiguous results.

Stressedparent i think the ambiguities between ages can occur in either of 2 ways if you imagine the data for each month as defined by 2 variables one that describes the distribution which could be strait line ie (SS= 15(rs-ms)/sd +100) but which actually seems to be a curve reasonably modelled by an exponential and this function is probably related to the structure of the paper and a second variable which has the effect of shifting the whole curve, this variable is more related to the overall performance of children in that birth month which as you say would impact on the mean score. So an ambiguity could arise between different birth months either because a particular birth month produces a different curve or over or under performs as a group affecting the whole distribution or a combination of both, however it appears that the general trend in improving results by about 0.4 rs per month of age is consistent across both papers as it was for last year

The other thing to remember is that the higher end of these curves is fixed ie they all have to end with 79/80 giving 141 this means that the difference between ages must become less as the rs tends towards 80 so the age correction i quoted as 0.4 per month does not work as well for the higher scores infact i chose this value as the best fit for around the 121 score which is what most people want to know

Jules7 i think we need to get some kent 11plus VR papers and give then to bucks children and see how they do !
Post Reply