Stats debate

[hammer]

Does anyone know how the standardisation is calculated or what it actually means mathematically? I understand the theory, but what are the actual statistical numbers, like the average and standard deviation, that kind of thing. Is this published anywhere or is it all top secret?

[Setanta]

The mean score in a paper (English say) is calculated once all the papers are marked. That is worth 100 standardised points.

The standard deviation for those scores is calculated. [As we don't have all the marks this is the first 'secret' bit]. The test result that would have acheived that score is given 115 (good) or 85(bad). So 15 standardised points denote one standard deviation.

The same is done for Maths and VR. The scores are then added up.

So a child who scored the mean score on English and Maths but was in the top 15.9% for VR would score 100 + 100 + 115 = 315.

Or in the top 15.9% for English & VR and the top 30.8% for Maths would score 115 + 107.5 + 115 = 337.5.

We can't work out the individual Bell curves for different subjects because we don't have the full data or a representative sample.

We can though, when we see the standardised scores infer the correlation with the Bell curve and work out relative ranking and - more usefully - the number of children between marks 330 and 331 (say) that will be in the results. You can also infer from that an idea of the number of applicants that are between your DC and a pass mark.

What you can't do is predict the vagaries of where people will apply - especially this year as boundaries have changed and we now know results before applying.

[hammer]

Thanks both, I was struggling to see where scores of 300-odd came into it, didn’t think of adding 100 together three times!

So a good estimate of where, say, a score of 330 sits in the distribution is to subtract 300 then divide by 3 to find your average adjustment for each paper (so (330-300)/3 = 10). Divide this by 15 (10/15 = 0.666) then look this up in the table (~74%). So a score of 330-ish puts you in the top quartile, give or take. Is that about right?

[Setanta]

Thanks both, I was struggling to see where scores of 300-odd came into it, didn’t think of adding 100 together three times!

So a good estimate of where, say, a score of 330 sits in the distribution is to subtract 300 then divide by 3 to find your average adjustment for each paper (so (330-300)/3 = 10). Divide this by 15 (10/15 = 0.666) then look this up in the table (~74%). So a score of 330-ish puts you in the top quartile, give or take. Is that about right?

Exactly right on what I told you. Okanagan is right though that that isn't 100% exactly what CCSE do, but it's as near as you can get without being inside and it reflects ranking : 330 is indeed top 26% approximately.

[aang]

I have not seen the actual statistical numbers or process that CSSE uses. Using a mean of 100 and a standard deviation of 15 is a common statistical approach for standardisation, but my analysis is that this is not quite how CSSE does it because:

1. The mean scores of all candidates sitting the 11+ would be 300. I am aware that the scores on this forum are definitely unrepresentative but there would have to be an awful lot of below 300 scores out there and some scores as low as 210 somewhere. (My best approximation, based on a small sample size, is that the actual mean of the standardised scores over the past four years is closer to 310)

2. CSSE have stated that the verbal reasoning paper counts for 50% of the overall score, with maths and english 25% each. This should mean you cannot simply combine the three papers individual standardised score (unless you "double" the VR paper in which case the top score would become 160, but then the max overall score would be 420 which doesn't work)

3. The "maximum" score following standard statistical process should be capped at 130 for any one subject. In this case scores above 390 would be impossible, but there are several scores above 390 this year and a few in previous years. For the record, I have not seen any score above 400 in the past four years.

Bottom line, I don't know how CSSE actually calculates the standardised score, despite spending quite a long time trying!

[Chai]

Quoting aang: "....unless you "double" the VR paper in which case the top score would become 160, but then the max overall score would be 420 which doesn't work."

Yes, it very well could be. There are scores above 400 for this year.

[Okanagan]

3. The "maximum" score following standard statistical process should be capped at 130 for any one subject. In this case scores above 390 would be impossible, but there are several scores above 390 this year and a few in previous years. For the record, I have not seen any score above 400 in the past four years.

It's more often capped at 140 or 141, which is consistent with what your seeing.

Not all areas apply a cap though - here in Warwickshire the theoretical maximum for the maths element of ours was 181 this year! I've never seen it that high (tough paper, low marks, not much variation in raw scores)!

[Chowder]

That's such a shame, my DS is a v v late Aug birthday and I am sure it made a huge difference compared to his friends a good 6-8 months older than him even though he got a good pass. Not so much from the ability angle but from the approach, maturity and concentration with applying the learning, the difference in 0-12 months at this age is enourmous.

[Setanta]

Does anyone know how the standardisation is calculated or what it actually means mathematically? I understand the theory, but what are the actual statistical numbers, like the average and standard deviation, that kind of thing. Is this published anywhere or is it all top secret?

Further to the other answers posted here. I have worked out the approximate MEan and Standard Deviation for this years three subjects.

(This done by 'trying' several hundred thousand combinations of mean test score and SD against 9 of the scores posted on this forum until the scores fitted: the calculated score matching the given score).

English approximate Mean= 21/50. Approx SD = 10
Maths approximate Mean = 25/40 Approx SD = 7
VR approximate Mean = 41 Approx SD = 11.5



F

[aang]

It's still a guessing game, albeit a statistical guessing game. I have contacted CSSE to understand more about how they standardise and I will post if I hear anything useful. In the meantime, for what little it is worth, here is my hypothesis:

1. CSSE award a score of 75 as being the mean for each of the three subjects.

2. Scores are standardised within the subject so that the scores range from 60 to 105

3. The three individual scores are then added up with verbal reasoning being added twice in order to weight verbal reasoning 50% with Maths and English being 25%

4. Using this assumption, the best fit set of values against the 70+ reported scores for 2013 gives me the following figures:

English: mean 19/50 (38%) with one standard deviation being 18%
Maths: mean 20/40 (50%) with one standard deviation being 19%
Verbal reasoning: mean 48/80 (60%) with one standard deviation being 17.7%

Using these figures gives a really high correlation score such that 99.6% of the reported scores can be explained. If I had a better set of lower standardised scores, then I would have an even higher confidence level as to the precise figures.

5. Using this set of figures and assumptions above, someone scoring 43/50, 40/40, 80/80 in the three subjects would get an overall standardised score of 410.45

If this hypothesis was correct, then this also provides an explanation as to why 300 would not be the mean overall standardised score (which is something I still suspect).

End of hypothesis.

If I hear anything from CSSE, then I will post here..