The difficulty is that the levels do not equate precisely between years - the papers sat in year 3 for example are not at the same level as those sat in year 6.
The standardised scores should be giving a better indication of progress and a significant fall in these should warrent an explanation from the teacher.
It could be as simple as having a bad day - in which case the teacher will have other work on which the teacher assesment is based.
You could ask to see the VR & NVR papers to compare them - did your child run out of time, make silly mistakes etc?
We are all keen to see our children making progress but in your child's case, given the very high year 3 levels it may be that their progress is not as great in year 4 than in previous years (many children develop in fits & starts), or maybe your child over performed in year 3 tests so there is a bit of levelling out in year 4.
They are still performing extremely well but if you have concerns I would ask the teacher what your child's targets were for end year 4 & whether they acheived them.
The teacher sets these for each pupil at the beginning of the year, overseen by a senior member of staff. If the targets were not met then the school should flag your child up for review - however bright they are they still need to make progress to fulfill their potential!
You could also ask what particularly targets (in the sense of specific skills) they will be set for year 5 & how they will be helped to achieve them.
As a general answer I would say that a child getting level 4 in year 4 should be a good bet for GS.
I wouldn.t get too worried just yet but I would keep an eye on what is happening as it isnt unknown for high acheiving children at primary school to get overlooked because they already doing well.
Hope some of this helps