I don't know all the circumstances at your son's school but can comment (with gusto) on what happened to my ds1 and it may comfort you.
My ds1 was always in top set for Maths (knew all times tables by Yr 2 etc), until Yr 5 when I moved him to a school that was 10 miles closer to us. It appeared (on paper) to be an excellent school and it seemed like a good decision. Immediately he was placed in bottom set Maths. I was surprised but was told that their standards were, oh, so much higher not like his old school (which was a pretty good school in terms of results I can tell you). Things went on...the teacher seemed to pick on him for everything, the way he was treated at times would take your breath away if I were to recall it all here. Ds1 was so frustrated and under challenged that I decided to ask a friend's daughter who was taking A level Maths to sit with him twice a week and go through some Maths workbooks so he felt he was making progress. He was happy doing this. Anyway at the end of Yr 5, many of his friends were set the 'single level' level 5 papers - my son was given level 4 as the teacher said he didn't understand any of the level 5 stuff. She laughed at my husband when he said he would take the 11 plus and said some kids just can't make Grammar schools.
I gave him an old multi-level SATS paper myself and he scored 96%, a very high level 5. They said I must have marked it wrong. Finally after one particularly nasty incident with this teacher the day after my dc5 was born, I got him out of there. Upon entering his new Primary, the new teacher said he was working around level 6. He sat his 11 plus a few months later, passing for his first choice Grammar school. He is now happily one of the top performers in Maths there and the teacher said he'd not met a Yr 7 with such a grasp of algebra.
I can only say from my experience:
Teachers are not infallible. Many are great, some are bad, and a few are just plain ugly.
Take responsibility for assessing your son yourself rather than rely on one teacher.
Keep your son's confidence up, whatever happens at school