Just some ideas here:
Children with high achieving older siblings often feel quite inadequate, even if they are top of their own class and age group. At home they constantly compare themselves to the older children and I think it is hard to grasp the fact that you are bright too when you are obviously the 'baby' of the family and know the least!
To boost my 2nd son's confidence we got him a little sister!
Ahem...(sorry!) Joking aside, it can be good for your youngest child to spend time with younger children, perhaps younger cousins, children of friends, and allow him to be the 'big one' for a change.
Another tip is to get him reading a book that has a series. Secret Seven, Famous Five, the Lemony Snicket books that sort of thing, because I think building up familiarity with characters book after book helps children really get to grips with a story line, and really get to understand the characters - thus feeling more confident to talk about them.
I'd also be asking your child subtle questions at home, basically comprehension questions on the book he is reading, but disguised as nothing more than general interest in what he is reading.
As you have other children in the house, what about reading a play together??? It could be a lot of fun, make it over dramatic and a good laugh - give your child plenty of encouragement and support. Ask your older children to offer a pat on the back every so often too.
On the whole, I wouldn't be too alarmed. People come in all different shapes and sizes and thank goodness we are not all loud! His academic work sounds fine and he will most likely develop greater confidence with time. Some kids just take a little longer to find their voice.