Well done to you for saying 'no to bullying'. My three younger boys were bullied daily by a year 6 boy at their old primary school. The bullying came about by way of a year 6 boy who had an older brother who was a major behaviour issue at the local high school. My kids dad dared to be intolerant of the behaviour and gave the kid a detention. The bullying on my young children then started. It was 'allowed' to happen, as apparently this poor boy came from a bad background lbah blah blah.....and it was never stopped at the school. My kids came to hate school, they would feign being sick so they didn't have to go in etc. We moved, they got into a new school - and wow is life different! Turns out tht the kid who bullied them at the old school - his parents were friends of parents of another kid at the new school - and about a week after they started - the bullying started - same type of 'foreigner' comments etc. It was reported to the teacher, the teacher was going to meet with the HT. Before they met though - five parents had made reports to the school of this unacceptable behaviour that their children had come home and reported. It was nipped in the bud that very day - and has never been a problem since. (turned out that the kid had repeated what he had been told to by the other kids dad when he'd mentioned that kids from his kids school had just joined his school). The confidence that has come about is unbelievable - back to enjoying school, loving their friends, and life is good again.
If you don't really want to take your child out of the school - is there another local school that just simply doesn't tolerate the bullying? School can be such a wonderful fun place for kids - and the interaction is so important as well.
Best of luck in whatever you decide - your child's happiness and safety is the main thing
and he's lucky to have a parent who cares enough to actually take some action over it.