The star was thrust into the spotlight as a young teenager when he was cast as blond-haired villain Draco Malfoy in the series, based on J.K. Rowling's best-selling books.
But he insists fame and fortune isn't any consolation for the torment he suffered at the hands of jealous peers.
Felton tells HeatWorld, "I would miss months of school and then return with bright blond hair. Needless to say, there was bullying. I wasn't beaten up daily, but there was name-calling and jealousy.
"You have to bear in mind that Harry Potter wasn't cool. I wasn't part of the Terminator franchise."
The 23 year old even suggests he regrets taking on such a big job so young - because it robbed him of his childhood.
He adds, "One thing that people keep saying to me is that the wealth and the fame must have made up for missing out on my childhood, but the idea of putting a price on your childhood is ridiculous.
"You will never get those years back and you can't put a price on them. And to me, fame is not a positive thing. The idea of being famous is a lot better than the reality."