Simon Cowell has promised fans he will return to the panel of his hit U.K. show The X Factor. The music mogul left his pet project behind in 2011 to launch an American version, leaving Gary Barlow in charge as head judge. The show has struggled to maintain its huge audience since his departure, and now he has vowed to return after he signed a new three-year deal for the series with bosses at Britain's ITV network. He tells Britain's Metro newspaper, "I will (return) at some point. I still miss that, I mean that show is my baby and that last year we did which was the One Direction year was the most fun I've had - it was fantastic. When I look back at the clips now and remind myself about it, it was a really cool year that." Cowell has also hinted Cheryl Cole, who he took to the U.S. X Factor with him before she was ditched by TV bosses, could return alongside him, as well as pop star Rita Ora. He adds, "Cheryl and I patched up our differences a long time ago. I saw her recently, we speak a lot on the phone and text each other, and I think both of us would like the idea of working together again. "She was a great judge. I mean, apart from being cute she's a great judge and she knows what she's talking about and, oh my God, she wears her heart on her sleeve! I met with Rita at my house recently and I was really, really impressed with her and you just know she's a future star." The show's current judging line-up is Barlow, Nicole Scherzinger, Louis Walsh and Sharon Osbourne, who returned to the project five years after quitting.