Describing her as "one of the most beautiful women I have ever seen in my life", Simon says no one could take their eyes off Cheryl when she arrived at the ITV studios.
He told the Radio Times: "I mean literally, when she walked on set on the first day in the UK I thought that she was one of the most beautiful women I had ever seen in my life."
And it wasn't just him who fell for her, he reckons most of the production staff were in love with her too.
He added: "I think my entire production staff - including a lot of gay people were madly in love with Cheryl."
Simon admitted that "a lot" of the women he hires for his reality shows are only given the job because he finds them attractive.
The 52-year-old is even said to have had a secret romance with Dannii Minogue, rumours which were printed in his unauthorised biography last year.
But despite his long line of ex-girlfriends and former fiancees, Simon says he's happy to be on his own.
He said: "I’m happy single, and when I’m in relationships as well. I tend to be in a relationship more than being single.
"But I’m very good on my own, a) because I never get bored and b) because there is always something I need to catch up on.
"I break up very well. I am a good breaker-upper."
Simon also took the the time to address rumours that he's gay - which he says are a load of rubbish.
He told the magazine: "I work in possibly the gayest industry in the world. Music and TV. It would make no difference to my life or my career.
"A lot of my friends are gay but I'm not and don't even think that way any more. I couldn't care less (if people think I'm gay) because it's nothing to be ashamed of."
[ Related article: Cheryl Cole gets naked for a photoshoot in the bath, looks AMAZING ]
[ Related article: Cheryl Cole ditches her strict diet regime as she heads to LA for some time off after split ]