They kept that quiet.
And it seems that Harry even wanted to make things between the pair official once he had completed his Australian tour.
An insider told Radar Online: "Taylor really liked Harry and even though they weren't exclusive, he hinted at making it official with her just before he took off to Australia."
"He even told Taylor he didn't want her to see anybody else while he was gone."
[Related article: Taylor Swift: I'd rather sing than talk about my personal life]
[Related article: Taylor Swift and Ellie Goulding - new BFFs]
So what went wrong?
Well apparently, Taylor was left with a broken heart after seeing pictures of Harry kissing another girl in the press.
The source explained that Taylor began to receive text messages from Harry Styles telling her to ignore the rumours on the internet that he had been dating other women while in Australia.
The source said: "Taylor had all these texts from Harry begging for a chance to explain and telling her not to believe everything she reads."
"She immediately googled Harry and found all the stories and pictures of him kissing Emma. She was heartbroken."
Suddenly the mystery man from Taylor Swift's hit song 'We are Never Getting Back Together' is looking all too familiar.
Well if she was indeed left heartbroken Taylor now seems to have got over it after falling into the arms of Conor Kennedy.
Although we bet Xtra Factor host Caroline Flack and Taylor Swift had an interesting chat after her performance on last weekend's X Factor.