Enraged viewers then took to social networking site Twitter where #FixFactor started trending after the Executive Producer was seen whispering in Louis' ear during Carolynne's performance.
After finding herself in the bottom two with controversial contestant Rylan Clark, Carolynne Poole had looked set to survive with two votes to one in her favour.
A seemingly torn Louis was pushed for a decision with the judge saying: "I'm going to go with Carolynne, I'm going to save Carolynne."
An ecstatic Carolynne's elation was unfortunately short lived when Louis suddenly changed his mind.
Louis told Dermot: "I want to take it to deadlock."
Dermot then revealed that Carolynne had polled the least votes and was therefore out of the competition leading to Gary storming off stage.
On the Xtra Factor an irate Gary Barlow told Olly Murs and Caroline Flack: "Two years running now and you've [Carolynne] lost out to joke acts."
"There was no deadlock there. You judge it on the performance, how can you compare those performances?"
Jeez, it's only week one and already The X Factor's had its first controversial decision.