That's a bit harsh isn't it?
According to The Daily Star, Simon is adamant that the former winner won't be allowed to perform on the show.
A source close to Matt told the paper: "It's a real slap in the face. Regardless of what's gone on, you'd think the show would celebrate the success of its winners.
"Matt would love to go back and perform but sadly won't ever get the chance. He's not sure what he's done to upset Cowell."
[Related Article: Review: Matt Cardle- The Fire]
[Related Article: Matt Cardle: JLS and One Direction are puppets]
Well it could have something to do with Matt bad mouthing Simon's label Syco in the past.
Matt has previously talked about the bad treatment he received from former boss Simon Cowell, claiming that One Direction had been given preferential treatment over him.
He even described One Direction and JLS as 'puppets' as opposed to artists.
Yeah, that would probably do it.
Matt Cardle previously admitted that he would be willing to perform on the X Factor but was doubtful as to whether he would be asked again.
He said: "If they ask me then I would definitely do it but they won't ask me, I can promise you that."
Matt released his second album 'The Fire' this week which placed at number seven on the midweek chart.