Noel is believed to be top of Simon's 'wishlist' for the next series.
A source told The Sun: “Simon is determined to get Noel on board. He sees him as his natural replacement as the real alpha male on the panel."
“He thinks Noel would be ideal, with his music knowledge, straight-talking and rapier wit.”
The source added: “Noel’s daughter Anais is a huge fan of One Direction and X Factor, so she’ll be nagging him to say yes.”
And this isn't the first time the seat has been up for grabs to the Oasis star, Noel was also offered the same role back in 2011.
However, at the time the star turned it down to concentrate on his solo project with High Flying Birds.
Meanwhile, the rest of the judging panel looks ready to change with Sharon Osbourne reportedly now in ‘pole position’ to return to the series and replace current judge Tulisa.
However, we reckon The X Factor without Louis Walsh is like imagining One Direction without Harry Styles - it just wouldn't be right.
[Related article: Caroline Flack: Rylan Cark would make an excellent Xtra Factor co-host]
[Related article: Tulisa 'embarrassed' and 'gutted' over X Factor axe]