Dermot, 42, presented the ITV singing competition for eight years before stepping down last year, being replaced with Olly Murs and Caroline Flack.
Now Dermot has insisted boss Simon Cowell should give the show a ‘rest’ to help the public fall back in love with it.
Speaking to the Radio Times, he said: “If I was Simon, I’d take it off air for a year. I’d rest it, then I’d get back to the [audition] rooms. For me it was always the room, arena, boot camp.”
He also hit out at the controversial Six Chair Challenge, admitting he doesn’t think it’s very nice for viewers for watch.
Perhaps in a dig at his former employers, Dermot claimed the public have swapped The X Factor for BBC One rival Strictly Come Dancing – because it’s a nicer show.
He said: “I know where they’ve gone, they’ve gone and watched Strictly. One of the reasons people like Strictly is because it’s a very warm show.”
Olly, 31, and 35-year-old Caroline were regularly criticised for their performances during live shows, and Dermot confessed this served as a reminder that not just anybody can front a prime-time TV show.
“You never want people to fail,” he explained. “But sometimes you need a few cock-ups to remind people it’s a finely honed craft.”
And will Dermot be passing on his friendly advice to 56-year-old Simon? Not likely, considering they haven’t spoken since he left the show.
He added: “We barely talked when I was doing the show. We have a very professional relationship.”
Do you agree with Dermot? Let us know on Facebook and Twitter.