In 2011 Simon, 54, took 31-year-old Cheryl over to America to judge the US version of the hit talent show, but she was dropped after filming just one episode.

PICTURE: TULISA MAKES X FACTOR RETURN AT JUDGE'S HOUSES IN BERMUDA - CLICK TO SEE

The firing led to a feud between the friends but, following their reconciliation and their returns to the UK version of the show, Simon has discussed his reasons behind dropping Cheryl.

Speaking to Radio Times, Simon said: "When I saw her I didn't recognise her. Literally. The hair and outfit were crazy. Just totally crazy."

He was referring to Cheryl's now infamous ruffled orange top and flared purple trousers. Simon continued: "I asked her manager, 'What's she wearing, what's happened to her hair?'

"He was defensive about it. I haven't a clue why she turned up like that. The only way I can describe it is that it wasn't Cheryl sitting in that seat. She'd lost her confidence."

READ: TULISA CONFIRMS SHE IS GETTING 'BACK TO BUSINESS' AND RECORDING NEW MUSIC

Looking back now, Simon admitted there was something 'not quite right' about the US version, adding: "In hindsight I don't think that first day worked anyway."

Surprisingly Simon said he was wrong to fire Cheryl, insisting it was the 'second-biggest mistake of his life'.

He said: "If I could go back in time I would have kept her on the show. It was stupid doing what I did. The worst it got between us was when she sent me a text message.

READ: LOUIS WALSH ON TULISA'S DRUGS TRIAL – "IT WAS COMPLETE ENTRAPMENT, I HOPE HE GOES TO JAIL"

"It was like when you get a really horrible letter from an ex-girlfriend. The general tone was something along the lines of, 'You've hurt me, you've let me down, I can never forgive you'. I shrank."

However, things are finally getting back to normal between the 'frenemies' as they both prepare to take their seats on the judging panel of this year's show.

The X Factor returns Saturday on ITV1 at 8pm.