For the first time in the show’s history, it was left up to the public to decide who should be in charge of each group - and that has left Simon with the Overs category he apparently did not want.
Cheryl Frenandez Versini got Groups - which she had wanted all along, and Rita Ora has been put in charge of Girls, which she also seems pretty chuffed with.
But it was the boys who lost out this year - with Simon getting the Overs category, and Nick Grimshaw getting Boys.
Taking to his Twitter page, Simon reminded everyone that he did win with Overs last year (with Ben Haenow winning the X Factor crown), so he will try and ‘do great things this year.’
Tune into the X Factor on Saturday at 8.15PM.