Nick, 30, joined The X Factor as a judge this year, along with singer Rita Ora, and the Radio 1 DJ is still wary of Mr Nasty.
Speaking to the Daily Star, Nick said: “Simon is intimidating to me… and I’m on the panel with him!
“Sometimes I’m like, ‘It’s Simon Cowell off the telly.”
Nick also spoke out about the backlash he received after replacing 63-year-old Louis Walsh on the ITV show.
Revealing fellow judge Cheryl Fernandez-Versini has been supportive, Nick continued: “She’s been very caring to me and welcoming.
“She phones me all the time and checks up on me. I knew there would be controversy coming on after Louis who is a national treasure – that’s big shoes to fill. Anyone would have got stick.”
This weekend, following the gruelling Six Chair Challenge, it’s the judges’ houses round, and Nick is taking his boys category to the Cotswolds.
It was recently revealed that controversial contestant Mason Noise re-joined Nick’s line-up, after Tom Bleasby quit the show.