Yep, even without any sudden stuff, the +4 modifier is already broken. You surely do not want a 6th-level irresistible Tasha's Hideous Laughter or the above-mentioned 8th-level irresistible Phantasmal Killer and stuff like that thrown around.
Irresistible Spell, if you were to actually allow such a feat, should have a +8 metamagic modifier, I'd say.
That would be remotely balanced.
If you are worried about breaking the spell level cap, then just use this general house rule:
When you apply a metamagic feat, that does not increase the level of the spell directly (i.e. via Sudden Metamagic, Divine Metamagic, Instant Metamagic, metamagic rods, etc), you can only use it on a spell with a level equal to the highest level you can normally cast minus the metamagic modifier of the *regular* metamagic feat the one in question is based on (i.e. Maximize Spell is +3, so Sudden Maximize could only be used on a 2nd-level spell at most, if the highest spell you can cast is 5th level).
Bye
Thanee