I'd say yes.
If the spell only has 20% of affecting you AND it only affects you by 20% of its full power, then it isnt really 20% anymore, its 4%.
Its like (maybe bad example) trying to hit an invisible opponent. You already have 50% chance of missing. On top of that, you don't divide damage...