Ok consider this.
If a guy plays a Fighter, and all they do is Dodge every turn, how are they helping the party?
This is exactly the position you're saying the Wizard should be in. Unable to do anything but focus on their survival, using up their valuable resources to do so. You might as well not even have them there (or, if you see value in them distracting enemies from the other players, you'd be better off with the Fighter Who Dodges wearing a floppy hat and a shirt that has "WIZZARD" scrawled across it).