Why do you have to only imagine your wizard never missing while the fighter does?
Wizards miss all the time, with spells from
ray of frost all the way up to
disintegrate.
The wizard spells that don't have to worry about missing are the ones that blast an entire area and everything in it. A
fireball doesn't "miss" because it's everywhere at once. At the moment of detonation, there is no place in the area of effect where the fireball is not hitting, and thus no way for it to miss.
The same is not true of a sword; no matter how fast you swing it, the sword can only be in one place at a time, and the enemy could manage to be somewhere else at that time. Fighters using true area attacks (nets, alchemist's fire, et cetera) should also not have to worry about missing.
(Of course, there is always the possibility of hitting the wrong area--your flask of alchemist's fire lands in the wrong spot. I would be quite open to imposing such a constraint on wizards, too. It always bugged me that a wizard could position a
fireball with such precision as to fry a hobgoblin, yet not even singe the paladin slugging it out with the hobgoblin in melee.)