There is no 5' step, only the disengage action.
But the ONLY thing that causes an AoO is moving out of an enemies reach, there is no reason for the caster to back away, he can cast spells all day long next to the enemy if he wishes. I will point out the shocking grasp cantrip is amazing for this, you attack the enemy if it hits does nice damage and the target can't take reactions, so the wizard just moves away.
The only penalty for being next to an enemy is disadvantage on ranged attack, this does apply to ranged spells and the only thing a spellcaster needs to be concerned with.
Also of note the ranged attack disadvantage penalty is only if within 5 feet, you don't provoke AoO until you leave reach, so against a giant for example you back up so you are 10' away and shoot your arrow or cast your ranged spell, you don't provoke an attack for moving and you don't take disadvantage on the attack roll.
One more thing just to be clear, moving out of the enemies reach is exactly that, you can run a circle around the enemy and not provoke an AoO, only moving away from the enemy and leaving his reach provokes.
And since I think it needs to be said, the Disengage action makes you immune to ALL attacks of opportunity for the turn so a common tactic is to use it to run through lines of enemies to get to the target in the back.