The way I saw it, arcane spell failure acted more as a deterrent than anything else. Just knowing that they'd have a chance that their spells would fail at all tended to be enough to make my arcane spellcasting players avoid armor in the first place.
This was my experience as well. If there was even the minimum 5% chance of failure, casters effectively played as if they were completely non-proficient with the armor. Some partial casters could avoid using any spells with a somatic component. And there were a handful of options from splat books to lower ASF, but they only mattered if it could be lowered down to 0%. Also, keeping track of spells that had a somatic component and doing an extra roll for every spell was an annoying process when ASF came into play.
That being said, I really liked the idea of a scale of efficiency for casting in armor rather than a hard yes/no option. I would like to see something like it in 5e. But the implementation of it 3.x leaves a lot of room for improvement.