No I don't think it does because if all you are doing is moving silently, the likelihood is that you have cover or concealment from those listening. Those asleep have their eyes closed, those inside the room have limited views through the door etc.This doesn't fix anything. It just forces the character to re-roll stealth every six seconds while moving down an empty hallways, because they "used movement" even though nothing else changed.
Forcing players to continuously re-roll their stealth checks just forces them to eventually fail, and is bad design.
But I agree, it should not be an absolute rule. Stepping up behind someone who is focused looking the other way should not automatically break your stealth.