But seriously, you asked to share experience and because AoO has been around since 1st edition, it's hard to do so as taking it away tends to punish players far more than the bad guys and you'd have to House Rule it away to know what it's like. Otherwise:
AD&D 1st edition: Free attacks on anyone leaving melee. That's plural. Don't flee from bears, hydras, etc.
AD&D 2nd edition: Same thing + added withdrawal rules.
D&D 3rd edition: Got crazy and now we have a table to check what triggers an AoO.
D&D 4th: I refuse to recognize it as a gaming system.
D&D 5th edition: Dramatic simplification, consumes a Reaction, limited to 1 attack.
Removing it would denigrate certain class abilities, such as the Monk's Step of the Wind, that allow them to whisk through combat threat zones, and Rogues, who can skirmish without risk by class design. Removing would also remove a layer of strategy for formation since enemies or players could simply walk by.
Now, if you wanted to adopt 1st edition rules, that if they aren't engaged in melee with you, they can waltz right by, that might be worth a test. You could define engaged as one of them took a swing at the other. But man, it's such an integral part of the game, I can't visualize it working.