So, we may be approaching adding an extra issue here....So, here's the added issue - the difference between the rules, and the presentation of the rules. What are we referring to when we speak about "the system"?
I don't think it is an added issue, so much as it was an example of the system coloring how the participants thought about play. The system is the text. In AD&D, the difficulty of the text and the freedom that DMs felt to add or subtract from it or just ignore it when it was handy to do so, led to the text playing an even smaller role than usual. And, further, the text could be and was read as endorsing this interpretation of the text.
As soon as you say "in practice" you are speaking about metarules, not rules, aren't you?
It certainly sounds like I should be, but in practice, I'm not sure that I am.
For one thing, I think there is a canonical answer to the question of what happens when you flash ankle at the guard that the designer expected to be applied in such situations, and it's certainly as clear or clearer than the designer's intentions with regard to surprise, initiative and weapon speed factors. That answer is on page 63 of the DMG, and it is in essence pretty much how you'd expect to handle this situation in any edition of D&D. The DM makes an encounter reaction check for the guard adjusted by the PC's charisma (and possibly by the NPC's personality, if known). The PC then opens a parley, the success or difficulty of which will be broadly determined by the degree of success in that initial reaction roll. The DM takes on the persona of the guard, judges for himself what the degree of success indicates, and in some fashion RPs out the encounter.
Compared to applying the rules exactingly when a 10th level ranger with a hand axe +4 in one hand and a torch wielded as a weapon in the other, wearing plate mail, attacked a mixed group of goblins, orcs and bugbears some of whom have missile weapons, from 6" away in semi-darkness, while a tribal caster is targeting the Ranger with a spell, and the leader of said foes being armed with a longsword of quickness +2, this is simplicity itself. I dare say most groups dispensed with weapon speed factors, odd initiative complexities that resulted from different weapon types, segments, oddities in the surprise rules, the rule that let fighters make level attacks per round against foes with less than one HD, weapon versus AC modifiers, and just rolled a D20. Some of them may have even dispensed with the distances and charge rules.
This is why I said I'm not sure which the system has more difficulty with, and why I feel the need to say things like "in practice". In practice, reaction rolls were rare at the tables I played at - though this one situation where I think the oft forgotten rule would have been remembered because it was needed. In practice, combat was typically simplified and streamlined so that it played something like the simplified system used in 2e or 3e. One way or the other both systems give you an answer. I can't imagine an experienced DM being stuck on either problem. Nor do I think it particularly likely that the RAW would hold sway all the time, which I don't think the RAW intended to anyway.
But to me, it's less important what the rules say, than how individual tables would have handled the encounter routinely and whether they even considered whether they were breaking the rules to have done so. If the DM routinely went straight to RP without a reaction roll, that's one process of play. If the DM only allows a victory through social interaction, if he calls out ahead of time in his preparation that it is a possibility, otherwise combat must be employed, that's another process of play. If first person RP is avoided all together, with or without a reaction roll, well that's another process of play. And it's not clear to me that any are not by the rules.