Quest XP was pretty standard, I think, but I didn't think adjusting combat XP was all that common. But my experience with early D&D is very limited.
Of course, very early on, most XP was from treasure, and that was tied directly to what treasure table the monster you fought had.
But the salient issue is that XP wasn't always an "encounter design" tool. It was a reward, nothing more or less. Creating "fair" encounters wasn't really the point. If the random encounter table says you encounter "Orcs", then you roll for how many. It could be 1. Or it could be hundreds.
In 3.5, we got the CR system, but it didn't work very well. It was confusing, and general game balance problems made it rather superfluous.
XP as measure of challenge was new for 4E. And I think it works quite well. But it relies on a few important assumptions: an encounter will have total XP of (standard monster XP of level [party level +/- 4]) * party size, that monsters will not deviate more than 7 levels from the party, that monsters are present and free to act from the start, and that parties will get short rests between them. Those are important assumptions, because it forms a context the monster will be used in; how big of an encounter it'll be part of.
The problem in the L&L article is with trying to generalize that to Adventure XP pools, and have that XP value make sense whether the monster is alone, or in a group with a bunch of others. And I don't think that really works. Once you take away that encounter context, a simple XP amount is not a good measure of challenge. Adding in some kind of "overhead" XP, counted against that Adventure pool, for additional monsters per encounter could work, but then we're basically back to Encounter-based design, which is what a lot of traditional D&D players don't like, and what I think Mike was trying to avoid.
In addition, if that Overhead XP is purely DM judgement, then I don't think there's much point in having the system. If you standardize on assumptions of monsters per encounter and encounters per adventure, then you're basically just back to 4E-style encounter design.
In all fairness, DM judgment will always play some role, but I think you're right in that it isn't sufficient. While an experienced DM might be able to eyeball the difficulty of a non-standard encounter, a newb will almost certainly be unable to do so.
That said, I think it is feasible to create a system that reasonably balances 10 fights in the same day with 1 big fight. I don't expect it to be perfect, but as long as it works most of the time, I'd be happy.
The reason IMO that XP works better than CR is that it is more nuanced. One CR up or down is a large jump in power; you could even often add extra lower CR creatures to an encounter for "free".
As I said, XP is more nuanced. A low xp creature might be a low level creature or a higher level minion, for example. If you add a lower level creature into an encounter, the xp total still increases (unless the DM rules that the creature's presence is negligible). Of course, although it's pretty good, 4e encounter design is by no means perfect.
Perhaps then, what we need is an even more nuanced approach to encounter design. It might be a table, it might be a formula, I don't rightly know. But the way I envision it, this method would distinguish between the big fight and numerous little fights, and adjust xp accordingly.
Let's assume, for the sake of argument, that 10 encounters with 5 creatures drains roughly the same resources as 1 encounter with 25 creatures. You'd then decide what kind of encounter you want to run, look it up in the table, and find the xp value (which you'd then deduct from the adventure total). So you might decide on two 5 creature encounters (100xp per encounter), and one 15 creature encounter (500 xp), with some xp left over for non-combat challenges.
In a sense, it's like layering aspects of the CR system over the xp system, in order to (hopefully) come away with a more precise metric for judging the difficulty of encounters.