Unfortunately, this is a generalisation so broad, and so regularly flouted, that it undermines pretty much all of the rest of your post. In particular, it was common with pre-3e games for the PC party to be composed of members of different, and sometimes widely different, levels.
I find XP enjoyable and helpful. They give granularity to the progression system, allowing for small rewards that culminate in leveling. You could skip that and go right to levels, but then it is hard to give anything for defeating a single encounter. The thing is, if you do not like XP it is easy enough to ignore. If you do like it, and they take it out, then it is very hard to house rule back in. Better to have the system, and let those who dislike it ignore it.
You gain XP to level up is pretty much the norm, even if some adventure or DMs may skip this part to decide when leveling occur, the norm shouldn't necessarily be relegated to an optional rule IMO. I certainly think it'd be good to have advices on various advancement methods in the DMG, but i for one wouldn't want to see XP removed from the core.
I calculate each experience point my players earn based on the challenges they overcome. I include in my adventures many challenges which are both below the average level of the PCs, and above. Sometimes well below, and sometimes well above. If they can overcome a challenge well above their level, they will get much more experience for that than they would for something of their level. There's no illusion about this methodology - same method I've always used, since 1e.
Your system to calculate the difficulty of a challenge, appears to me to be level dependent,
So you could have decided that they level up after, say, 10 encounters. Much simpler solution, no? And you could still have calculated level-dependent challenges in exactly the same way. However, you would have done away with the need to track and calculate XPs, and the same for your players.
That only started when adventure designers decided D&D needed to be a railroad (sorry, "Adventure Path"). Obviously, if that's the game you're running, an XP system doesn't do the job. XP was designed for a sandbox game.However, use of experience points is an illusion...whatever the level a group of PCs is, that group will only undertake adventures designed for that level. Not only that, but commercial D&D campaigns and adventuresd even calculate how many XPs the PCs are likely to earn during each adventure, and the level of each adventure is accordingly calibrated.

(Dungeons & Dragons)
Rulebook featuring "high magic" options, including a host of new spells.