This is already not true.
4e, for instance, uses XP first and foremost as a measure of encounter difficulty in combat encounter design. In the DMG, it then explains how these XP measures of monster/trap strength can then be translated into rules for PC level gain, but it also flags the option of simply having the players level after X encounters/Y sessions.
The first time I encountered "level when the GM feels like it" was in a 2nd ed AD&D game over 20 years ago. I don't know where that GM got the idea, but he wasn't the greates innovator of all time in other aspects of his play, so I'm guessing that he didn't make it up from whole cloth. (In a game where classes and multi-classing are balanced across varying XP charts there are obvious issues with this, but I don't think they bothered this GM. And he was in good company here: plenty of classic D&D modules said "You'll need X levels worth of PCs to play this module", without noting that a level of paladin costs about twice as many XP as a level of thief.)
XP as a measure of advancement is
crucial for Gygaxian play. It can also be important for certain simulationist sensibilities, in which XP earned reflects experience gained by the PCs within the fiction, and hence reflects (in some loose fashion) the learning that the PCs have undergone. But these styles of play haven't been exhaustive of approaches to D&D for probably over 30 years.
The question of what the game should default to is, I would have thought, basically a marketing issue. How should WotC present their game so as to maximise the likelihood of uptake and play by their target market?
This is an empirical question to which I, at least, don't have the answer. But it seems to me that the range of approaches to XP is one of the higher-variety aspects of play, based on the posts I read. (In this respect it seems to resemble alignment - lots of different approaches regularly get expressed in posts - and differ from the core combat mechanics of roll to hit, roll to damage, subtract from hp - I think there is a clear default here of doing it the same way its been done from Gygax through to 4e inclusive.)
I'd be gobsmacked if there was no XP advancement chart in D&Dnext. I would also be gobsmacked if the chart is not uniform across classes. But that doesn't mean that the use of the chart can't be optional, and the rationale for using it or choosing not to use it be set out in a senisble discussion of approaches and playstyles, as is the case in 4e.
I'm not sure what it is you're asking to have removed.
For instance: if defeating an orc is worth 100 XP; and the 5 players know that they need 1000 XP each, or 5000 XP in total, to gain a level; then they know that if they defeat 50 orcs they can gain a level. If the PCs then scout out the Caves of Chaos and work out that, in cave X, there are 50 orcs, then they can decide to assaul those caves. And if things go according to plan, they
will get their level.
In otherwords, in any situation in which (i) a certain achievement has a certain XP value, and (ii) players are in a position to have their PCs aim at those achievements, then (iii) players are in a position to dictate their XP acquisition.
I don't see how you can remove this from a game in which the players and their PCs are the driving force of the story, unless you remove the players' knowledge of the value of achievements, so that they can't make plans around that information.
I don't agree with this at all. The players dictating when they got their cookies and medals was at the core of Gygaxian play, as described by him in his PHB: as players, first you scout out and work out what treasures (= XP, in that edition) are where; then you plan expeditions to go and get it. And assuming you play well, your expeditions will succeed and you'll get the treasure (= XP, in that edition) that you were aiming for.
In Gygaxian play, only unskilled players don't try and dictate their own rate of progression and leave it up to random chance - for instance, by simply fighting whatever they encounter, by not using divination magic and sages to work out where the best treasures are, etc.
4e removes the "skilled play" aspect of this, because it removes the "scouting" aspect of play and simply mandates that certain treasures (= player power-ups without the intermediation of XP, in that edition) is accrued per level. But I don't think hat has anything to do with "entitlement". As @
Hobo said, it's about changing sensibilities in fantasy RPGers - since some time in the late 70s and early 80s, a good number of them - perhaps even the majority - have not been wargamers, and so aren't that interested in the "skilled play" idea.