The objective in games that aren't storygames isn't to tell a story.
You may think I'm about to play a game of semantics, but there's an important point to be made, so bear with me....
Games don't have objectives - we may use that phrasing, but it is actually a bit sloppy. Games are non-sentient, without will, and so cannot seek goals. Goals are for people. Games, in the sense you're discussing, may have win conditions. It is the players who have objectives.
Golf has a clearly set win condition - get the ball into all the holes, using the fewest swings, within some restrictions of how you can move the ball. Now, maybe when some folks play golf, that is their objective. Their entire purpose is to get the lowest score, and that's the only reason for them to play. Me, that's not why I play golf. I play golf to go outside with some friends, have a nice walk, and an activity around which we can frame the afternoon. The win condition of the game is honestly not high on my list of objectives. Some other guy on the course that day may have the objective of getting an his girlfriend's father to accept him as a potential son-in-law. The old guy out there may really have the objective of getting exercise, to keep his arthritis at bay. There's tons of reasonable objectives for folks on the golf course.
And that's the problem here. You seem to be taking a fairly simple interpretation - the player's objectives are merely to hit the win conditions of the game. And that is "the objective". Singular. There's only one.
Now, we note that nowhere in the D&D rulebooks do they state what the game's win condition is. In fact, we generally all have a chuckle at the idea of someone saying, "I won D&D". This means that the simple interpretation starts to break down. The players must set their own objectives. And once people are doing that for themselves, all bets are off as to what "the objective" of the game really is.
Now, I agree with you, that D&D really needs to support the tactical-wargame objective crowd. It has always done so, so this is sacred-cow territory.
But then, we should be honest, and remember that the game became popular not just for it's tactical-wargame objectives, but because it has always supported a rather wide range of objectives. Supporting creation of a story - to think of your character as the protagonist of a series of novels - has always been there, too, and that's just as sacred a cow. It has also always supported the beer-and-pretzels objective of hanging out with friends with the game as a framing activity, akin to how I play golf.
Now, to fit all those objectives in, means that the game is not fine-tuned for any one objective. It has to have some mechanics you don't like, that you may want to house-rule away because you're not interested in the objective it supports. D&D should have XP, because it does serve the tactical-wargame objective. And, honestly, I don't think anyone has to worry that somehow it won't be in the core rules. It is a sacred cow, and a few folks on the internet aren't going to take it away. But, to be fair, that means some of the things the story-folks want, that make story-creation easy, also ought to be in the game, too. And the basic play needs to be fairly simple, for the beer-and-pretzels folks, and so on.