Why do we need to do mechanical negative effects long term?
I mean because first of all in real life there are mechanical negative effects and in stories there are actual defeats that can happen that have real consequences and cause characters to suffer real hardships.
And if my stories at the gaming table can't do that, then I don't feel like they are very satisfying. I don't want to play in a game where there are never any meaningful long-term consequences to either overcome or represent some sort of failure state. What exactly would I be playing for at that point? Where is my payoff? For that matter, as a GM, one of the most satisfying moments is the "Aha" when your players solve your puzzle, or your mystery, or pull off the clutch tactical move that saves another PC or the party from loss and everyone cheers because something was at stake. That's the vicarious thrill that I love as a GM. I hate going, "Maybe I should knock 30 hit points off this monster just to make sure the game doesn't derail here?" or "Hmm... the players are stuck, how can I have an NPC get them back on track without it feeling like I'm solving my own mystery."
Why would I play a game where I'm basically guaranteed to never suffer a meaningful loss? Why would I role play a character that has such small stakes in the story that they can never lose anything that is meaningful?
What you seem to be saying here is that if your player interest is self-centered and your player interest is entirely on investing your ego in the character, that the game ought to inherently validate that. Like sure, maybe if you and the character you animated really cared what happened to an NPC or what NPCs thought about you, then there might still be something at stake in such a game, but if you don't then you're staking nothing. And if you staking nothing then you are demanding a railroad from the DM that takes you straight to validation land with no detours.
There is more than one way to take away player agency. If the game rules say, "Go straight to victory, do not pass failure", then well how is that any different than pure illusionism where the DM always puts on the padded gloves? Is the only difference who is engaged in the illusion? Is the only difference the player trust in whether he will always get his validation?
Pardon me, but go play a single player game and use cheats. Or go play a multiplayer game with cheats for that matter. That will deliver the experience you want and I won't have to be involved.
It's a lot harder to add consequences to a game that is lacking them than it is to remove them from a game that has them. You don't want challenge as part of your aesthetic of play, then fine. Why deny it to the rest of us?