Let me put it this way: I've played a lot of modded games in my life. Mods can be absolutely great. But they also bend the experience. Bend it too far, and it's...not the game anymore. And something which makes one specific class horrendously, ludicrously overpowered is pretty much exactly that. No class, caster or not, should be that powerful for the kind of game D&D purports to be.
Well, I did say that the rogue damage thing was exaggerated.
Even so, let's go with it for a minute. Imagine a game where the entire group wants to play rogues for the campaign. Perhaps they want to start a guild or something. Enhancing the damage doesn't create any sort of PC disparity and could be very fun for the players. The DM doesn't mind because he can always include more creatures to create a challenge, or perhaps combat doesn't play a large part in the campaign. It could still be lots of fun under certain circumstances.
As for bending the game experience to the point where it's no longer "the game"(which I assume means D&D), that line is going to be different for every person and group. For some if you don't play RAW or very nearly RAW, it's not D&D. For others you can take the rules and stretch them, wrap them together again, stretch them some more, and create a weird taffy concoction which is still D&D to them.
D&D purports to be a general sort of game that isn't great at any one style, but is decent to good at all of them, AND is eminently customizable. That last part makes it D&D for the group no matter how much is changed, so long as they are happy with the changes.
What you have experienced may bend the game to the point where it no longer feels like D&D to you, but that doesn't make that particular bending point a general break in what D&D is. It's a personal thing, which is absolutely fine. We all have our lines in the sand about various aspects of D&D from races to classes to house rules to realism, and beyond.
The key is finding a like/similar minded group to play with.
I'm not speaking of the DM only doing it to their own stuff. I'm speaking of the game that prompted me to actually become a proper GM myself. Specifically, a friend of mine, call them Adam, went through a very very bad breakup about seven or eight years ago now. As part of working through his emotions regarding that breakup, he decided to get in on this whole "D&D" thing everyone was talking about (given we're all MMO players, we're all already D&D-adjacent anyway.) Adam's first DM was brand-new to the game. I don't know the specifics, but one of the players had the DM wrapped around their little finger, basically the DM giving that player and only that player anything they wanted and more.
So...in Adam's very first actual TTRPG game...there was a player who was playing a custom demigod race that gave a ninth-level spell as a racial feature. And that was only the MOST egregious thing. I vividly remember Adam discussing it with me and being, frankly, emotionally shredded by it. He didn't have the heart to complain because (a) first-time player, (b) first-time DM, (c) he genuinely wasn't sure if TTRPGs were worth this sort of experience. After only like two sessions, he was VERY much on the "bad gaming is worse than no gaming so maybe TTRPGs just aren't for me" point.
Ugh. Games like that are awful. That reminds of during 4e. I had read the rules and had discussion here from the playtest well past the release of the game, and I was told time and time again that the game plays very differently from how the rules read, and was very fun.
Along comes one of the local Los Angeles game conventions and one of my players and I were hanging out trying to decide whether to just do some open gaming or get into one of the scheduled convention games. I suggested we try out a 4e game to see if it really did play better than the rules read. We got to the game and the DM had about a dozen players(first bad sign at a con game) show up. He handed out a bunch of 30th level PCs, as well as some special characters. I got handed an ancient gold dragon. As you can imagine, the game became more and more absurd as time went on.
About an hour or so in, he took a short break. During that break my friend and I quietly packed up our dice and extended that break to forever. If that had been my first experience, I probably would have tried again to see if it was a fluke, but I can see where someone else might just have walked away from D&D.
That was what finally kicked me in the butt to actually GM. I was, for a very long time, afraid of doing it wrong. Of having precious DMPCs and eyeroll-inducing storylines and cheap, trite drama etc., etc. Impostor syndrome at its finest. But I took one look at that and knew, beyond any doubt, that I could not possibly be THAT bad as a DM--and that I could do better for Adam than what he'd endured. So I did. Turns out I'm a pretty good DM (at least my players think so; I'm always doubtful.) And Adam paid me one of the finest compliments I've ever received when, for IRL reasons, he needed to pull out of all of his TTRPG games. He said that my game was the only one that it actually felt difficult to leave. That meant a lot to me.
I feel you here as well. I've been DMing for as long as I've been playing, 41 years and climbing. I know that I'm a good DM as I've been told so for decades, but I still feel that anxiety and doubt during a campaign. A lot of that stems from the pressure of wanting to provide a fun experience I think.
So....yeah. The thing you described is pretty much
identical to the unequivocal rank abuse of Rule Zero that I have personally known. I'm really not in favor of anything like that. Doesn't mean I don't think people should work together to make the rules sing for them. I just think there need to be
some reasonable limits. As
@Micah Sweet said, Rule Zero is best deployed to fix the places that don't quite work, or that do something bizarre or inappropriate. Homebrew is totally fine, but homebrew is
quite distinct from Rule Zero and merits a clearly different approach and understanding vis a vis Rule Zero.
Here's where I'm going to depart from agreeing with you. I've been using Rule 0 to house rule the game since I started DMing. Have the changes always been good? Hell no. Some of them have been downright bad, but when that becomes apparent, I change it to get rid of the bad. A lot of them have been neutral. They change the game, but it only makes the game different, not better or worse. Those I ditch at the end of the campaign. If different doesn't equate to more enjoyable, there's no point in it being there. A lot of them, though, have improved the game and made it more enjoyable, and those times alone make all the rest worth it.
If I had feared the bad or neutral changes and stopped tinkering, I would be playing a less enjoyable version of the game than I play now. And so would my players. Making improvements upon the game is worth the speedbumps that you encounter along the way.