It sounds like the game became really complicated as the characters gained levels and the game itself matured, something 3.x suffered from as well.
This is...kind of right, but mostly wrong.
It's "kind of" right in that, yes, higher-level characters were more complicated than lower-level characters at basically every level of play...but that obfuscates the
degree of difference, which is what's really relevant here. After all, 15th level characters in 5e are more complicated than 1st level characters, even if you're looking at options specifically designed to be low-complexity.
The thing with 4e is, after a certain point, you stop gaining quite so many
extra options and instead start
replacing older options. E.g. no character gets more than 4 regular Daily powers, and even then you only get that 4th power at level 20. No character gets more than 4 regular Encounter powers. (Some classes have exceptions to these patterns, but those exceptions are uncommon.) Instead, at (for example) 13th level, you
replace one Encounter power you know with a new one of your current level (or not, you could always choose to just keep your old powers if you truly like them better...but there's rarely much benefit to doing so.)
This means that, while the game does ramp up noticeably across the 1-10 range, it actually slows down
dramatically in new complexity from 11 on up. It does still gain complexity, I don't want to pretend it was some perfectly flat power curve or anything remotely like that. But the power curve significantly flattens out at that point.
By comparison, 3e goes full-on
exponential growth as you get into the higher levels, and their complexity grows
faster as you get out that far due to the accretion of items and feats and abusable optimization strategies. You emphatically do not see such things in 4e, and 4e does a very good job avoiding the problems of "rocket tag," "scry-and-fry," and the "five-minute workday" that plagued 3e so badly.