You can certainly have D&D variants that are intrinsically compact to keep track of (at least for non-spellcasters). The ones I'm familiar with are all OSR products. Once you get where there's any significant variation from character to character, its going to require some kind of reference sheet, simply by consequence of the fact that all these games are firmly in the exception-based design world.
You lose a lot though, if you drop the variation from character to character you've entered a space where a lot of players and GMs can't really follow-- a lot of us actively seek out a high degree of character customization and strategic texture in our games, revolving around character ability sets. Like, we recently rejected Stars Without Number in favor of Starfinder for a space-and-lasers type experience because of it (and after we realized what HP is like in that game.) I think its great that the OSR niche exists for those that don't care about that, but I'd rather not have their viewpoint become 'the only acceptable standard' for where complexity and depth should be for games.
Pathfinder 2e does a great job in streamlining its complexity down without cutting out depth, its obviously more complex than some games, but its also way less complex than some people seem to be suggesting, it really is only marginally more complicated than 5e in my eyes. The three action system is simpler to understand than that game and while there's more to character building, its all modular and built as elegantly as can be-- that's the fundamental function of it's lego brick feat design. There's a lot of rules elements, and if you don't value in them I can see why they would be considered clutter-- but that viewpoint is mired in its own tunnel vision, because obviously different games are made for different people.
For the rest of us, those rule elements are simultaneously designed in such a way that they only matter if you take them, and they open a wealth of customization and tools you can equip a character with to do interesting things in the actual game. Its fun being able to differentiate your character using all the pieces the game gives you and its design makes it like a placid but deep and wide lake you can swim in, while the old complex systems were like these oceans where the waves would batter you even as you tried to enjoy swimming in the water. The game is balanced enough that you don't have to look at every option, you can take whatever looks fun on your glance through and the game will still work well for you, while people who like tinkering can obsess as much as they desire, and both people can play in the same campaign without it devolving into a moral issue.
By comparison 5e makes me feel like when I was growing up and got taller, and i always had to kneel in a friend's small pool, I couldn't really swim or exercise myself much in it. OSR gives its own benefits (and this is why you'll notice I'm super interested in the potential 2e has to support some of the elements of an OSR) but the systems themselves give up way too much for me to be happy with them, you know?
I think that its great that 5e exists for the people who enjoy its particular degree of complexity and depth, I think tis great that OSR exists for those who don't care for all the customization, but I'd like to keep the system that does such a great job of streamlining complexity so much, that even my players who have major issues prefer its depth and their action variety. I can't help but feel that some people find that diversity threatening, as if they need the game to fail in order to demonstrate that their preferences are valid.
Edit: for reference, this grew into more than response to you, as I'm circling back around to the essence of the thread as an attempt to spin Pathfinder 2e as failing due to its key differences from 5e.
