I do agree with you regarding Random Monster Tables in the DMG/MM. I don't want these either. I have not used them since I started creating my own games around 20 years ago. However, I certainly create random Encounter Charts (discussed above) for each area of my own campaigns and believe advice on how to do so creatively would be great.
I agree that at least the 3e random encounter tables are all but useless.
Instead, I would like to have tables for environment types, where all appropriate monsters would be weighted by rarity. You could use this to populate areas and generate your own encounter tables. If it's also sorted by level, you could easily limit yourself to a part of it.
For example, imagine you have a temperate highland to populate and you decide to place three main communities there. You roll elves (gray), orcs and griffons. The elves obviously have an alliance with the griffons to defend against the orcs' slaver raids and egg-stealers. Then you can just add some appropriate animals and maybe roll a rare monster or two to add to the tables.