That's the thing, when you start pulling on one thread of the the game or another, you inevitably find other areas that could use a tweak or buff. You either end up with a dozen house rules, or a new iteration of the game, with its own problems.
If I were trying to stick to "small" tweaks though, I'd aim squarely at the skill system.
- History becomes Lore and you use it for any knowledge your character might know from background, class or race. If it's out of your wheelhouse of experience, inflict disadvantage or change the DC.
- Expertise grants you double the proficiency bonus in two subskills, with a few specified for each skill and the option for the DM to add more should they come up. Each class has a skill or two they can get Expertise in (the Rogue can do it for any, and perhaps gets more every few levels). For example, Athletics would have the subskills Jump, Climb, Swim. Later in the game, after the players have to dig themselves out of a cave-in, the DM decides to add Dig as a selectable subskill for future play. (You could go a step further and roll some of the Int skills as subskills of Lore - Arcana, Nature, Medicine, even possibly History itself)
- Mastery gives you a 1/short rest bonus of +10 to a specific skill roll, and any time when not rushed you can treat the roll as a 10. Like Expertise, each class gets one or two skills they can pick up Mastery in - the Rogue can get Mastery in any skill and has the opportunity to have in multiple skills.