You have my sympathy, mostly because your homebrew resembles my homebrew. So with that in mind...
- It makes sense for players to increase what their character does well. D&D resolves this issue with classes - required progression. In other words, a Fighter gets so many skill points, but he's required to spend some in fighting, saves, and hit points. He's free to distribute the rest. So fighting increases quickly, but not too quickly.
Take my homebrew (Modos RPG). It's a point-buy system, classless, but with three requirements: you must spend one point on an ability score, one on a skill, and one on a hero point. The last, a perk, is basically a wild card. Since the GM determines when you get a level-up, there are no experience points, and there is definitely not an increase in points available at higher levels. (So you might consider altering the increasing XP in your game).
- If players are maximizing a combat skill by level 5, they're neglecting others. Why would they neglect others? Should you force them to make balanced characters? Or should you encourage GMs to spread the mix of encounters, so that combat skills are only useful, say, 1/3 of the time? (If a player wants to make an uber-hacker, great! But that won't be too useful when his quest is to make friends...)
- I'm with you on the level progression. I've named my levels, they look like this:
1 - Amateur
2 - Professional
3 - Expert
4 - Paragon
5 - Master
6-9 - Legendary
10+ - Epic
I have set caps for how high stats and skills can reach per level, basically to accomplish the same thing as your three requirements. Oddly enough, they used to be set lower than they are now, and with slower progression, but the same person who is now complaining about the current issue was the one who pushed me to make them higher and have faster progression...
Anyway, my goal for my game has almost always been that a min-maxed one-trick-pony character will be FAR from optimized. Most of the 9 main attributes in my game influence derived stats that are directly useful for just about anyone - even your standard "dumb brute who just hits things" can directly benefit from increasing 6 of those 9 main attributes. In case you're curious, the main attributes are the following, in order that they appear on the character sheet:
-Strength: How hard you hit, how much you can lift, overall how much force you can exert with your body
-Constitution: How healthy and resilient your character is, and how long your character can do stuff
-Agility: How fast and flexible your character is
-Knowledge: How much your character knows and may comprehend
-Instinct: Your character's senses, coordination, and reflexes
-Wisdom: How fast your character thinks, as well as your character's cleverness and ability to understand others
-Willpower: Your character's force of personality and adrenaline
-Morale: How hard it is to scare or discourage your character
-Charisma: Your character's ability to influence others with persuasion
Of those, Knowledge, Wisdom, and Charisma are next to useless for the dumb brute. Wisdom is the least useless, since it helps the Skepticism defense. The rest help said dumb brute in the following ways:
Strength will influence how accurate the character's attacks are, how much damage they deal, and how many attacks can be made per round, not to mention the Fortitude defense, Encumbrance, and some applications of the Athletics skill. Constitution and Agility may also influence accuracy and damage, depending on the type of weapon used.
Otherwise, Constitution helps with HP and reserves (similar to Healing Surges from DnD 4th edition), the Blocking and Fortitude Defense, resisting diseases, and performing physical labor.
Agility will help with moving around, making more attacks per round, the Dodging defense, certain applications of Athletics and other skills, and the like.
Instinct will help with the character's ability to make Granted attacks (Opportunity attacks in DnD), the Blocking, Dodging, and Skepticism defenses, and Perception checks.
Willpower will help with Hero Points, which may be spent to give temporary bonuses in combat; it also help with certain features of some professions, such as the Paladin or Barbarian.
Morale will help with not running away at the first sight of danger, as well as resisting certain Psychomancy spells, by boosting the Dedication defense.
Of course, whether or not any of his will be applied in any campaign is up to the GM, and so it's mostly beyond my influence.