You could also avoid the swim and ride skills entirely.
You have access to a lot of other skills, just at a lesser rate of advancement.
Ride, Jump, Climb, etc. are class skills for fighters, but others like Survival, Knowledge skills, Balance, etc are cross-class skills, which means you can gain ranks in them, but it costs twice as many skill points (2 points for each rank).
So, if you think your Dwarf has never gone swimming and is unlikely to ever ride anything, you can take those 4 skill points and spend them on two ranks in another skill.
If he's a religious character, you could put them in Knowledge: Religion. If he's spent time in the wilderness, you could put them in Survival. Or any other skill you felt fit the character.