What about the smaller races? This is my imagination here(so based on nothing solid), but it seems to me that smaller peoples would be more likely to develop cultures where warfare was something to be avoided, so more gatherer and agricultural, which would affect other practices and traditions. And that the taller than average peoples would be more apt to fight than flight, prize strength and those things would affect their practices and traditions.
More likely: as they get better, more reliable access to food, they get taller, and use the weapons best adapted to penetrating their opponent's armor.
The genetic limits on height are a minor variable. For example, in the 20's Franz Boaz noted that the children of European immigrants to the US were almost universally taller that their parents, regardless of where in Europe they immigrated from. At the time, being poor in a city in the US meant more and better food than being poor in a rural village in Europe.
Basically, if you moves a group of !Kung (The shortest tribe in Africa, IIRC) to Northern Kenya and had them live like Maasi, they wouldn't change how they herded, they'd just get taller as their diet switched to more red meat and tubers. The main cultural shift would be towards longer bows and thinking about cattle a lot more.
There's some genetics to height, but it's mostly nurture, in humans. Which, presumably, wouldn't apply to halflings living in the mountain alongside goliaths.
If I had to guess: one of the bigger externally-selective differences would be the relative size of animals. To a halfling, a horse is
huge and way too big to ride like humans to; you'd be unable to grip with your legs. You have to ride them like humans ride elephants, with howdahs or some other kind of platform. You could put a whole house on the back of an actual elephant, and you could comfortably ride many smaller animals like dogs, goats, or ostriches. All of which do not work quite like horses, which would have knock-on effects on a number of things, like trade, war, and the meaning of wealth.
Goliaths would be hard-pressed to find any kind of riding animal, so I'd expect them to just not do that. They also wouldn't use many beasts of burden given that the ones who are big enough to be worthwhile are also too big to go many places (they need a lot of food) and are a lot slower than horses. So you'd have an entire unmounted culture, who'd learn to live with only things you can personally carry (or maybe put on a big goat), which in turn starts having knock-on effects.