Mechanically, the biggest offender by far in the "why does everyone use weapon X" is the Exotic Weapon Master prestige class. The only non-double weapon that you can easily get flurry of blows with is the spiked chain. If there weren't a special "must use a spiked chain" prestige class that gave an extra attack, the spiked chain wouldn't be nearly so popular.
As for the rest, I really don't think it should be that much of a bother that people will often take the same weapons and armor. IRL, nearly every culture had their preferred weapons (which were usually the ones that were effective in that particular era). The classical greeks? Longspear, shield, breastplate and shortsword/dagger for hoplites. Peltasts and auxiliaries might have just carried a buckler and javalins or slings. The Romans? javalins, spear, tower shield, shortsword, for the legions. Longsword, heavy shield for cavalry. Lorica segmentata, and lorica hamata (and one other armor I don't remember atm) for armor. The viking era? Chain shirt, heavy shield, spear, javalin, shortbow, battle axe, great axe, longsword, and dagger for the vikings. The era of the Norman conquest? Chain mail, lance, longsword, heavy shield, and shortbows for the Normans; chainmail, greataxes, heavy shields, longswords, and shortbows for the saxons. Feudal japan? composite longbows, naginata (glaive?), katana (bastard sword), wiashizi (short sword), dagger, and banded mail for the samurai. Primarily bows and spears (though cannons and handguns were used on occasion) for the others. Even in the sixteenth century, you had longbows for the English, crossbows for rest of Europe, pikes, halberds, and greatswords for the Landsknecht and Swiss; longsword, buckler, and breastplate for the spanish, etc. A few muskets showed up in the wars, but they hadn't entirely displaced the crossbow and longbow (as is demonstrated by the longbows on the wreck of the Mary Rose and Cortez's use of crossbowmen in the conquest of Mexico).
As for what weapons I've actually seen in play...
my characters:
Fighter/mage: Guisarme, longsword, light flail, composite longbow
Cleric: Greatsword, longsword, heavy mace
Ranger/Rogue: Composite longbow, rapier, light mace
halfling melee mutt: longsword, lance, shuriken
cleric: spells--he owns a crossbow and a club but hasn't used them for five levels or so
Fighter/barbarian: Bastard sword, javalin
Fighter: Heavy mace, heavy shield
Cleric/Rogue/Shadowbane Stalker: Lancea (Player's Guide to Arcanis), flintlock pistol, dagger
Fighter/Barbarian/PsyWar/Warmind: Tralian Hammer (Player's Guide to Arcanis), Composite Longbow
Cleric: spells (but she has a gladius (Player's guide to Arcanis) and a spear)
Paladin: Initially, halberd, now scimitar
Prospective character (replacing the paladin who became king of a kingdom and had to retire): werebear--long axe, claw, claw bite
That's actually a pretty good variety, come to think of it. I've seen nearly every other martial or exotic weapon used by other players except the light pick, light hammer, urgrosh and doublesword. The ones I see used the least are double weapons (I've only seen one dire flail and the character was later rebuilt so as not to use it), picks (again, I've seen one character with a heavy pick... but only one), and Ranseur (disarming isn't nearly as nice as tripping and those who don't care about manuevers pick the glaive for the extra damage). The one I've seen way too many of is the spiked chain. (See above for the cause of this).