For me, cohorts are all about story. They should always have a good reason for being there.
Some players will simply take cohort X to reload their crossbows or act as a meat shield while the PC escapes. That's a bit cheap.
Classy cohorts I've seen include:
A feeble gnomish rogue who attached himself to a savage half-orc barbarian PC and begged to be tutored in warfare.
"Disco Stu", half-elf bard who did "queer-eye" makeovers on the party's warrior-types, then refused to be associated with the gauche PCs whenever they encountered influential or stylish NPCs.
Leadfooted half-ogre determined to out-sneak the halfling rogue. Died young. Nearly TPKed everyone else.
Paladin's squire and caddy (bard). Stood at the back with a golf-bag full of cold steel, adamantine, magic, flaming, and mundane weapons. Otherwise, his role was to shout "good shot, my liege!", and "look out behind you, sir!". We made him roll a profession (squire) check to successfully proffer the right weapon at the right time. Out of combat, his task was to defend the paladin's armoury at all costs. Naturally, the DM couldn't resist having it stolen with the loyal squire bound, hand & foot, to the bag.
In short, cohorts are effective comic relief or need to have a justified commitment to their PC. If the cohort is just a battle robot, it probably shouldn't be included.