With ogres as the backbone, you could use a face monster- something more cunning and articulate than the ogres, but weaker. And smaller, definitely Small or Medium. The face might be a clever goblin (with a few minions in tow) but I'm thinking something a bit bigger...
How about some bugbears? Hulking brutes from the standpoint of most PCs, but little guys when put next to the ogres, available in the right level range, and noted for their sneakiness.
Say you have one Daj Skullboiler, a quick-witted Bugbear Skinner (Skirmisher 7, Int 10, Cha 16) who came out the loser in a tribal power struggle but lived to tell the tale. He brought along a few followers, their mates and their children, and headed for the ledges and chasms of this area, where he had a few ogre acquaintances.
Daj sucks up to the ogre chief and provides several useful services.
1. His scouts cover a lot of ground, leaving the ogres free to lounge around when there's no prey in the offing.
2. Daj has a keen eye for strength and weakness; if a caravan or party looks too tough for the ogres, he'll make the call and prevent the wrong kind of bloodbath.
3. When there's enough warning (switchback trails and silhouettes on distant ridges are very useful for this), Daj can set up a good "toll" situation, where he eloquently offers the passers-by a chance to move along with their lives intact... Daj plays "good cop" while the ogres play "bad cop" by just standing there looking like they always do.
4. Sometimes, a lone traveler or small party is an easy target for the bugbears alone. Daj has greatly impressed the ogre chief by turning over the proceeds of several such encounters without being asked. (He only does this about one time in four, or if he suspects an ogre saw his thugs moving out.)
5. And here's the big one- the bugbears have some lickspittles of their own, a pack of goblin slaves who know how to make ale and wine. The ogres love the stuff- especially since they have more free time to laze around with the bugbears doing most of the hunting. Ale is fairly easy but for good wine they need to get grapes; when the PCs deal with the ogres, the chief could be away with some of his warriors and goblins to steal some, setting up a future encounter with a few elites and a little narrative weight.
I guess this is turning into a whole plot seed. Sorry about that. Anyway, Daj would have a Plan, of course. He's building up a heap of goodwill and getting the ogres to see him as indispensable. When the time is right he'll either go back to his tribe with his accumulated raided wealth and impress everyone into naming him the new boss, or if that seems unlikely he'll show up with some ogres and (in his words, with a melodramatic sigh) "Perpetuate the cycle of violence that makes a mockery of any peaceful goblin heart."
I find I'm liking this guy more and more. I'm going to have to use him myself at some point. Maybe give him a signature weapon (harpoon-like shortspear attached to a grappling hook on a fifty-foot line that he uses to cross chasms and scale cliffs and pull people off said cliffs), make him an Elite, and set up his wife as a shaman-type (level up and bugbear-ize a Hobgoblin Warcaster) and give her a few rituals... give them a bickering but loving relationship that becomes apparent during interaction, throw in opportunities for the PCs to help the bugbears take over their old tribe because Daj clearly understands the value of negotiation over murder...
...but that's probably a lot more than you were looking for. I do go on.
(Oh! His whole group can be equipped with the spear-hook-rope things, using them to rappel or scramble into ambush positions, and he can get a "didn't find the body" death if he goes off one of the cliffs in battle- the PCs later find not his corpse but his grappling hook, jammed into a crevice fifty feet below the ledge they were fighting on, the rope frayed but not entirely snapped off, the spear swinging in the wind just above another, smaller ledge...)