I think it is just a matter of how the characters are built. For example, here is a fairly simple Rogue build:
Rogue:
A rogue with a reach weapon (generally one of the good tactics for a rogue) can take something like Exotic Weapon: Great Spear (or Heavy Polearm), both from Complete Warrior. Both have reach, and do 2d6 damage. If human, they can take improved initiative as well. So flank the Ogre, or win initiative and attack the flat footed Ogre, or attack after your spellcaster casts Grease on the floor the Ogre is standing on (Ogres do not have 5 ranks in balance, so they are balancing while on the grease, and therefore lose their dex bonus to AC and might fall prone) and you are doing 3d6 damage on a hit, plus strength (probably about a +1). So Average damage 11.5. An Ogre has 29 hit points, so you've reduced 1/3 of it's hit points in one blow, and you are still 10' away.
It shouldn't be that difficult for a Fighter to do similar damage. And between the two, with the arcane spellcaster and and divine spellcaster buffing them and healing them and doing battlefield control (or direct damage at the end if the Ogre just needs one magic missle to go down), I'd think the party should be able to take the Ogre down in 2-3 rounds.
The battle will expend the spellcasters resources, and a lot of hit points, and will likely see the rogue going down before the end. But, it should be a survivable while tough encounter.