Errr... they won the battle, didn't they?
How are they supposed to know that they aren't supposed to fight when the battle is so close to thier abilities that it is winnable? Sure it is 4 CR's higher than thier effective level, but in my experience many a min/maxed party is effectively several levels higher than thier level would indicate. It is close to the limits of thier ability, but ought they not to suspect that you as a DM will test them right up to the limits of thier abilities? Isn't that standard practice? Isn't anything less considered Monte Hall?
If you want your players to flee combat, you have to do two things.
First, you have to draw clear distinctions between winnable fights and unwinnable fights. The above fight wasn't clearly unwinnable. If you really wanted them to flee you should have said, "You see a vast host of undead shambling toward you. There must be at least several score ghoulish figures, and unnatural incorporeal forms flit amongst them." Forty ghouls, twenty ghasts, and a dozen shadows would have convinced most parties of 6th level characters that its time to head for the hills/temple/nearest civilization. (And if they do win, well, they have a story to tell.) It seems to me that your encounter was chosen such that it wasn't quite overwhelming should the party choose to stay and fight, so you shouldn't complain that the party choise to stay and fight.
Second, you have to demonstrate to the party at the very beginning of the campaign that in this campaign they will occassionally be put into situations that they can't handle and will be expected to deal with those situations by some means other than combat. This is a very very important lesson to teach, IMHO. Starting out as first level characters, throw them into an encounter with something insanely overpowered. You as a DM know that you have fiated that no matter what the characters do, the creature(s) is in a fairly good mood and won't kill/eat the PC's. It is up to the PC's to realize ASAP that fighting the creature is insane, and using thier wits minimize the ill effects of encountering TBN (tm). For instance, this creature may want news or information, bribes, to rob the character, or simply to be left alone. Alternately, the creature isn't much into pursuing the PC's (or capable of long pursuit) and will tire after a short chase if the party makes a good go for it. Make the encounter absolutely overpowering so that the villain can toy with PC with no real danger of harm to himself.
I have in my time ran a party of first level characters against 100 goblin worg riders (Bandits wanting a bribe but not that interested in a fight. They subdue the characters with whips, lassos, and nets should the PC's fight, liberating them of thier best equipment. Otherwise, the PC's can negotiate for a minimal 'fee'.), a Elder Red Dragon (well fed on mastadon and mostly interested in news of the human world and recieving flattery from awed PC's. He subdues the PC's if they make a pathetic attempt at slaying him, promising to eat them, before forgetting and falling asleep. (An attempt to kill the sleeping dragon would have resulted in death, as the dragon was in no mood to be roused.)), and a group of werewolves (who cheerfully beat the party unconscious after the party attempted to ambush 'the bandits'). The latter lesson was particularly important because it tells the party that the best plan isn't always to shoot first and ask questions latter. This encourages role play, and allows you to actually get a chance to introduce your finely crafted NPC's to the party before they kill them.
Another example of this sort of play was a low level multi-solo (everyone is in the same world but party interaction is not critical) campaign I ran for while. One character wanted to play a thief. Early on, the thief had an encounter with a local street tough you accused the PC of 'holding out on him'. The PC fought, and the street tough (being 4 times his level), disarmed him and beat the crap out of him. Lesson learned, latter when the PC saw three members of a rival gang approaching, flight was considered the first option. In the course of this flight, the PC climbed a wall and jumped into a courtyard. When apprehended by the owner of the courtyard, murder was again not considered a first option. Instead, role play pursued, and it was learned that attempted murder would indeed been a very bad move - since the man who first grabbed his arm was a Prelate of a leading religion in the city.
Sometimes in order to achieve role play, you have to break out of the mold of 'murder or be murdered' that characterizes so many of the PC's actions with NPC's in D&D. If not all of your monsters consider violence with a heavily armed PC party thier first option, then your PC's will probably figure out that they have other options too. Consider at least how bad it usually turns out for creatures that start violence with heavily armed PC parties.
