Here's an interesting approach: give the party a couple of items that emulate divine feats or have unique abilities- but that require turning attempts to activate. Encourage them to burn through some of those turning attempts that way.
The Jester: Interesting idea, but all of them have now Divine Feats (and during a time the paladin had an armour that needed turning to get special abilities working but has lost it). As I have said, the number of turning attempts they get is too high to be severely affected by spending them on divine feats/activating items: All of them have Extra Turning, and all of them have high charisma:18 the first cleric of Lathander, 16 the second cleric and 22 (with enhancement) the paladin (who of course loves her Divine Might divine feat). Also, the first cleric has a Greater Turning each day and the second one has 2 attempts at Greater Turning each day.
At a total of 11+1Greater, 10+2Greater and 13 turnings each day they usually have enough to last the whole adventure, even after divine feats and all.
May I suggest that maybe is the greater turning and the prestige classes what need to be fixed? In any case (excuse me for not having read carefully your rules change) allow me to suggest to simply tie the turning success not to the undead HD, but CR. That should fix the problem with liches and vampires. Make greater turning a regular turning that also deals damage.
In any case, seems that the party is simply very strong in the anti-undead department. I would think on house ruling golems if I had a couple fighters with adamantine greatswords.
Someone: the Greater Turning as is written is the bane of any undead you can think of. I have already fixed the prestige classes by removing the 3+Cha Greater Turnings each day that the Radiant Servant had!!one extra at first level with another more each 4 days a lot more reasonable. I am trying now to fix Greater Turning (and Turning) by providing an alternate way to handle it. Instead of making it an encounter deciding ability I think that with the damage turning (and greater turning) will help in the same way that a fireball or other area spell helps against multiple opponents, but without making the encounter a sure-win situation.
1. No con bonus problem: Add the constuct hp bonus.
2. Make the turn skill: force the clercs to place skill points into a normally automattically scaling ability. Interesting consequence would be that all characters could take it and then you could expect the party to have the ability even though they dont have a cleric. Allowing for better designed encounters.
3. Make them use spell slots: treat the ability as a spontaneous cast of a cure spell level 1-9 if they are coughing up spell slots every time they want to do it they may use it more sparingly. I like this option.
4. Do the ravenloft thing and essential bump up all of their turn resistance. I assume that is what they do anyway in the 3 ed rules, I only have the 2 ed ravenloft.
5. Make them use spell slots: Again they are burning spell slots but they have to use their cure spells as touch attacks to damage them. So cure light wounds would do 1d8+5 etc.
6. Get rid of the turning ability altogether: The players will hate you. I dont suggest this one.
Sadrik:
1:Already considering that approach myself. Construct bonus hp would help.
2:Interesting idea. When I was trying to get at a system that I liked I toyed with the idea of making it dependant of Knowledge(Religion) checks and work with a table similar to the one for the Iajutsu-Focus from OA.
3:This is also interesting. Will think about it.
4: Making Turn Resistance go up is another thing I was toying with. Also, I thought on simply making Turn Resistance charisma dependant so each undead had an extra turn resistance equal to its Cha bonus. Will think about it too.
5: This is making the ability almost useless as they already can do so by spontaneously casting cure spells.
6: I don't want to die young
Remember, the first of the clerics has been working hard on improving that class ability... It would be too harsh to make it go away.
I have long attempted to do something similiar.
My questions:
You say as long as a creature has damage dealt, it counts as shaken. I assume you mean that if the undead takes damage it becomes shaken from the effects of the damage. How long does this shaken condition last?
My only real concern is you end up with three seperate rolls: the turning check, turning damage (or effect) and damage.
DreamChaser: Yes, the shaken result lasts as long as it continues to have damage that was inflicted by the turning. So that screws non-intelligent undead but not so the intelligent ones that will heal given time. They all will benefit greatly from having evil clerics around to heal them with negative energy (rebuking or inflict spells)
Having 3 separate rolls: can be an issue as it would slow things down a bit, but I don't care too much about that so it's not a problem to me. See it that way: in the end you are saving yourself a lot of dice rolling when the clerics blast the 30 zombies that are cannon fodder in the encounter with the BBEG.
Your system seems interesting, but I don't like too much it being based on ability damage. That needs a lot more book keeping than just damaging undead. I prefer Turning becoming an undead only fireball-type than having to record which undead has its scores diminished and when it will recover and so on. But interesting anyways
Thanks you all for all the feedback, keep them coming please!