My houserule to make it take a little more effort to subdue/knockout a foe instead of killing them is this: when the PC hits for enough damage to kill a foe, they can either take the kill, or roll a second attack vs the target's Fortitude defense. If this attack succeeds as well, the target is KO'd; if not, he suffers no damage (or half, or whatever, so long as he still as a positive HP total, he can still fight). This way, it's a gamble the PC may not wish to take, and it represents how much harder it is to subdue physically tough foes (instead of, say, low-fort casters). Is this what you were looking for?