This is the ruleset I wrote up and put in Tournaments, Fairs, & Taverns for special duels. You might want to take a look at it. In both your way and mine, it's the defender that decides to back up, but in this version, the defender is avoiding an attack, instead of the attacker trying to position his foe.
Marks
Once per round, as a free action, a character can declare another creature as a ‘mark,’ meaning that the character is devoting almost all his attention toward the chosen target. This lets the character react more quickly to his mark’s actions, but puts him at a disadvantage against other creatures around him. In most sports, this is not much of a problem, but in a large melee it can be a risky tactic. A character can choose to stop marking any character as a free action, returning to normal combat rules. However, you can only choose to mark or stop marking a creature on your own turn.
Creatures with no Intelligence score, or with an Intelligence score of less than 3, cannot mark their opponents.
Penalties of Marking: While marking a target, you are considered flat-footed to all characters other than the marked target, and thus lose your Dexterity bonus to AC (if any) and cannot make attacks of opportunity except against your mark. Additionally, you cannot target any actions against anyone other than yourself or your mark. You can still shout to others out of your line of sight, and if by attacking your mark you are able to Cleave against other foes, you can do so, but you suffer a –8 penalty to your attack rolls against creatures other than your mark. And of course your area of effect spells might catch others, but unless you succeed a Spellcraft check (DC 15 + spell level), you must target your spell so that your mark is as close to the center of the area of effect as possible. Finally, you suffer a –4 penalty to Spot checks not in the same 90-degree line of sight arc as your mark.
If you have Uncanny Dodge, you may retain your Dexterity bonus to AC, but you still suffer the above penalties and restrictions to your actions. If you have the Combat Reflexes feat, you can make attacks of opportunity against anyone who would normally incur them, and the penalty to your attack rolls is only –4 instead of –8, but you are still considered flat-footed. If you have Whirlwind Attack, you can choose to make a whirlwind attack even if your mark isn’t in range, but you still suffer the –8 penalty to your attack roll. As an example of this, imagine a hero trying to pursue the main villain. The villain sends a swarm of his henchmen to try to tackle the hero, but the hero shoves them off or cuts them down, and then continues to chase the villain, paying little heed to the cannon fodder.
Benefits of Marking: In exchange for these penalties, you can react very quickly to your mark’s actions. Once per round, you may do one of the following things as a reaction to something your mark does:
• Move: Take a move-equivalent action, such as moving your base speed or drawing a weapon. On your next turn, you are treated as if you have already taken a move-equivalent action, thus reducing how much you will be able to do.
• Counter: Take a partial action, such as attacking, tripping, making a partial charge, or casting a spell. On your next turn, you are treated as if you have already taken a move-equivalent action, thus reducing how much you will be able to do.
• Lunge: Move 5 ft. toward your mark and make an attack. By the core rules, you normally must move at least 10 ft. to charge, even a partial charge, but sometimes your opponent is only 5 ft. away. This reaction lets you close the distance and make an attack. You suffer a –2 penalty to your AC for 1 round, but you do not gain any bonuses to your attack, since you have not moved far enough to build up momentum. If you have already charged once this round, this AC penalty stacks. On your next turn, you are treated as if you have already taken a move-equivalent action, thus reducing how much you will be able to do.
• Dodge: Move 5-ft. in any direction. This reaction actually takes place before your mark’s action. If your mark tries to attack, you can back out of reach or sidestep, and if he casts a lightning bolt at you, you might manage to jump out of the path of the bolt. You incur no Attack of Opportunity from your mark for this movement, but other foes can still make Attacks of Opportunity against you if you move through their threatened areas. On your next turn, you are treated as if you have already taken a move-equivalent action, thus reducing how much you will be able to do. You must declare that you intend to dodge before any attack rolls, damage rolls, or saving throws have been made for the attack you are dodging.
Timing of Reactions: A reaction takes effectively ‘no time,’ as far as initiative is concerned. You always resolve a reaction before anyone else can make another action (the exception to this is when multiple people have marked each other or the same target, but this is detailed below).
For Move, Counter, or Lunge reactions, the reaction takes place just slightly after your mark’s action. Thus, if your mark attacks you, you cannot use a these reactions to avoid being hit. However, you can use a Dodge reaction. The drawback of dodging is that you basically give up half your action next round to move only 5 feet now, but it does fairly accurately reflect how a nimble person can keep out of harm’s way by staying on the move.
Repeated Dodge reactions can be somewhat abusive, but this tactic is easily defeated. First, by repeatedly dodging, the character is giving up his move actions each round, so he probably will not be able to harm his opponent. Second, two creatures working in tandem can surround or outmaneuver the dodging character, since he can only dodge one of them. Finally, the easiest way to defeat this tactic is to choose the dodging character as your own mark. Whenever he dodges, you can lunge after him. Once you get within melee reach once, it should be easy to keep up with him however he dodges.
Also, certain attacks will hit regardless of where your foe tries to dodge. He might be able to leap out of the way of a lightning bolt, but a fireball will probably hit him, and if you have a spiked chain or a missile weapon and have multiple attacks, you shouldn’t have a hard time hitting him. Lastly, with the appropriate terrain, you can eventually back him into a corner so he’ll not be able to dodge anymore.
Multiple Marks: These rules may become somewhat confusing if more than one person has marked the same target. In these instances, if multiple people want to react to the same action, just go in the normal initiative order when determining who gets to act when. Since only characters who have Combat Reflexes can make Attacks of Opportunity while marking a target, this usually prevents the battlefield from devolving into a hideous mish-mash of Attacks of Opportunity.
Cross-Marking: To the above rules, if two characters have marked each other (such as in a duel), you can indeed make a reaction to a reaction. Thus, Allar might start a full attack, so Babb backs off 30 ft. as a reaction, and Allar pursues the same 30 ft. From an initiative perspective, Allar attacks Babb several times, then Babb backs off and Allar pursues. From a storytelling perspective, however, Allar takes his full attacks at a run as Babb backs off, so that their blades connect continually as they move across the battlefield. As an example, imagine the classic Hollywood swordfight, as two duelists push each other back and forth in the vicious to-and-fro tide of battle. Normal D20 combat rules do not handle such duels very well.
Abstract Marking: You may choose to mark an object rather than a person, so that you can react to whoever has that object, even if it exchanges ownership. Thus, you might mark a football, so that if the ball is passed to a different player, the person you’re marking shifts to the new person. For simplicity’s sake, for the few brief moments that a marked object is in transit, you are considered to still be marking the original target. This is rarely, if ever, an issue however.
Example One: Kerrith and Jeane, two master duelists, square off in a line fencing duel (see page xx), both armed with rapiers. This is a duel to the death, and if one of them commits a foul, the other is allowed one free attack. Since neither begins the fight flat-footed, they both set each other as marks, and then the judge gives them the signal to start.
Kerrith wins initiative, and since duels always start off with a surprise round, he advances 10 ft. in a partial charge to attack. Jeane takes a Dodge reaction, moving back 5 ft. out of range, ruining Kerrith’s attack. Kerrith decides to continue on, however, and takes a Lunge reaction to Jeane’s dodge. Until his next round, Kerrith has a –4 penalty to his AC because of the partial charge and lunge. To make matters worse, his attack misses.
Because of their reactions, on their next turns, Kerrith will act as if he has already taken a partial action, and Jeane will act as if she has already taken a move-equivalent action. Jeane still has a partial action this round, however, so she attacks, easily scoring a strong hit because Kerrith is off-balance.
On the beginning of the second round, Kerrith has to regain his footing, so he only has a partial action. He wants to try to force Jeane off the end of the fencing strip, which would constitute a foul, so he attacks, but Jeane decides not to dodge. He hits her, but only does minimal damage. Jeane is feeling confident, so on her turn she makes a full attack against him, hitting him twice, doing serious damage.
In the third round, Kerrith nervously attacks and hits weakly, then backs off 15 feet. Jeane responds with a Counter, a partial charge to catch up with him, and she hits again. Then, on Jeane’s own turn, she makes another attack, hitting. Kerrith is scraped and sliced, almost ready to fall down, but he courageously makes a full attack, scoring a critical hit and a hit, and then he makes a 5-ft. adjustment backward. On Jeane’s own turn, she takes a 5-ft. step forward and makes a full attack. Kerrith desperately Dodges back 5 ft., ruining her attacks, but as a reaction, Jean Lunges and lands a killing blow.
