Benchmarking

[Nifft]

We have the Iconic 4 party to test adventures out.

Now, IMHO, we need an Iconic Challenge Suite to test out classes and/or party combos. Here's my initial proposed set of situations. They're based on tactical concerns more than specific, exact CR concerns, and are driven by available tactics.

1: Goblins!
Terrain: Forest -- trees, brush, clearings
Foes: A hoard of goblins (Specifics: 20% Worg mounted, 30% Rogue snipers, 40% Warriors, 10% Spellcasters (Evil Cleric or Wizard))
Tactics: Mobility, stealth, rush-as-hoard, "aid another" action to help few high-level commanders.
Variants: Kobolds in a trapped dungeon; halfling thieves in an urban slum.

2: Necromancer!
Terrain: Dungeon -- traps, slime, rubble
Foes: One living spellcacster and his many undead minions (Specifics: relatively high-level Evil Wizard or Cleric, many skeletons/zombies, and a few "cool" undead (shadows, ghouls, a morgh or a mummy))
Tactics: Ambush and spellcasting without regard for minions
Variant: Necromancer in a graveyard (above ground), or in a wasteland (desert, bleak mountains, etc.)

3: Dragon!
Terrain: A big cave that's full of tricks & traps
Foes: A dragon & cohorts (Specifics: 1-2 Dragons and some Giants, Magical Beasts, Aberrations or Monstrous Humanoids)
Tactics: All good Dragon tactics, especially including flight.
Variants: In an open field; above a city.

4: Demonologist!
Terrain: A Wizard's Tower (magical traps and summoned minions)
Foes: Summoner + Cohort (Blackguard) + Bound Minions (specifics: 1 high level Wizard/Sorcerer or Evil Cleric, plus many Outsiders, plus some Constructs)
Tactics: Spell-Like Abilities, special attacks, etc.


Obviously, these each reward different skill-sets, and are a better test than simply saying, 'well PC1 could kill PC2', since what PCs mostly do is kill monsters and not each other.

So, I'm looking for terrain/foe combinations that, in as small a unit as possible, reward ALL Core PC choices, and many Core PC party combos.

This suite will be used to settle the age-old "Fighter/PsiWar" debate, and the "Cleric/Druid" debate, and every other debate out there.

So help me, please!

-- N

 

[Elder-Basilisk]

Since a test suite exists to compare characters, there are several challenges here:

1. Accounting for the actions of other party members. Since this would test the way characters work as a team, it should properly test them as a team. However, this requires taking account of their actions.
2. Narrowing these situations down to a few specific tactics. If we have a group of NPCs and run them against the PCs with playtesters, that's fine though the results would reflect the playtesters' skill level and their experience with particular types of characters. On the other hand, for a useful online forum benchmark, it would need to be narrowed down.
3. Accounting for the different tactics a character might adopt. For instance, in the PsyWar/Fighter thread, Scion's PsyWar wants to buff in round one and charge in round two. A fighter might well want to delay until the foe charges him and then full attack in round 1.
4. (This may well be the most difficult) A way to account for things like high initiative.

So, I think the kind of thing that a benchmark like this would need would be a set of tactical situations drawn from the encounters above. For instance in the dragon flight benchmark, you might have the following situations:

(These examples assume a CR 20 dragon; the dragon's CR should probably be party level+3)
1. The dragon flies overhead and breathes. How well do you avoid the damage? How likely are you to make the save? How many hit points do you have left over on a failed save?
2. The dragon uses the hover feat to stay 20 feet above the uppermost portion of your character and bites at you. How do you respond to this tactic.
3. The dragon uses the snatch feat to grab you with its bite. How do you resist?
3b. If you don't get out before the next round, you get no save against its breath weapon, can you get out.
4. The dragon casts anti-magic shell and then lands on you, making a crush attack. How well do you avoid the attack and how likely are you to be pinned?
5. The dragon lands next to you and other party members and is ready to execute a full attack next round (and keep the mage and cleric in its threatened area). How much damage can you do in your full attack?
5b. You had a bow out because until this round, the dragon had refused to close, how much damage can you do? (Can the dragon afford to ignore you while he tears the rest of the party apart).
6. You've got the dragon's attention. It full attacks you making optimum use of its power attack feat. How much damage do you take?
7. The dragon has retreated. How much damage can you do and how much damage do you take (from the AoO)?
7a. There is a pillar between you and the dragon so charges are prohibited. How much do you do now?
7b. The dragon is sixty feet away from you.
7c. The dragon is sixty feet away from you and you can't charge.
8. The dragon cast tenser's transformation when it retreated, now it has beaten you silly. You're at 10% of your hit points. The cleric's initiative immediately follows yours. Your distance from the dragon is [your reach]-5 feet. The cleric is sixty feet away and has a move of 20. He has a heal spell prepped if you can get within his range and can channel that to a mass cure moderate if you can't. Assuming the dragon will attempt to finish you off on its initiative which is right after the cleric's, what are your odds of surviving?
9. The dragon hits you with a repeated, targeted greater dispelling. How do you do in situations 1-8?

Of course, the challenge is that for some situations (like 5b), the answer will sometimes be: "I don't get into that situation; I cast fly from my ring of spellstoring and closed with the dragon." Fine. That's a good plan and it will work unless you are dispelled, etc. However, it's rather difficult to give a numerical evaluation to the plausibility of such claims. It's also difficult to give them a numerical weight in comparison to the other claims.

 

[Nifft]

I agree. I think it's a complex task, and even though Dragons are perhaps the ultimate test of a party's combat abilities, we should probably start at the low end (Goblins!).

Basically, I want the smallest unit that rewards all Core party abilities -- from Cleave to Tumble/Flank/Sneak attack, and of course all spellcasting.

I do very much like your DragonScript(tm) program.

-- N