Our group has playtested for Malhavoc Press ever since Malhavoc started using external playtesters.
Some do's and don'ts:
Don't simply state such and such didn't work (ie. Rule XXX is broken.) State why your group believes the mechanic doesn't work. Support your assertions by comparing to existing rules if possible and whatever empirical evidence you can.
Don't simply diss rules because that is how your group or you would or would not 'do it'. The publisher wants you to evaluate and validate 'their' rule, mechanic, etc. So, don't fall into the trap of thinking of how 'you' would do it. Does their ruling make sense? Is it balanced in the framework of the game system? Is it too simple or too complex? That sort of thing.
Avoid overused and usually meaningless proses. Don't use the words "broken", "sucks" and other such very generalized terms. If a mechanic has a problem, use more concise words to focus the publisher in on exactly what the problem is.
Understand what 'Balanced' is. This one will be a hard one and difficult to define, because 'balanced' in many cases is subjective. Look long and hard at the mechanic in question and analyse it from a variety of situations or contexts. For example, you may be playtesting a prestige class and your group deems that the prestige class is too much over the top in combat. Or is it? The prestige class might be a melee monster but what about ranged combat or spell combat? And while combat is a big part of the game, it is not everything. Does this same prestige class have the same appeal when there is no combat at all? These are the sort of questions one asks yourself when looking at if something is 'balanced'.
When considering 'balance', most publishers want you to only consider their material in relation to the core books. But ask if they want you to consider other publications. There is hundreds of D20 products out there and is impossible for you to consider all of them in your deliberations of 'balance' and possible synergies. I would go so far as to say that some issues discussed on-line of something being 'broken' usually is the case of Publisher 'A's prestige class being used in conjunction with Publisher 'B's feats being use in conjunction with..... you get the point.
Do keep notes of each play session and the salient points. The final playtest document may be due in months, so notes keeps your observations handy for the final writeup.
Make a playtest plan - playtesting is more than just 'using' the playtest material in your campaign. This is what our group does:
Each member reads over the material and makes notes on potential problem areas. Then we get together and discuss the material in depth to make sure we are all understanding the material and the intrepretations.
If it is alot of material, prioritize what you want to playtest, if time allows. Trying out dozens at new rules in the first playtest session will be daunting so break it down in logical blocks and playtest each block. In the final phases of the playtest, bring all the parts together.
We then use the material in-game. We might do a combat to test new feats, classes or prestige classes. If we notice something quirky or odd, we will replay the combat. Sometimes multiple times. Note any observations and carry on. Our group fully integrated new material into the campaign and run characters with new feats, classes, etc as normal characters. In this fashion, we get the best gauge of the 'balance' of the material since we are using the playtest characters in all aspects of the campaign.
Check with the publisher. Some want their material playtested and that it all. Others are open to suggestions for improvement if the material is problematic. If the latter, ensure that you genuinely have a problem with material and that you are not using the playtest to 'get how you would do XXX if you were writing the rules' shoved upon the publisher. If the publisher encourages suggestions, make any suggestion in regards to 'their' material - don't completely write their mechanic as you would do it.
Damn.... probably too long. But I hit the main points. Enjoy the playtest experience.
).
