There are of course numerous threads in these forums and the WotC forums about weapon balance, but as far as I know, nobody has actually tried to do a statistical analysis to correlate the different variables associated with each weapon with the damage done - and then to use that information to determine whether there are any weapons that are overpowered or underpowered.
My first objective was to estimate a formula that reproduces the existing weapons' damage values - in order to determine about what tradeoffs between different weapon features and higher damage values the designers thought to be balanced. What I did was I made a list of all the melee weapons in the game and coded each one according to the following criteria:
+3 Prof (P): The weapon has a +3 proficiency bonus rather than a +2.
Superior (S): The weapon is a superior weapon.
Two-Hand (T): The weapon is a two-handed weapon.
Light (L): The weapon is a weapon in the light blade category.
High Crit (H): The weapon is high crit.
Reach (R): The weapon is a reach weapon.
Additionally, I made the following changes to simplify the analysis and get more reliable results:
1. I did not include simple weapons in the analysis. I made this decision because many of the simple weapons are not balanced (e.g. the club is exactly the same as the mace except it has a lower damage) and this was throwing off some of the results. Since simple weapons are rarely used by PCs in combat (every class that does not have military weapon proficiency is a class that does not use melee attacks as its main attack powers) this does not normally unbalance the game.
2. I did not include either the versatile or off-hand properties in the analysis. This was because these keywords were extremely closely correlated with other properties (e.g. only one weapon is light but not off-hand, and none are off-hand but not light, and all but two weapons are exactly one of versatile, light, or two-handed), and this throws off the results of the analysis. Versatile should not have any relationship with damage if the weapons were balanced (because it is rare that a PC will actually want to switch between one handed and two handed weapon styles mid-combat), and off-hand is almost the same thing as light.
The results of the regression analysis gave a formula as follows:
Expected Damage = 5.52 - 1.22P + 1.03S + 1.52T - 0.86L - 0.87H - 1.74R
where the variables are each 1 if the weapon has that property and 0 if not.
Now in order to determine whether any weapons are overpowered (or underpowered) we need to look at the following two questions.
1. Are there any weapons that have damages significantly above or below the above formula?
2. Does the above formula actually represent balanced trade-offs for the given abilities? (For example if high crit were actually worth 2 points on the [W], then all high crit weapons would be way overpowered since they're only "paying" 0.8 points to get something worth 2. This turns out not to be the case, see below.)
The answer to question 1 is NO. It is easy to calculate the "residuals" - the amount of damage above or below that predicted by the formula - for each weapon. The highest is the greataxe with 0.34, and the lowest is the greatsword with -0.32.
The answer to question 2 is harder to figure out, but let's look at each one individually:
+3 Prof (P, -1.22): An additional +1 to the proficiency bonus means about a 10 percent increased chance of hitting (going from 50 to 55 percent), so a 1.22 point reduction in damage is balanced if the overall damage done before is around 12. At low levels, this is reasonable (say an average of 5 for the [W], then +4 for the stat bonus, +1 for magic, and possibly a little extra from feats or the power used). At higher levels damages will be higher, but the multiplier on the [W] gets higher, so it balances out.
Superior (S, 1.03): Using a superior rather than a normal weapon costs one feat, so we need to see if this feat is balanced. An appropriate point of comparison for feats which give extra damage is Weapon Focus, which gives one additional point of damage per tier. Getting a superior weapon gives about one additional point of damage per [W], which is roughly comparable.
Two-Hand (T, 1.52): The cost of using a two-handed weapon is that you can't use a shield in the other hand, thus eliminating a chance to raise your AC by 2 (assuming you had heavy shield proficiency.) Since attack bonuses are generally more important than defense bonuses (since you can choose where to target your attacks, while if you have a very high AC the opponents can often just attack someone else) a +1.5 per [W] to damage for 2 less AC seems like a reasonable trade-off.
Light (L, -0.86): The main reason it matters if a weapon is a light weapon is for use with the Rogue's sneak attack. Figuring out whether the given reduction in damage is the right amount would lead into a discussion of the balance of Rogue in general, which is beyond the scope of this post.
High Crit (H, -0.87): Assuming a 50% chance of hitting, 10% of hits will be criticals. This means that high crit, which adds +1[W] per tier to the critical damage, adds an average of 0.1[W] damage per tier to each hit. Since [W] is usually around 5, 0.1[W] is around 0.5, which means that high crit weapons are not usually a good choice at heroic tier because you are giving up almost a full point of damage (or even more if you're using a multiple [W] attack power) for something worth around half of that. On the other hand high crit weapons can become much more useful at paragon and epic tiers because they do more damage and you also have more ways to increase your critical range (e.g. with the epic weapon mastery feats.)
Reach (R, -1.74): The extra value of a reach weapon depends entirely upon positioning and tactics, and will be significantly different for each character and party, and thus would be impossible to calculate here. (For example defenders would have much less use for reach weapons as they are normally in the front lines and thus adjacent to the enemy anyways.)
-----
The overall conclusion of this analysis is that there is no weapon that is obviously overpowered or underpowered. The deviation from the computed formula is at most a few tenths of a point in either direction, and there would not even be a way to fine-tune it more as [W] values can be adjusted at best in 0.5 unit increments. The formula itself seems like it captures the relative game value of the different weapon effects reasonably accurately, although of course the exact values would depend on other aspects of the character and his battle strategy. So, overall, we have a system in which different weapons are useful for different characters, and no weapon is clearly more (or less) powerful in all situations.
My first objective was to estimate a formula that reproduces the existing weapons' damage values - in order to determine about what tradeoffs between different weapon features and higher damage values the designers thought to be balanced. What I did was I made a list of all the melee weapons in the game and coded each one according to the following criteria:
+3 Prof (P): The weapon has a +3 proficiency bonus rather than a +2.
Superior (S): The weapon is a superior weapon.
Two-Hand (T): The weapon is a two-handed weapon.
Light (L): The weapon is a weapon in the light blade category.
High Crit (H): The weapon is high crit.
Reach (R): The weapon is a reach weapon.
Additionally, I made the following changes to simplify the analysis and get more reliable results:
1. I did not include simple weapons in the analysis. I made this decision because many of the simple weapons are not balanced (e.g. the club is exactly the same as the mace except it has a lower damage) and this was throwing off some of the results. Since simple weapons are rarely used by PCs in combat (every class that does not have military weapon proficiency is a class that does not use melee attacks as its main attack powers) this does not normally unbalance the game.
2. I did not include either the versatile or off-hand properties in the analysis. This was because these keywords were extremely closely correlated with other properties (e.g. only one weapon is light but not off-hand, and none are off-hand but not light, and all but two weapons are exactly one of versatile, light, or two-handed), and this throws off the results of the analysis. Versatile should not have any relationship with damage if the weapons were balanced (because it is rare that a PC will actually want to switch between one handed and two handed weapon styles mid-combat), and off-hand is almost the same thing as light.
The results of the regression analysis gave a formula as follows:
Expected Damage = 5.52 - 1.22P + 1.03S + 1.52T - 0.86L - 0.87H - 1.74R
where the variables are each 1 if the weapon has that property and 0 if not.
Now in order to determine whether any weapons are overpowered (or underpowered) we need to look at the following two questions.
1. Are there any weapons that have damages significantly above or below the above formula?
2. Does the above formula actually represent balanced trade-offs for the given abilities? (For example if high crit were actually worth 2 points on the [W], then all high crit weapons would be way overpowered since they're only "paying" 0.8 points to get something worth 2. This turns out not to be the case, see below.)
The answer to question 1 is NO. It is easy to calculate the "residuals" - the amount of damage above or below that predicted by the formula - for each weapon. The highest is the greataxe with 0.34, and the lowest is the greatsword with -0.32.
The answer to question 2 is harder to figure out, but let's look at each one individually:
+3 Prof (P, -1.22): An additional +1 to the proficiency bonus means about a 10 percent increased chance of hitting (going from 50 to 55 percent), so a 1.22 point reduction in damage is balanced if the overall damage done before is around 12. At low levels, this is reasonable (say an average of 5 for the [W], then +4 for the stat bonus, +1 for magic, and possibly a little extra from feats or the power used). At higher levels damages will be higher, but the multiplier on the [W] gets higher, so it balances out.
Superior (S, 1.03): Using a superior rather than a normal weapon costs one feat, so we need to see if this feat is balanced. An appropriate point of comparison for feats which give extra damage is Weapon Focus, which gives one additional point of damage per tier. Getting a superior weapon gives about one additional point of damage per [W], which is roughly comparable.
Two-Hand (T, 1.52): The cost of using a two-handed weapon is that you can't use a shield in the other hand, thus eliminating a chance to raise your AC by 2 (assuming you had heavy shield proficiency.) Since attack bonuses are generally more important than defense bonuses (since you can choose where to target your attacks, while if you have a very high AC the opponents can often just attack someone else) a +1.5 per [W] to damage for 2 less AC seems like a reasonable trade-off.
Light (L, -0.86): The main reason it matters if a weapon is a light weapon is for use with the Rogue's sneak attack. Figuring out whether the given reduction in damage is the right amount would lead into a discussion of the balance of Rogue in general, which is beyond the scope of this post.
High Crit (H, -0.87): Assuming a 50% chance of hitting, 10% of hits will be criticals. This means that high crit, which adds +1[W] per tier to the critical damage, adds an average of 0.1[W] damage per tier to each hit. Since [W] is usually around 5, 0.1[W] is around 0.5, which means that high crit weapons are not usually a good choice at heroic tier because you are giving up almost a full point of damage (or even more if you're using a multiple [W] attack power) for something worth around half of that. On the other hand high crit weapons can become much more useful at paragon and epic tiers because they do more damage and you also have more ways to increase your critical range (e.g. with the epic weapon mastery feats.)
Reach (R, -1.74): The extra value of a reach weapon depends entirely upon positioning and tactics, and will be significantly different for each character and party, and thus would be impossible to calculate here. (For example defenders would have much less use for reach weapons as they are normally in the front lines and thus adjacent to the enemy anyways.)
-----
The overall conclusion of this analysis is that there is no weapon that is obviously overpowered or underpowered. The deviation from the computed formula is at most a few tenths of a point in either direction, and there would not even be a way to fine-tune it more as [W] values can be adjusted at best in 0.5 unit increments. The formula itself seems like it captures the relative game value of the different weapon effects reasonably accurately, although of course the exact values would depend on other aspects of the character and his battle strategy. So, overall, we have a system in which different weapons are useful for different characters, and no weapon is clearly more (or less) powerful in all situations.
