Originally posted by Tempest_Stormwind:
All right, numbers geeks. Here's the R script I promised. (It's deliberately brute-force rather than elegant, just so people can actually follow what I'm doing. And yes it's heavily commented.)
For R nerds only.
[sblock][UNKNOWN=pre]: scrolls<-numeric()#Number of scrolls found.for(i in 1:100000){#Over this many campaigns loot<-0 #Scrolls found this campaign for(j in 1:7){#Rolls on CR 0-4 if(sample(c(1:100),1)<=24){#Chance of Table A result for(k in 1:sample(c(1:6),1)){#Roll 1d6 on Table A if(sample(c(1:100),1)<=20){#Chance of level-1 scroll on A loot<-loot+1 } } } } for(j in 1:18){#Ditto for CR 5-10 if(sample(c(1:100),1)<=16){ for(k in 1:sample(c(1:6),1)){ if(sample(c(1:100),1)<=20){ loot<-loot+1 } } } } for(j in 1:12){#Ditto for 11-16 if(sample(c(1:100),1)<=14){ for(k in 1:sample(c(1:4),1)){#Here it's d4 rolls on A if(sample(c(1:100),1)<=20){ loot<-loot+1 } } } } scrolls<-c(scrolls,loot)}#Do any of the results have the needed 72?mean(scrolls>=72)#Didn't think so. How many did we get?max(scrolls)#Bootstrap a 95% CI on the average number of scrolls.boot.scroll<-numeric()for(i in 1:1000){ this.scroll<-sample(scrolls,replace=T) boot.scroll<-c(boot.scroll,mean(this.scroll))}quantile(boot.scroll, c(0.025,0.5,0.975))#95% of the time, you'll get 1st level scrolls between those numbers#What about the overall distribution?#(I bet upthread it's Poisson.)hist(scrolls,probability=TRUE)#That sure looks like a poisson distribution to me!library(MASS)fitdistr(scrolls,"Poisson")#That's a pretty low standard error for Lambda=4.xcoord<-c(1:max(scrolls))ycoord<-dpois(xcoord,lambda=4)lines(xcoord,ycoord,lwd=2)#Decent fit. Let's quantify.#The standard for count data is the chisquare:chisq.test(scrolls)#But if I wanted to get fancy I can do Kolmogorov-Smirnov:ks.test(scrolls,rpois(length(scrolls),4))#Either way, that's phenomenally low p.
[/sblock]This code generates 100,000 sample campaigns, which (on the d% scale) is
far more than enough to be representative of all of them. For each campaign, it rolls the suggested number of times on each table, and if the result is a 1st-level scroll, it adds it to the pile. I get a list of numbers showing how many scrolls each campaign sees.
Recall that, in order to know "all the spells", you need to get at least 72 1st-level scrolls, and all of those have to be unique. (The actual odds of this happening are quite a bit lower.) The rest of this post assumes that every scroll you get is unique.
I start out by checking how frequently we saw the 72-scroll result:
0/100,000. No surprises there, I told you it's virtually impossible even if you assume every scroll is unique.
So let's take a look at how many scrolls we actually got. 95% all campaigns got fewer than 8 scrolls. The absolute maximum that I saw was 19, which showed up once out of 100,000. I
bootstrapped a 95%
confidence interval on the mean, and found
(4.016 4.046), with a median result of 4.032. (We've seen that number before!). This means that I can say with 95% confidence that across any given set of campaigns, people will see an average of 4 scrolls - or, more apropos of our discussion, will not average more than 4 scrolls.
Now, that's still an average over a set of campaigns. To figure out how frequent each
individual result is within an
individual campaign, it's worth plotting the thing. When I did that, I noticed that it was pretty similar (visually) to a
Poisson distribution, which isn't exactly surprising. So I fit a Poisson distribution to that curve, found that it was
damn close to λ=4, and ran
chi-square and
Kolmogorov-Smirnov tests on the result to see how well the fit was in fact. In both cases, the
p-values were on the order of 0.0000000000000002. A reminder that, by convention, p-values less than 0.05 are considered "significant", which in this case means that
the distribution is, in all likelihood, Poisson-distributed with λ=4. (Well, in truth, it'll be λ=4.032, as that's the expected value we determined upthread. But that's close enough.)
(If you like pretty pictures,
here's the graph. You'll note right out that most results are clustered around 4 (about a 15% chance of that specific result.))
So, the question arises: What's the
actual chance of a Poisson-distributed (with λ=4) process returning 72 or higher? Well, thankfully,
that's something we can compute exactly, since we know the Poisson distribution's equations. The
probability of getting at least 72 scrolls is 3.86*10^-64 or about 1 chance in 10^145, which is
amazingly low.
(For something more familiar, the odds of getting straight 18s on 3d6 straight down are 1 in 10^14, which, for visualization purposes, is about one yard out of the
orbit of the outer planets. And this is 48
orders of magnitude higher than that!)
Note that I was assuming 35 schema in my balance sweeps, up from 24 base. Since we now know the distribution from which they're pulled, we can determine that the odds of getting the remaining 13 from random drops of 1st level scrolls (the most common one by far) are therefore 1 in 7*10^15, which is about one yard out of a
light-year - or about four paces relative to the distance to the nearest
star. (Note: This is not a comprehensive answer, since realistically you'll get higher-level scroll drops as well, but I'm not about to repeat this analysis for you unless it becomes clear that this is actually being read instead of being dismissed out of hand.)
(EDIT: Although, assuming the other levels coincide with Poisson distributions with λ=their expected values (which is very likely to be true), an interested party could compute the probablity of getting any number of scrolls of any given level - in other words, exactly the data we'd need to determine how likely worst-case scenarios are. This is
good, hard data, if it holds. A far cry from "unbalanceable", I'd say!)
EDIT2: Okay, I went ahead and did it because this is actually really simple once you know it's a Poisson distribution. Here are my results from 10,000 sample campaigns, using the expected values determined above for λ (as per the definition of a Poisson distribution).
Show
[sblock]First, summary data on the number of scrolls of each level found in this simulation.
[UNKNOWN=pre]: first second third fourth fifth sixth seventh Min. : 0.000 Min. :0.000 Min. :0.0000 Min. :0.000 Min. :0.000 Min. :0.000 Min. :0.00 1st Qu.: 3.000 1st Qu.:1.000 1st Qu.:0.0000 1st Qu.:1.000 1st Qu.:0.000 1st Qu.:1.000 1st Qu.:1.00 Median : 4.000 Median :1.000 Median :0.0000 Median :1.000 Median :1.000 Median :1.000 Median :2.00 Mean : 4.004 Mean :1.653 Mean :0.6804 Mean :1.381 Mean :1.189 Mean :1.489 Mean :1.71 3rd Qu.: 5.000 3rd Qu.:2.000 3rd Qu.:1.0000 3rd Qu.:2.000 3rd Qu.:2.000 3rd Qu.:2.000 3rd Qu.:2.00 Max. :14.000 Max. :8.000 Max. :5.0000 Max. :8.000 Max. :7.000 Max. :7.000 Max. :8.000
Second, bootstrapped 95% confidence intervals on the mean (that is, I can say with 95% confidence that, over all games played, the average number of scrolls found will be between the numbers reported (this is to show tha the extreme values found at the "max" above are
not likely):
[UNKNOWN=pre]: first: 3.97 4.04second: 1.63 1.68third: 0.66 0.70fourth: 1.36 1.4fifth: 1.17 1.21sixth: 1.47 1.51seventh: 1.68 1.74
Finally, the upper bounds of bootstrapped 95% confidence intervals on the 95th percentile (i.e. 95% of all games will be lower than this), along with the approximate odds of getting that many scrolls or more:
[UNKNOWN=pre]: first: 14, 1 in 45850second: 8, 1 in 16754third: 5, 1 in 12771fourth: 8, 1 in 65341fifth: 7, 1 in 32564sixth: 7, 1 in 6337seventh: 8, 1 in 12827
tl;dr: Absolute worst case scenario: 81 spells known. Likelihood of this happening: about 1 in 1,000,000,000,000,000,000,000,000,000,000. For comparison, the odds of winning the grand prize jackpot in Powerball are 1 in 175,223,510.
(This assumes conditional independence, which isn't
quite true since some of the scrolls show up on the same loot rolls, but it's a good first-order approximation and is sufficiently small as to
never actually happen. The impact of violating conditional independence is also
much smaller than the impact of assuming that every single scroll you get is unique.)
Most likely scenario (with 95% confidence): 32.75 schema, assuming you do not get a single duplicate. Because there's a chance of duplicates in reality, the most likely scenario is actually lower than this.
(Including the spells in Elemental Evil, and assuming that every spell of a given level is equally likely, the probability of at least one duplicate within any single spell level is 13%. (I made a mistake earlier and computed the odds of
no duplicates, but forgot to subtract it from 1. This 13%
is accurate, when selecting the expected number of scrolls from the pool of all published spells.))
What I've been balancing with: 35 schema as an upper limit, with 24 as a lower limit.
*booyah*
[/sblock]
So, yes, I am
quite satisfied that, given the assumptions with which the game is balanced, my artificer will not break the game by having an expandable spellbook. DMs who deviate from the assumptions do so at their own peril and understand the risks - it's not up to the game designer to foresee every single possible houserule and plug it, particularly in 5e (which assumes the DM isn't a mindless automaton and will be able to make common-sense adjustments if something goes south). Rather, it's up to us to provide people a well-balanced toolkit within a standard set of assumptions - assumptions that are presented in the Dungeon Master's Guide.
Now, can we please discuss some of the rest of the class?
