It makes a huge cumulative difference.
Okay, I tweaked my simulation script so that it now calculates your average cumulative roll per day (basically, the average sum of all rolls), and so that it can optionally allow the die to increase by one size. That actually gets a better view on how much you get out of the d4/d6 split on rolls. As before, with the rules as per the UA you can expect 27 uses per day. Here's the stats:
As per the UA:
Attempting 27 uses per day, you can expect:
Median number of uses achieved: 27
Average number of uses: 21
Able to use the die on every attempt for the day 50.0009% of days.
Assuming all of those rolls are damage, you get: 72.4 extra damage per day.
Allowing the die to increase by one size:
Attempting 27 uses per day, you can expect:
Medium number of uses achieved: 27
Average number of uses: 25
Able to use the die on every attempt for the day 84.82% of days
Assuming all of those rolls are damage, you get: 100.7 extra damage per day
Remember, the expected damage per day from a battlemaster is 72. That puts the expected use as per the UA to be, on average, exactly the same as the battlemaster. With your update, you end up getting 28 extra damage - more than a third extra - over the day. That's a lot.
Not sure if your simulation showed this, but I was talking about it increasing up a step, then immediately decreasing when used the next time.
So, if you have a d6 and roll a 1, the next time you roll a d8 and then it goes back to a d6
no matter what you rolled.
I'm not sure if that is what you modeled in your simulation, but I certainly don't think that would add 28 extra damage, especially since it would require you to have rolled a 1 first (so those four extra uses would have had to come after rolling a 1 on the die)
If you did do it that way, I admit to being stunned, because that is a massive change that I did not expect, but reading what you wrote you had it go up to a d8 then stay a d8 until it rolled an 8 and downgraded that way. Which is not my suggestion.
On to a different discussion. Been looking at the psi die for the other classes, already discussed how poorly it works for the Soul Knife, but looking at the sorcerer... do they have enough for the psy die?
Level 1 gives them Psionic Discovery and Telepathic Speech which won't give them a lot, because it will rarely be used. Their Psychic Sorcery can be spammed with every spell and cantrip they cast (yeah, casting cantrips with no components, and only needing to roll a zero because that is an odd interaction) so that might actually be enough by itself to keep the die moving.
Level 6 gives them a damage boost, so rolling twice per spell potentially. I do hate that damage boost though, seems boring and it only hits one target, so no AOE boosts.
Mind over Body gives us a roll, but it won't get used much because it is very niche and it is a bit convoluted. Abilities equal to sorcerery points for hours equal to psi die roll. Now, it is 1 point for flight, or 1 point for See Invisibility, so it is cheap, but if you don't think you'll need it, you won't spend it.
Psychic Aura is good, rolls pretty often, has no recharge cost. Frankly, it might be the best ability in the UA at the moment.
So, Sorcerer is a maybe for me.
They have no ability that lowers their die automatically, but they also have very few times they need to use their die. If you don't care about damage, and you don't need to cast silently you will only roll the die to learn new spells (which means you need to know you need that spell) or creating a telepathic bond with someone. I think they don't use the die enough to really justify it, and they kind of have the opposite problem of the soul knife, who will rarely have a chance to increase their die, the sorcerer will be much more rarely decreasing their die.