Historically, looking at videogames which undertook this kind of in-depth self-selecting surveying? Which is the only comparison point I'm aware of.
It's good for one thing - point specific pain points with specific abilities/spells/etc. that the people in charge of the game might not be aware of. Sometimes that's misleading, because it's a pain point only for the elite, or because the elite want to continue to abuse a broken ability, but more often than not, it's general issue, when we're talking about like the specific stuff. That's the upside of the "rate every ability of this class individually" thing - usually the ultra-hardcores will somewhat align with the playerbase in general on specific. You do have to be wary because there are exceptions, but generally.
What's it's very bad for, unfortunately, is the determining the general direction of a game, of classes, of larger issues in general, because the same exact people who are genuinely pretty good at picking out pain points, and fantastic axe-grinders with weird ideas that they often share with each other, but not with the bulk of the people playing the game. They're also ultra-conservative, typically, in a way that most players are not. You see this with MMOs that have undertaken these sort of in-depth self-selecting surveys - the information they get from them will often conflict with more straightforward surveys which reach a larger number of players, or surveying which isn't self-selecting but put out by the company.
Like, if you were running an MMO, and the main thing you wanted, was to retain the most serious raiders and PvPers, and you didn't care about the 95% who were the bulk of your paying customers, surveying like this and following their feedback. But that's focusing on 5% of the market at the cost of the rest. History suggests when you're a market leader, that will actually work for a while, whether you're EQ or WoW or whatever, but then it stops working.
A lot of companies did some fairly extensive surveying and research into what MMORPG players said they wanted (a lot of them with self-selecting surveys) back in the '00s, and this lead to quite a lot of failed MMORPGs. Not even ones that went F2P and lived - ones that outright died. I don't think the same will happen with a market leader like D&D but I also don't think anything good will happen from listening to ultra-nerds (which 100% includes me) about the direction of your game, and not listening to Jonny or Jenny who plays D&D but less seriously. Anyway, this kind of "What do people SAY they want?" stuff lead RPGs like Vanguard, WildStar, and a bunch of near-FFA PvP MMORPGs, all of which ended in tears. Because what the serious, in-depth, detailed-oriented, hmmmm how to put this... often neurodiverse (I say that as a seriously neurodiverse person) people say they want is NOT, absolutely NOT the same as "what people who play these games actually like". There's almost no finer evidence of that than FFXIV's success. It's basically done everything hardcore people said they didn't want in an MMORPG. WoW has also painfully and with difficulty moved away from listening to a tiny group of ultra-hardcore people, and they only really made the break when all the old guard had to leave or got fired in the sex abuse scandal and related fallout, and now we have Dragonflight, which is easily the best WoW expansion in over a decade - and it's primarily younger people, blue-haired people and so on, designing it - and designing it for people who actually play WoW!
This kind of attitude is part of the problem and how companies get into trouble with this kind of surveying.
Because conflating "enough knowledge" and "cares enough" is extremely dangerous, and assuming that he hardcore people have that knowledge, rather than just specific axes they grind, is dangerous too.
As I said, these kind of people can be useful for specific pain points and issues, but for general directions of classes? For how your game should look? For what people want in a broader sense? No. Absolutely not. These people are not reliably representative of the bulk of people playing. The most recent example is the apparent insistence that the entire arcane/divine/primal spell list concept be abandoned solely for the sake of Wizards. That is the sort of thing hardcore axe-grinders obsess about, and can't see the bigger picture about. One of the big things with MMORPG design and the like was designers realizing that, at some point, they have to be able ignore feedback and say "No players, actually you are wrong, and we are right".
A specific example of that would be WoW's obsession with high-end raiding. The feedback they were getting was that it was everything, and the devs actually were aligned with that feedback because they were from a similar background to the people giving it. Eventually, however, the CEO of the company had to step in, because WoW's own metrics (metrics sadly unavailable to TT RPGs) showed with the last high-end raid they'd released, what % of the playerbase had even seen the inside of it?
0.5%
And yet their design and class-balancing efforts for months had been focused on that raid and its environs. And the self-selecting feedback they had said they were right to be doing that.
Anyway, I could go on and on, but the point is, it's worth having that feedback for pain points and specific issues, but as a game designer, you need to able to say "Yeah that feedback says X, but what we actually need to do is Y". Frankly if WoW hadn't been able to do that, The Burning Crusade would have represented it's peak population, and it'd been F2P for years by now.
