D&D General Todd Kenreck Let Go from WotC

[Ulorian - Agent of Chaos]

When Staffan admitted he wasn’t too familiar with academic publishing it seemed clear this wasn’t a conversation of two experts but an expert and someone more familiar than you but still a layperson. You might not have intended this impression but your response to this to ask them be more persuasive and “try again” when they admitted reaching their limit of knowledge is what came off as demanding.


To be clear when we say scientists pay publication fees this almost never is their personal funds (I say almost since I think I remember hearing about some who did cover costs as a political point early on when concepts were first being put forward but currently it would be very unusual). Generally this would come out of grant funds but that would mean a trade off for money that otherwise would go into research. And mandates for OA publishing haven’t kept up with corresponding increase in grant funding. In our department we have had people request publishing support from department budget that would also go to things like conferences or training workshops or there’s smaller funding opportunities the university offers that could offset publication fees (this isn’t what they were created for but is the main use in chemistry where the total grant might cover a few weeks worth of chemicals) Ideally I’d like to see grants cover publishing better too, but in practice there’s a few issues. There’s a range of fees in different journals even in related fields so setting the amount may be tricky. Doing something like saying funding supplement would be given to match publication fees can run into problems with incentivizing more predatory journals to raise fees since they know they’ll be paid. There’s also tricky situations where a grant might be awarded for doing research but not be renewed at the time publication happens. There might be ways to get around some of these hurdles and I think making an effort to address them would be worthwhile in the long term but there aren’t easy answers.

I think there are some cultural shifts happening to acknowledge review is more than just formal peer review prior to publication. There are meaningful discussions happening with preprint servers that were mentioned before - though there’s also a dumping of papers that are fundamentally flawed and never changed. Also with my experience with computational chemistry I have seen there are influences from open source software where people are making data in addition to code available in public repositories.

Yeah it has been a tangent - such that I almost forgot which thread this was in. I’ll probably try to avoid derailing more moving forward

In an ideal world, the cost of publishing is refunded (or paid for right off the bat), and is separate from the general research grant. But I read and understand your concerns with that model.

What is the impact of what you're calling data and code being made available?

 

[Ulorian - Agent of Chaos]

I don't have your experience with the trial so I will trust your results there. But in general I think these issues speak more to a mismatch in expectations than failure of concept.

Archiving comments properly is a technical concern and solvable.

As for the readers looking at just the abstract (and, I assume, trusting it) -- frankly, that is an issue even if the research is peer reviewed. The replication crisis, not to mention the outright fraud crisis, is fully exposed at this point. I'm not sure what use case you imagine -- is it doctors reading the abstracts and then treating patients based on this? That would speak to a need for an intermediary between research and clinic (like UpToDate) even with peer review.

Or if it is researchers just reading the abstract, for purposes other than evaluating if they should read further or not...then I am not sure I trust their scholarship.

But yes, under a "public post, public comment" regime, abstracts are going to be less trustworthy. I think this is a good thing, not a bad thing, because it means people stop trusting things they ought not to trust just because it has the peer review stamp.

Another way to phrase it is--peer review is designed to strengthen the "weakest link", the weakest papers. But peer review isn't an effective tool to prevent them, and can make weak links look strong because they have a seal of approval. In a more public regime, the job of vetting and compiling all this information for end users who don't want to read the papers can be done systematically rather than by ad hoc committees.

Unfortunately I am not sure you can capture how the dynamics will shift with a pilot study, because it (hopefully) will lead to a cultural shift in how practicioners engage with scientific papers. But that cultural shift is desirable anyway.

I sort of get where you're coming from, but in this case I would not trust a) the general public to provide results that are i) sensible ii) to be of a low enough volume that that the 'non-sensible' comments would be weeded out by folks who actually know what they are about, given that they have actual lives and b) that bad actors wouldn't take advantage of this flood to skew results away from the truth.

i.e. this should be gate-kept by known and recognised expert peer reviewers