Discussion: Shifting some load off the judges.

CaBaNa · May 26, 2011

Cheers,

Was brainstorming a bit on this concept.

How would everyone feel about;

Removing judge approval requirement for PC's
1. altogether, allowing anyone to check characters
2. altogether, making "character checker" an official position
3. for every approval after the first, allowing anyone to check characters
4. for every approval after the first, making "character checker" an official position.

Nothing restricts a judge from also being a character checker in any of these.

Pentius · May 26, 2011

Hm...I think it deserves talking about, at the least. I'm new, so I'm not intricately familiar with how things work, but judges seem to have a lot on their plates. They approve first level characters and all level ups, and new adventures, then check finished adventures for accuracy. And that's just what I remember hearing about, I'm sure there's more. On top of that, these folks should have a bit of time to play as well!

I'd be hesitant to shift it to where just anyone could check characters, though, and for two main reasons. First, it could be abused. If a couple people got together and decided to approve each other's illegal characters, for example. Secondly, it can be hard to get things done when you lay the responsibility on a large group of people but none of them have individual responsibility. In other words, if everyone can do it, most people will just wait for someone else to do it. A 'character checker' position could work out, though. That way they could even have a character checker email, like the judges one currently, to receive notifications and whatnot.

I'm in the air over whether to still call judges in for first levels or not. I think it'd depend on the level of expectation placed on the character checkers.

Iron Sky · May 26, 2011

Alternately, we could usher in some new judges that are fresh, new, and ready to review.

There would, of course, be some prerequisites; having access to up-to-date character builders and/or all the books would be a good place to start.

covaithe · May 26, 2011

I've been thinking about this a lot recently. Some points:

with the exception of DMing, character judging is by far the most labor-intensive thing we do. It would be awesome to spread that load around some.
To my knowledge, we've never had anyone try to gain an advantage in L4W by deliberately putting wrong numbers on their sheet. I admit that when I was writing down the rules for character reviews, lo these many years ago, I had cheaters on my mind. But that doesn't seem to actually be a problem here.
That said, I think it's still the case that many, if not most, character sheets have minor errors that really should be fixed. That includes my sheets, which always seem to have something wrong, no matter how I try.
Email sucks as a technology for handling character approvals. It involves a lot of manual administration to add and remove people from the list of reviewers. (And gmail has made this harder recently, requiring verification before allowing me to forward things to someone.) I think there's still a hard cap on the number of people who can get the notifications, and while we've never actually hit that limit, we've been close. You have to spell the subject line right, or it gums up the works. It takes work to figure out who has been waiting for approvals the longest. Emails can get lost or misfiled (or marked as spam), and people fall off the queue and get forgotten. When someone wants to help out, it's surprisingly hard to point them to a list of work needing to be done.
The Official Approved Character thread has fallen into disuse. It was never terribly reliable, but in its day it was more reliable than the ApprovedCharacters category in the wiki. Which is still not very reliable. What I'm getting at is that there's no *reliable* list anywhere of who is approved at what level, and there hasn't been for a long time (if ever).

I think we still need approvals. I think we still need more than one person to look each sheet over. It would be nice if we had some way to indicate that at least one of those people was "trusted" or "senior" or "veteran", in the sense that they had demonstrated the ability to find and point out mistakes and not just rubber-stamp it. I *don't* think it's necessary to require that judges be involved.

My ideal solution would involve some kind of work-tracking software of some kind, e.g. Bugzilla or Trac or JIRA or something. What I'd want it to be able to is something like this:

When you first sign up, you put in your email address.
When you submit a character for approval, you open a ticket, including a link to your character sheet.
Anyone who wants to can view the list of open tickets, i.e. approvals that still need doing, sorted and filtered in various ways.
Ideally, we'd be able to enter custom workflow steps for tickets, i.e. "needs two approvals", "needs one approval (veteran)", "needs one approval (any)", and "approved". Ideally there'd be a way to assign permissions to actions, i.e. only veterans can put it into the "needs one approval (any)" state, or move it from the "NOA(V)" state to "accepted".
Email notification when someone comments on your ticket or changes its workflow step.

It seems like we ought to be able to bend Bugzilla or Trac to be able to handle something like that. Heck, it seems like that wouldn't be too hard of a website to just write from scratch. On the other hand, putting together a system like that sounds like a fair amount of work. I'm not volunteering to do it myself. (Though, if anyone put together a working prototype, I'd be delighted to look at it.) Also, there are the standard objections to relying on external services (mainly that whoever owned them might go away, leaving us screwed).

Shorter term and less effortfully, I'm certainly open to suggestions for an approval workflow that would improve on the current situation.

VanderLegion · May 26, 2011

One of these days if I get bored I might look into what it would take to set up a basic website to do it. Seems like it wouldn't be too hard to link it to a database. You have a database table with fields for email, character name, approved, then anything else you might want included in that. When a player submits a character, it adds that info to the database, when a judge approves it, you just mark the approved column as true. From there it would also be really easy to get accurate lists of approved characters and characters waiting approval. Could also include the date the character was submitted so it'd be easy to see who'd been waiting the longest.

ScorpiusRisk · May 26, 2011

So if you had a form like this form, which fed to a spreadsheet like this spreadsheet, and this spreadsheet sent a daily digest to l4w.judges@gmail.com of each new submission and approval, would that be useful?

Son of Meepo · May 27, 2011

Agreed, something needs to happen. Right now the game Desolation Island is going to slow down to a crawl because the DM isn't willing to keep the adventure going as we've all "earned" enough XP to level up, but most of our characters still need a judge approval.

So is there anything we can do to turn this into an official proposal?

I'm willing to look over sheets as I have time, but I've been stymied because most of the sheets I've tried to look at already have one non-judge approval.

TwoHeadsBarking · May 27, 2011

If any characters in Desolation Island still needs an approval, have them PM me. Heck, if anybody is risking missing an xp reward because of lack of approval, and I haven't already approved you, PM me.

covaithe · May 27, 2011

ScorpiusRisk said:
So if you had a form like this form, which fed to a spreadsheet like this spreadsheet, and this spreadsheet sent a daily digest to l4w.judges@gmail.com of each new submission and approval, would that be useful?

I like the entry form. The spreadsheet... could be useable, with some tweaks (e.g. need the current approval status to be visible; dont' care so much about race/class).

The hard part would be managing the workflow. What do I do when I want to approve a character? What if I'm the second person to approve? What if I want to give comments, but not approve yet? If I've given comments but not approved, how can someone else who comes along to work on approvals tell that that's what's happened?

Thanks for doing this. It's a good starting point.

ScorpiusRisk · May 27, 2011

The worksheet is pretty flexible. In an ideal world, how would you want the workflow to go?

I think the easiest thing might be to make the spreadsheet publicly editable.

I've got two columns at the end, one for approval 1 and another for approval 2. It could be that we expand that so theres 2 columns for approvers and two columns for approval status. Write you're name in one the approver fields to signify that you've begun reviewing a sheet. The approval status could be: Approved, Pending (for waiting on the judge), or Comments (where you've made comments and the user needs to respond/update).

Discussion: Shifting some load off the judges.

CaBaNa

First Post

Pentius

First Post

Iron Sky

Procedurally Generated

covaithe

Explorer

VanderLegion

First Post

ScorpiusRisk

First Post

Son of Meepo

First Post

TwoHeadsBarking

First Post

covaithe

Explorer

ScorpiusRisk

First Post

Similar Threads