Proposal: More open character judging

JoeNotCharles · Jun 10, 2009

So, I see N3rday's been added as a character judge. Welcome, N3rday! But I'm not sure adding more char-only judges is going to help with the backlog right now, since the rules say each character needs to be judged by at least one full judge. Most of the characters in the queue have been reviewed by me and are waiting for a full judge, which doesn't leave much for N3rday to do.

The point of needing two judges for each character is that sometimes, one judge will miss something or get a rule wrong, so it's always good to have a double-check - don't want to change that. And the point of requiring a full judge is that judges have been around for a while and we know that they have a good handle on the rules, while some character judges just show up out of the blue and volunteer to help clear out the queue, so we don't necessarily know whether they're skilled or are trustworthy. (I'm amazed that I haven't seen any trolls here, but we have to plan for it in case one shows up...)

However, this means that even if we expand the number of character judges, we still have a lot of characters bottlenecked until a full judge can look at them, and the full judges are very busy.

I'd like to propose a change to the rule: instead of having character judges and full judges, let's have character judges and probationary character judges. Every full judge is automatically a full character judge, but omebody who signs up to be a character judge is "on probation" for a certain number of characters - lets say 10. At least one approval for each character needs to be from a full character judge. Once they judge 10 new characters without consistently screwing things up, a probationary character judge graduates and is able to act as a full character judge.

I'm pretty sure Lord Sessadore and I have done enough to graduate from probation immediately, which would let N3rday start doing the second review on all the characters that are sitting there with "1 approval from JoeNotCharles". Not sure about Ata - he mentioned in the discussion thread that he hadn't done very much of it. The number 10 is up for discussion as well, of course.

Lord Sessadore · Jun 10, 2009

I like the idea. It would certainly ease some of the load on the judges; like you said, the only reason we're requiring that one approval has to come from a judge is to make sure that they get at least one good hard look. Obviously the judges aren't the only ones capable of that, though, which is why I like this idea.

Oh, and I am one of the judges already, Joe. You seem to keep forgetting that

Now if I could just find time to finish off approving some of the characters you've done ...

JoeNotCharles · Jun 10, 2009

Sorry, I was going by this post here, where you aren't listed.

Wait, yes you are, you made that post! I just skimmed it looking for names, and "myself" didn't jump out at me. I thought it was weird that you weren't a full judge, but you're the one who kept sending around updates. "Wow," I thought, "He really threw himself into this character judging!"

(I may have to change my vote on the being a jerk proposal...)

Lord Sessadore · Jun 10, 2009

Ah, no worries. And your link's broken, it goes to this thread

garyh · Jun 10, 2009

I endorse this idea (speaking as a Judge who just never seems to have time to review characters).

Atanatotatos · Jun 10, 2009

Seems like a good idea. 10 also looks like a good number. Not sure how many characters I reviewed, might be 7, or 10, or 13, I don't really know...

covaithe · Jun 10, 2009

Mmm... As another judge who's very far behind on character approvals, I'm tentatively in favor of the idea. Two concerns:

1. What does "without consistently screwing it up" mean, in practice? It kind of sounds like a judgement call from the judges, or at least from the full character reviewers. If that's the case, I'd rather leave it as an open judgement call, rather than putting a number (10 character sheets) on it.

2. There's still one step in the process that requires a full judge: updating the official character list. I don't think I'd support handing the Facilitator account password out to people who aren't judges. That isn't really a problem, as it's no different from the current situation, if a judge makes the first approval and a non-judge makes the second; a judge still has to be paying enough attention to come along and do the paperwork.

Lord Sessadore · Jun 10, 2009

The thought of just making it an open judgement call as well, though we'd have to establish whether it has to be a judge that makes that call or if a full character reviewer can as well. I also don't feel like keeping statistics on which character reviewers have done how many sheets and how well

I don't think that updating the official character list is much of a problem, really. As you said, all it takes is for one of us to keep an eye on what's going on, and update the list when needed. Which takes about 10 minutes.

stonegod · Jun 10, 2009

By-the-By: Judges should let character waiting folks know that they at least got their email. Not that I sent one yesterday late and haven't heard anything...

LEW uses a similar system, IIRC. Might want to have a look there to see if they had any other thoughts on the matter.

JoeNotCharles · Jun 10, 2009

Lord Sessadore said:
The thought of just making it an open judgement call as well, though we'd have to establish whether it has to be a judge that makes that call or if a full character reviewer can as well. I also don't feel like keeping statistics on which character reviewers have done how many sheets and how well

I think it would only be a problem if it really stands out that somebody's making a lot of mistakes. If the second reviewer sees somebody doing a really bad job on a single sheet (tons of mistakes missed, or, I dunno, defacing somebody's sheet or something) they should make a note of it.

Basically if somebody's probation is about to finish and another judge sees the name and says, "Eesh, THAT guy? He sucks!" then there's a problem. If it doesn't stand out enough for anyone to notice, it should be fine.