[Proposal] Judge Credits

Iron Sky · Mar 23, 2011

Right now, judging - especially character approvals - is a thankless job, everyone bemoaning how long it takes to get their character/adventure/etc approved with no real reward to the judges.

I propose a reward for judges, call them Judge Points (JPs). Obviously an individual "judgment" doesn't take as much time and energy as running an adventure, so they wouldn't be worth as much as a DM Credit, but how about 1/10 of one?

Suggested awards:
*Reviewing a character awaiting approval(1 JP)
*Being "the Judge" for an adventure(1 JP/month?)
*Approving adventure awards(1 JP)
*Approving an adventure(1 JP)

I'm not sure how much time judges invest in threads they are "the Judge" for, so you'd have to inform me on that. Only current judges would be eligible for any but the first award(since one of the character approvals can be a "peer" review), but I'd like there to be some way that they can be rewarded for all the time they volunteer to keep L4W going.

And no, I'm not a judge if anyone was wondering. I just don't think renau1g has enough DM Credits.

Mewness · Mar 23, 2011

Just in the area of character reviews (of which I do quite a few, and I am not a judge) I have somewhat mixed feelings. On the one hand, it would be nice if I got something for doing character reviews. On the other, I really am anal enough to not want to encourage people to do them sloppily. =)

Iron Sky · Mar 23, 2011

Mewness said:
Just in the area of character reviews (of which I do quite a few, and I am not a judge) I have somewhat mixed feelings. On the one hand, it would be nice if I got something for doing character reviews. On the other, I really am anal enough to not want to encourage people to do them sloppily. =)

I thought about that but didn't post it. Maybe limit awards to 1 JP per category per day?

In an unrelated note, I've been gaming too much - when I first typed this I put gm tags around it, then thought, "I'm not DMing in this thread" and replaced them with ooc tags. Only when I was about to hit post did I realize this wasn't an adventure. :lol:

horticulture · Mar 23, 2011

I'm kinda partial to this idea, though if JPs are worth 1/10 a DM Credit I think they should get more JPs/month for judging an adventure. 3 JP/month perhaps?

covaithe · Mar 23, 2011

I'm gonna go straight to a NO on this one. Judge credits were discussed way back in the dawn of time, around the same time as DM credits. I felt then, and still feel, that in-game rewards for judging would create the potential for a conflict of interest. It could create the perception that judge decisions might not be impartial, and that perception, even if there's no fire behind that smoke, would be detrimental to the community. Not worth it, IMO.

JoeNotCharles · Mar 23, 2011

I think there should be some reward for judging characters, since there's such a backlog there. But other judging tasks aren't as onerous.

renau1g · Mar 23, 2011

covaithe said:
I'm gonna go straight to a NO on this one. Judge credits were discussed way back in the dawn of time, around the same time as DM credits. I felt then, and still feel, that in-game rewards for judging would create the potential for a conflict of interest. It could create the perception that judge decisions might not be impartial, and that perception, even if there's no fire behind that smoke, would be detrimental to the community. Not worth it, IMO.

Not that I'm leaning either way, but I'm curious why you think it may create a conflict of interest? If I'm judging a game, or reviewing a PC, how would earning a few credits affect my review? Personally, I'm indifferent. I have enough DM credits to put a PC up nearly a whole tier so there's no horse in the race for me there

I know I've been slowed up by the RL monster, but hope in the summer to have it calmed down a bit...

JoeNotCharles said:
I think there should be some reward for judging characters, since there's such a backlog there. But other judging tasks aren't as onerous.

I concur, judging an adventure is typically fairly minimal time, reviewing an adventure takes me usually 10-20 minutes and as they're so infrequent it's no biggie.

I wouldn't mind something in place for character reviews, but (I think as we discussed before) it could create a problem over who gets to review a PC (especially amongst non-judges.

covaithe · Mar 23, 2011

Exactly what the conflict of interest was would of course depend on the details of how the rewards are applied. Suppose, for the sake of argument, that we decided to give some token bit of XP for reviewing a character. Now suppose that one of my PCs is in a game, and we reach an extended rest and he's just short of leveling up. I look at the queue of characters awaiting review, and do a quick bit of mental arithmetic and discover that if I reviewed them all and turned in my rewards immediately, Quagmire would level. But suppose further that I don't have time to properly review them all (or, more realistically, that some of them rely on information that I don't have, such as books published since I let my DDI subscription lapse). I'd be lying if I said it wouldn't occur to me to just mark all of them approved without actually checking them over in detail, take the xp, and go on my merry way. There's the conflict of interest: my desire for in-game rewards has tempted me to take shortcuts in pursuing my judge duties.

This is a fairly well-known problem in business management, particularly in the software field: whenever you give people performance incentives based on some set of rules, sooner or later someone will try to game the system and optimize the performance metric instead of actually performing well. A classic example is rewarding software developers based on the number of bugs they fix. What eventually happens is that they write *more bugs*, so that they can get paid for fixing them. It's exactly the opposite of what you want.

I'd really like for there to be no reason for people to even think about trying to game the judging system.

JoeNotCharles · Mar 23, 2011

We're not building bridges here, though - the consequences of a character slipping through with a mistake in it aren't that big. We already have a second judge looking at them in case the first judge misses something, and if a character has major problems every DM that has that character in an adventure has a chance to notice. We don't seem to have a big problem with people intentionally submitting cheating characters, the judging is mainly to make sure they didn't make any mistakes in character creation.

I'd say that right now characters sitting in the queue for a long time with no feedback is a bigger problem than characters being accepted without enough scrutiny.