The problem with the notion that qualitative elements cannot be objectively gauged is that it ignores some pretty large areas where it's done all the time.
Take teaching. Teachers evaluate qualitative elements all the time. A significant portion of your grade is the result of qualitative assessements Now, if qualitative assessments were subjective, then the assessments you received should vary widely depending on the assessor. So, you might get a great grade in one year, but a failing grade in the next year despite producing the same quality of work simply because these assessments are subjective.
But ... it's not.
Just because (for example) multiple people can use an assessment rubric and generally agree on something,
does not mean that the underlying thing being assessed can be measured using an objective standard.
In order to understand this, I will use the following two example:
A. Using an agreed-upon standard of time and an atomic clock (these are the objective external referents), 30 different people measure the passage of two periods of time (despite the listing in order, the periods of time are generated randomly within each set). They do it in various intervals and repeat it 40 times:
1. 20 seconds and 22 seconds.
2. 1 hour and 2 hours.
3. 35 hours and 1 second and 34 hours and 59 minutes and 59 seconds.
At the end of this, the compare notes to determine, for sets (1, 2, 3) wh. Now, absent mechanical error there should be
universal agreement on the passage of time within each set and which was more.
B. The same sets are done, and 30 different people repeat it under the same conditions, but without any ability to measure the passage of time. Instead, they simply report which one felt longer. Given the lack of an external and agreed-upon (objective) referent, then you will see that while there is general consensus (and probably greatest consensus for 2 and perhaps none for 3), there is no ability to determine
objectively which is correct.
In this example, I am specifically using something that there is, in fact, an underlying objective measure ... but no way to objectively measure it because the people measuring it are necessarily using subjective means.
Compare this with a rubric for
quality. You have two problems with this-
A. The initial problem is defining quality. I'm not going to keep banging this drum, but most things that we think of as "quality" when it comes to art or design are societally determined. Hitchcock was relegated to the genre category until he wasn't. When Melville died, no one was reading Moby-Dick, until it gained a following after WW1. A Confederacy of Dunces was rejected for publication, after the suicide of the author, his mother found a smeared copy of the manuscript and she persisted (after numerous rejections) in getting a college professor to advocate for a university press to publish it 11 years after the death of her son. Authors, artists, styles ... most of them go in and out of fashion. Some are just mostly forgotten or not as popular as contemporaries despite the same technical merit (such as Juan Gris or Clyfford Styll) or sometimes styles are in, then out, then in again with the current people who choose what art should be emulated (such as Fragonard).
And the reason that this is important is if you are applying certain rubrics, you end up
excluding a lot of very good stuff. This is something that is familiar to us geeks- it's only recently that our genre favorites have been "allowed" at the big boy's table, because ... people would turn their noses and say, "OBJECTIVELY this isn't quality. OBJECTIVELY this isn't art." Cool, right?
Well, those trash comics by people like Ditko and Kirby now sell for millions. Did the
objective quality change? Or did the gatekeepers change their rubric for assessing what quality is? And if you can just change the rubric like that, is it measuring something objective? How do you define quality?
B. Which moves to the second issue; the assessment itself. As you acknowledge, different people will have different results when using these assessments. Just like, in the law, different judges (and juries) will have different results when applying the "reasonable man" standard. That doesn't mean that they are useless or that they don't do anything - far from it. But they aren't objective. Instead, they are an attempt to formalize and structure against a standardized and repeated form. Overall, this is a good system, and in the aggregate (assuming the rubric is designed to actually tease out what, inter alia, teaching quality is- which is not always the case) they should be able to generally distinguish qualities, especially when it comes to extremes (one hour or two hours).
But unless the assessments always end up with the exact same result, regardless of the person doing the assessment, means that they aren't objective.
And I will again reiterate the main point-
It is completely fine that something isn't objective. Because most objective things, most things that are discussed, have nothing to do with that. But calling things like "artistic quality" objective is simply an attempt to say that your views as to artistic merit cannot be discussed, because you are objectively correct. That is something that should always be rejected, and has traditionally been used to suppress those without the power to affect the rubrics and assessments being used. 
:max_bytes(150000):strip_icc()/great-recession_sourcefile-1-26478e8a0b5e48ce9df18f92cc991a7e.jpg)
