This is my first article, and I'm submitting it in the discussion forum, so hopefully I've done this correctly and we can discuss!

Anyway, I have a group of friends who are really interested in movies, and they feel very strongly about them.  I find their convictions interesting.  Specifically the way they adamantly argue that, for instance, Midnight in Paris is a "better" movie than Bridesmaids or whatever.  I got to thinking about how one would create metrics by which you could evaluate any movie.

First attempt: A simple scale by which you give rankings (1-10) to a list of movie attributes (the metrics), sum up the total, and highest number is the best movie.

Some metrics might be:

Plot/Story

Acting

Effects, Costumes, Editing, etc.

Script/Dialogue

Humor

Drama/Passion

Suspense

So we can argue about what the metrics should be and how many we need, but since we're not worried about justifying our system objectively we can include whatever criteria we want.  We could even add a weighting component so some metrics are worth more than others.  My system can even be different than yours.  The problem, though, is that in reality movies don't need to excel at all metrics to be perfect for what they are.  Would Schindler's List be a better movie if they were cracking jokes the whole time?  Would 12 Angry Men be better if it had more special effects?  And it's a little weird to evaluate the acting in Up or Toy Story 3.  (No offense to voice actors.)

The idea of ranking movies is really about the challenge of comparing things that are the same class (movies) but very different types (comedy, horror, drama, etc) -- in content, goal, method, etc.  Is it possible to come up with metrics by which to compare anything in the class regardless of type?  Assuming you can come up with which metrics you find valuable/relevant, some of them will apply to one type but not another.  But you also can't completely disregard metrics that are not common between all types, because you've just said you find them valuable/relevant (in this case, to your enjoyment of a movie).

These thoughts led me to the question which I will pose here: How do you evaluate items in a class based on multiple metrics when not all metrics are ALWAYS relevant?

Some brainstorming to try to answer that question (modifying the system proposed above):

Allow "N/A" for a metric and then divide the total points by the total possible based on applicable metrics.  But this ignores, for example, humorless movies that could have used some humor.

Ok, so maybe give a movie with no humor a 10/10 in the humor metric IF it was perfect without it, or some other X/10 if it needed some humor.  But that seems to inflate the movie's rating by giving some amount of credit for an attribute that it didn't actually have.

I briefly considered having flexible weightings assigned subjectively to the metrics for each movie rated.  But the whole point of this is to have standard criteria for all movies -- not different scales.

Anyway, any ideas?  Are there already systems for this sort of thing in different arenas of which I'm not aware?  Could you develop a system for this sort of evaluation that could also be used to evaluate businesses, school classes, marketing techniques, or just about anything else?