You're welcome! And I'm sorry if I went a little overboard. I didn't mean it to sound confrontational.

The calibration you get, by the way, will be better represented by the fact that if you assigned 50% to the candidate that lost, then you'll necessarily have assigned a very low probability to the candidate that won, and that will be the penalty that will tell you your calibration is wrong.

The problem is the definition of more specific. How do you define specific? The only consistent definition I can think of is that a proposition A is more specific than B if the prior probability of A is smaller than that of B. Do you have a way to consistently tell whether one phrasing of a proposition is more or less specific than another?

By that definition, if you have 10 candidates and no information to distinguish them, then the prior for any candidate to win is 10%. Then you can say "A: Candidate X will win" is more specific than "~A: Candidate X will not win", because P(A) = 10% and P(~A) = 90%.

Since the proposition "A with probability P" is the exact same claim as the proposition "~A with probability 1-P"; since they are the same proposition, there is no consistent definition of "specific" that will let one phrasing be more specific than the other when P = 50%.

"Candidate X will win the election" is only more specific than "Candidate X will not win the election" if you think that it's more likely that Candidate X will not win.

For example, by your standard, which of these claims feels more specific to you?

A: Trump will win the 2016 Republican nomination

B: One of either Scott Alexander or Eliezer Yudkowsky will win the 2016 Republican nomination

If you agree that "more specific" means "less probable", then B is a more specific claim than A, even though there are twice as many people to choose from in B.

Which of these phrasings is more specific?

C: The winner of the 2016 Republican nomination will be a current member of the Republican party (membership: 30.1 million)

~C: The winner of the 2016 Republican nomination will not be a current member of the Republican party (non-membership: 7.1 billion, or 289 million if you only count Americans).

The phrasing "C" certainly specifies a smaller number of people, but I think most people would agree that ~C is much less probable, since all of the top-polling candidates are party members. Which phrasing is more specific by your standard?

If you have 10 candidates, it might seem more specific to phrase a proposition as "Candidate X will win the election with probability 50%" than "Candidate X will not win the election with probability 50%". That intuition comes from the fact that an uninformed prior assigns them all 10% probability, so a claim that any individual one will win feels more specific in some way. But actually the specificity comes from the fact that if you claim 50% probability for one candidate when the uninformed prior was 10%, you must have access to some information about the candidates that allows you to be so confident. This will be properly captured by the log scoring rule; if you really do have such information, then you'll get a better score by claiming 50% probability for the one most likely to win rather than 10% for each.

Ultimately, the way you get information about your calibration is by seeing how well your full probability distribution about the odds of each candidate performs against reality. One will win, nine will lose, and the larger the probability mass you put on the winner, the better you do. Calibration is about seeing how well your beliefs score against reality; if your score depends on which of two logically equivalent phrasings you choose to express the same beliefs, there is some fundamental inconsistency in your scoring rule.

Yes, but only because I don't agree that there was any useful information that could have been obtained in the first place.

"Candidate X will win the election with 50% probability" also implies the proposition "Candidate X will not win the election with 50% probability". If you propose one, you are automatically proposing both, and one will inevitably turn out true and the other false.

If you want to represent your full probability distribution over 10 candidates, you can still represent it as binary predictions. It will look something like this:

Candidate 1 will win the election: 50% probability

Candidate 2 will win the election: 10% probability

Candidate 3 will win the election: 10% probability

Candidate 4 will win the election: 10% probability

Candidate 5 will win the election: 10% probability

Candidate 6 will win the election: 2% probability

...

Candidate 1 will not win the election: 50% probability

Candidate 2 will not win the election: 90% probability

...

The method described in my post handles this situation perfectly well. All of your 50% predictions will (necessarily) come true 50% of the time, but you rack up a good calibration score if you do well on the rest of the predictions.

For the problem of how to phrase the 50% confidence predictions, it might be useful to use the more specific proposition (or one that has the smaller uninformative prior probability). For instance, if you have a race between 10 candidates, and you think candidate X has a 50% chance to win, you should phrase your prediction as "Candidate X will win the election" rather than "Candidate X will not win the election".

If you consistently phrase your 50% confidence predictions this way, then your prediction results tell you something useful. If your 50% confidence predictions come true only 10% of the time, then maybe your problem is overconfidence in your reasons to deviate from uninformed priors or overconfidence in concocting reasons to believe a certain hypothesis in the vast hypothesis space.

edit: The biggest weakness of this approach is what do you do when you're choosing between something like "this coin flip will come up heads" and "this coin flip will come up tails"? Or when it is unclear what an uninformed prior would even be.

In my imagination, there's a terrible accident that leaves someone other than the helmet-wearer paralyzed or dead, and investigators are surprised to see that one driver was wearing... a helmet?? It's almost like he knew he was going to get into an accident -- perhaps even intended to. Certainly, that's what people would think reading the articles about it. Perhaps a jury would, as well.

If this happens to you say you did it because of my blog post. If you pay transportation costs I will even testify for you at a trial.

anti-lock breaks are said to increase risky driving behavior, after all

Economists call this the "Peltzman effect" and it seems robust. It does reduce the social benefits of driving helmets. One economist took the implications of this effect to their logical conclusion and suggested that steering wheels should have a spike pointed at their drivers.

Good suggestion. I'm a cyclist and sometimes I walk around wearing my bike helmet. Some people give me funny looks, but I don't care. To regularly wear helmets, you have to recognize that avoiding TBI is more important than looking slightly silly or messing up your hair. Unfortunately most people don't get this. I think if helmets become more popular people will find that they don't actually mind the look, as you suggested in another comment.

Risk compensation is one potential problem with wearing a helmet. An example was mentioned in the original post: drivers passing cyclists who wear helmets closer. Also concerning is that cyclists ride more dangerously if they wear helmets (here's an article which I haven't read in full, but it discusses this issue). I imagine seat belts have a similar effect, as might drivers wearing helmets. Based on this idea backwards, I've read a proposal to add a spike to steering wheels to reduce dangerous driving.

Also worth noting is that avoiding long commutes and drives improves safety. Risk (probably) increases monotonically with duration.

One last suggestion would be to drive with a dashcam to make yourself slightly safer. I ride my bike with a very obvious helmet cam and I have observed that some drivers seem to drive safer around me when they notice it. I'm recording them, so if they make a bad move they'll be much worse off in court. I also think the camera makes me ride safer for the same reasons.

We would run into the same problem for any description of a quale/sensation.

Only if you decide you're defining a sensation and not some physical phenomenon.

The highly contextual nature of color perception is ubiquitous. Human color processing is always making contextual adjustments from scene illumination.

Yes, I understand that very well. But all that tells you is that different definitions will diverge in many cases.

Also, don't you mean objective?

"Subjective" was probably the wrong word. I distinguish:

• A physical approach which defines color through spectral power distributions

• A human objective approach which defines color via the tristimulus model (the CIE color space, etc.)

• A human subjective approach which defines color as a particular perception

The human subjective approach has -- as you have pointed out -- all the issues associated with talking about subjective sensations, that is, they are essentially unobservable and it's very hard to get a good handle on them. That, to me, makes defining color through qualia a definition that isn't useful all that often.

Cheers! I used blue and green because the grue and bleen example is a standard philosophical one, and it's not hard to make pretty rigorous (as Lumifer suggested, with the radiation frequency, and some outside conditions added to it). The pretty rigorous definition is only a partial match for the subjective blue and green, as you pointed out, and I'll try and make that clearer in any subsequent write-up.

Thanks :-)

that is not sufficient nor necessary to ensure that the object would typically generate a red color sensation in humans

You are now dealing in circular logic. If you criterion for "red" is a "color sensation in humans", you have already defined red. That's it, we're done. My point is that you do not have to define "red" in terms of human qualia.

Definitions should be judged by their usefulness. Sometimes "a human would call that red" is the right defintion, sometimes "this light peaks at ~700nm" is the right definition. For example, if the sensor in your telescope captured a few photons from a dim distant star, you might call them "red" even if the human visual system will be unable to process these photons (or associate with them the qualia of "red").

it's not too hard to make color an objective creature

It is easy to make color an objective creature -- just define it a particular mixture of wavelengths of visible light. To produce a workable definition of subjective color is much harder -- this is a practical matter in photography and graphics and whole books are written on the subject.