There's an important category of choices: the ones where any good choice is "acting as if" something is true.
That is, there are two possible worlds. And there's one choice best if you knew you were in world 1, and another choice best if you knew you were in world 2. And, in addition, under any probabilistic mixture of the two worlds, one of those two choices is still optimal.
The hotel example falls into this category. So, one of the important reasons to recognize this category is to avoid a half-speed response to uncertainty.
Many choices don't fall into this category. You can tell because in many decision-making problems, gathering more information is a good decision. But, this is never acting as if you knew one of the possibilities for certain.
Arguably in your example, information-seeking actually was the best solution: pull over and take out a map or use a GPS.
It seems like another important category of choices is those where the best option is trying the world 1 choice for a specified amount of time and then trying the world 2 choice. Perhaps these are the choices where the best source of information is observing whether something works? Reminds me of two-armed bandit problems, where acting-as-if and investigating manifest in the same kind of choice (pulling a lever).
Yeah. I mean, I'm not saying you should arrive late to class.
The way to work what you're saying into the framework is:
The cost of consistently arriving late is high
The cost (in minutes spent waiting for the class to start) of avoiding consistent lateness is less high
Therefore, you should pay this cost in minutes spent waiting
The point is to quantify the price, not to say you shouldn't pay it.
the soft sciences have to deal with situations which never exactly repeat
This is also true of evolutionary biology--I think it's not widely recognized that evolutionary biology is like the soft sciences in this way.
iii. Emphasize all rationality use cases evenly. Cause all people to be evenly targeted by CFAR workshops.
We can’t do this one either; we are too small to pursue all opportunities without horrible dilution and failure to capitalize on the most useful opportunities.
This surprised me, since I think of rationality as the general principles of truth-finding.
What have you found about the degree to which rationality instruction needs to be tailored to a use-case?
Several of these had the form “I, too, think that AI safety is incredibly important — and that is why I think CFAR should remain cause-neutral, so it can bring in more varied participants who might be made wary by an explicit focus on AI.”
I don't think that AI safety is important, which I guess makes me one of the "more varied participants made wary by an explicit focus on AI." Happy you're being explicit about your goals but I don't like them.
Wow, I've read the story but I didn't quite realize the irony of it being a textbook (not a curriuculum, a textbook, right?) about judgment and decision making.
The alternative I would propose, in this particular case, is to debate the general rule of banning physics experiments because you cannot be absolutely certain of the arguments that say they are safe.
Giving up on debating the probability of a particular proposition, and shifting to debating the merits of a particular rule, is I feel one of the ideas behind frequentist statistics. Like, I'm not going to say anything about whether the true mean is in my confidence interval in this particular case. But note that using this confidence interval formula works pretty well on average.
I don't know about the role of this assumption in AI, which is what you seem to care most about. But I think I can answer about its role in philosophy.
One thing I want from epistemology is a model of ideally rational reasoning, under uncertainty. One way to eliminate a lot of candidates for such a model is to show that they make some kind of obvious mistake. In this case, the mistake is judging something as a good bet when really it is guaranteed to lose money.
Inquiring after the falsifiability of a theory?
Not perfect but very good, and pretty popular.
I think that someone who merely believed they were happy, and then experienced real happiness, would not want to go back.