I don't like it very much. Instrumental rationality is the art and science of pursuing goals, and epistemic rationality is the special case of that where the goal is truth-seeking.

This OB post covered similar ground. What I took away from that post was that log odds are the natural units for converting evidence into beliefs, and probabilities are the natural units for converting beliefs into actions.

As an instrumental rationalist, I would say that 0 is a better approximation than 1/2 ....

How is this train of thought "instrumental"? You aren't making any choices or decisions outside of your own brain.

To make it a real instrumental example, consider whether or not you should go buy a meteorite shield. Lets say the shield costs S and if the meteorite hits you it costs M, and the true probability of the strike is p. So buying the shield is best if pM > S.

Now if you go with 0, you'll never buy the shield, so if pM > S you have an expected loss of (pM - S) due to your approxamation.

If you go with 1/2 then you'll buy the shield if M/2 > S. If M/2 > S and pM <= S then you bought the shield when you shouldn't have, and you lose and expected (S - pM).

So you see, it all depends on how big M is compared to S

M < S/p : 0 is the better instrumental approximation

M > S/p : 1/2 is better

In other words, if the risks (or payoffs) are small compared to the probabilities involved and the costs of shields, round to 0. Otherwise round to 1/2.

This is a bit tangential, but perhaps a bounded rationalist should represent his beliefs by a family of probability functions, rather than by an approximate probability function. When he needs to make a decision, he can compute upper and lower bounds on the expected utilities of each choice, and then either make the decision based on the beliefs he has, or decide to seek out or recall further information if the upper and lower expected utilities point to different choices, and the bounds are too far apart compared to the cost of getting more information.

I found one decision theory that uses families of probability functions like this (page 35 of http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.37.1906), although the motivation is different. I wonder if such decision systems have been considered for the purpose of handling bounded rationality.

I admit that I've learned about the KL divergence just now and through the wiki-link, and that my math in general is not so profound. But as it's not about calculation but about the reasoning behind the calculation, I suppose I can have my word:

The wiki-entry mentions that

Typically P represents the "true" distribution of data, observations, or a precise calculated theoretical distribution. The measure Q typically represents a theory, model, description, or approximation of P.

So P here is 10^-18 and Q is either 0 or 0.5.

What your epistemic rationalist has done seems like falling pray to the bias of anchoring and adjusting. The use of mathematical equations just makes the anchoring mistake look more fomal; it's not less wrong in any way. So while the instrumental rationalist might have a reason to choose the arbitary figure of 1/2 (it makes his decisions be more simple, for example) the epistemic rationalist does not. If the epistemic rationalist is shown the two figures of 0 and 1/2 and is asked what approximation is "better" he would probably say 0. And that's for several reason: First of all, if he is an epistemic rationalist and thus trueseeking, he wouldn't use the KL equation at all. The KL takes something accurate (or true) P and makes it less accurate (or less true) KLD, and that's exactly against what he is seeking - having more accurate and true results. But you tell me he has to choose between either "0" or "1/2". Well, if he has to chooce between one of these numbers he will still not choose to use the KL equation. The wiki mentions that the Q in the equation typically stands for "... a theory, model, description, or approximation of P" while the number "1/2" in your example is none of these but an arbitary number - this equation, then, does not fit the situation. He will use a different mathematical method, let's say, subtraction, and see the absolute value of what difference is smaller, in which case it will be 0's. Also, since 1/2 and 0 are arbitary numbers, an epistemic rationalist would know better than use any of these numbers in any equation, since it will produce a result that is accurate just as if would use any other two arbitary numbers. He would know that he should do his own calculations - ignoring the numbers 0 and 1/2 - and then compare his result to the numbers he is "offered" (0 and 1/2) and choose the closest number to his own calculation. Since he knows that the "true" probability is 10^-18 he will choose the closest number to his result which seems to be 0.

Of course, everything that I said about "1/2" above holds true about "0".

(I'm sorry in advance if my mathematical explentation are unclear or clumsy. If I explain arguments through math badly, then I explain arguments through math in English much worst as I was studying mathematics in a different language)

So what lesson does a rationalist draw from this? What is best for the Bayesian mathematical model is not best in practice? Conserving information is not always "good"?

Also,

I will simply rationalize some other explanation for the destruction of my apartment.

This seems distinctly contrary to what an instrumental rationalist would do. It seems more likely he'd say "I was wrong, there was actually an infinitesimal probability of a meteorite strike that I previously ignored because of incomplete information/negligence/a rounding error."

It might clarify things to note the connection between Kullback-Leibler divergence and communication theory. The Kullback-Leibler divergence is the utility function to use when minimizing the expected length of the signal encoding (i.e, recording or communicating) what actually happened. The choice of "1/2" or "0" is equivalent to to constraining the agent to choose between using one bit or or an infinite amount of bits to record/communicate the state of "improbable event did (not) occur".

In short, KL divergence isn't about truth-seeking per se. It's about the resources necessary to encode signals -- definitely an instrumental question.

I admit that I've learned about the KL divergence just now and through the wiki-link, and that my math in general is not so profound. But as it's not about calculation but about the reasoning behind the calculation, I suppose I can have my word:

The wiki-entry mentions that

Typically P represents the "true" distribution of data, observations, or a precise calculated theoretical distribution. The measure Q typically represents a theory, model, description, or approximation of P. So P here is 10^-18 and Q is either 0 or 0.5.

What your epistemic rationalist has done seems like falling pray to the bias of anchoring and adjusting. The use of mathematical equations just makes the anchoring mistake look more fomal; it's not less wrong in any way. So while the instrumental rationalist might have a reason to choose the arbitary figure of 1/2 (it makes his decisions be more simple, for example) the epistemic rationalist does not. If the epistemic rationalist is shown the two figures of 0 and 1/2 and is asked what approximation is "better" he would probably say 0. And that's for several reason: First of all, if he is an epistemic rationalist and thus trueseeking, he wouldn't use the KL equation at all. The KL takes something accurate (or true) P and makes it less accurate (or less true) KLD, and that's exactly against what he is seeking - having more accurate and true results. But you tell me he has to choose between either "0" or "1/2". Well, if he has to chooce between one of these numbers he will still not choose to use the KL equation. The wiki mentions that the Q in the equation typically stands for "... a theory, model, description, or approximation of P" while the number "1/2" in your example is none of these but an arbitary number - this equation, then, does not fit the situation. He will use a different mathematical method, let's say, subtraction, and see the absolute value of what difference is smaller, in which case it will be 0's. Also, since 1/2 and 0 are arbitary numbers, an epistemic rationalist would know better than use any of these numbers in any equation, since it will produce a result that is accurate just as if would use any other two arbitary numbers. He would know that he should do his own calculations - ignoring the numbers 0 and 1/2 - and then compare his result to the numbers he is "offered" (0 and 1/2) and choose the closest number to his own calculation. Since he knows that the "true" probability is 10^-18 he will choose the closest number to his result which seems to be 0.

Of course, everything that I said about "1/2" above holds true about "0".

(I'm sorry in advance if my mathematical explentation are unclear or clumsy. If I explain arguments through math badly, then I explain arguments through math in English much worst as I was studying mathematics in a different language)

I admit that I've learned about the KL divergence just now and through the wiki-link, and that my math in general is not so profound. But as it's not about calculation but about the reasoning behind the calculation, I suppose I can have my word:

The wiki-entry mentions that

Typically P represents the "true" distribution of data, observations, or a precise calculated theoretical distribution. The measure Q typically represents a theory, model, description, or approximation of P. So P here is 10^-18 and Q is either 0 or 0.5.

What your epistemic rationalist has done seems like falling pray to the bias of anchoring and adjusting. The use of mathematical equations just makes the anchoring mistake look more fomal; it's not less wrong in any way. So while the instrumental rationalist might have a reason to choose the arbitary figure of 1/2 (it makes his decisions be more simple, for example) the epistemic rationalist does not. If the epistemic rationalist is shown the two figures of 0 and 1/2 and is asked what approximation is "better" he would probably say 0. And that's for several reason: First of all, if he is an epistemic rationalist and thus trueseeking, he wouldn't use the KL equation at all. The KL takes something accurate (or true) P and makes it less accurate (or less true) KLD, and that's exactly against what he is seeking - having more accurate and true results. But you tell me he has to choose between either "0" or "1/2". Well, if he has to chooce between one of these numbers he will still not choose to use the KL equation. The wiki mentions that the Q in the equation typically stands for "... a theory, model, description, or approximation of P" while the number "1/2" in your example is none of these but an arbitary number - this equation, then, does not fit the situation. He will use a different mathematical method, let's say, subtraction, and see the absolute value of what difference is smaller, in which case it will be 0's. Also, since 1/2 and 0 are arbitary numbers, an epistemic rationalist would know better than use any of these numbers in any equation, since it will produce a result that is accurate just as if would use any other two arbitary numbers. He would know that he should do his own calculations - ignoring the numbers 0 and 1/2 - and then compare his result to the numbers he is "offered" (0 and 1/2) and choose the closest number to his own calculation. Since he knows that the "true" probability is 10^-18 he will choose the closest number to his result which seems to be 0.

Of course, everything that I said about "1/2" above holds true about "0".

(I'm sorry in advance if my mathematical explentation are unclear or clumsy. If I explain arguments through math badly, then I explain arguments through math in English much worst as I was studying mathematics in a different language)

I am itching to ask why we are approximating in such a terrible fashion. But I know better.