I admit that I've learned about the KL divergence just now and through the wiki-link, and that my math in general is not so profound. But as it's not about calculation but about the reasoning behind the calculation, I suppose I can have my word:
The wiki-entry mentions that
Typically P represents the "true" distribution of data, observations, or a precise calculated theoretical distribution. The measure Q typically represents a theory, model, description, or approximation of P.
So P here is 10^-18 and Q is either 0 or 0.5.
What your epistemic rationalist has done seems like falling pray to the bias of anchoring and adjusting. The use of mathematical equations just makes the anchoring mistake look more fomal; it's not less wrong in any way. So while the instrumental rationalist might have a reason to choose the arbitary figure of 1/2 (it makes his decisions be more simple, for example) the epistemic rationalist does not. If the epistemic rationalist is shown the two figures of 0 and 1/2 and is asked what approximation is "better" he would probably say 0. And that's for several reason: First of all, if he is an epistemic rationalist and thus trueseeking, he wouldn't use the KL equation at all. The KL takes something accurate (or true) P and makes it less accurate (or less true) KLD, and that's exactly against what he is seeking - having more accurate and true results. But you tell me he has to choose between either "0" or "1/2". Well, if he has to chooce between one of these numbers he will still not choose to use the KL equation. The wiki mentions that the Q in the equation typically stands for "... a theory, model, description, or approximation of P" while the number "1/2" in your example is none of these but an arbitary number - this equation, then, does not fit the situation. He will use a different mathematical method, let's say, subtraction, and see the absolute value of what difference is smaller, in which case it will be 0's. Also, since 1/2 and 0 are arbitary numbers, an epistemic rationalist would know better than use any of these numbers in any equation, since it will produce a result that is accurate just as if would use any other two arbitary numbers. He would know that he should do his own calculations - ignoring the numbers 0 and 1/2 - and then compare his result to the numbers he is "offered" (0 and 1/2) and choose the closest number to his own calculation. Since he knows that the "true" probability is 10^-18 he will choose the closest number to his result which seems to be 0.
Of course, everything that I said about "1/2" above holds true about "0".
(I'm sorry in advance if my mathematical explentation are unclear or clumsy. If I explain arguments through math badly, then I explain arguments through math in English much worst as I was studying mathematics in a different language)
What is the probability that my apartment will be struck by a meteorite tomorrow? Based on the information I have, I might say something like 10-18. Now suppose I wanted to approximate that probability with a different number. Which is a better approximation: 0 or 1/2?
The answer depends on what we mean by "better," and this is a situation where epistemic (truthseeking) and instrumental (useful) rationality will disagree.
As an epistemic rationalist, I would say that 1/2 is a better approximation than 0, because the Kullback-Leibler Divergence is (about) 1 bit for the former, and infinity for the latter. This means that my expected Bayes Score drops by one bit if I use 1/2 instead of 10-18, but it drops to minus infinity if I use 0, and any probability conditional on a meteorite striking my apartment would be undefined; if a meteorite did indeed strike, I would instantly fall to the lowest layer of Bayesian hell. This is too horrible a fate to imagine, so I would have to go with a probability of 1/2.
As an instrumental rationalist, I would say that 0 is a better approximation than 1/2. Even if a meteorite does strike my apartment, I will suffer only a finite amount of harm. If I'm still alive, I won't lose all of my powers as a predictor, even if I assigned a probability of 0; I will simply rationalize some other explanation for the destruction of my apartment. Assigning a probability of 1/2 would force me to actually plan for the meteorite strike, perhaps by moving all of my stuff out of the apartment. This is a totally unreasonable price to pay, so I would have to go with a probability of 0.
I hope this can be a simple and uncontroversial example of the difference between epistemic and instrumental rationality. While the normative theory of probabilities is the same for any rationalist, the sorts of approximations a bounded rationalist would prefer can differ very much.