To summarise: absorbing sharp effects of your utility function into biased estimates can be a decent temporary computational hack, but it is dangerous to call the partial results you work with in the process 'estimates', since they in no way represent your beliefs.

It seems to me that it's best to use "your beliefs" to refer to the entire underlying distribution. Yes, you should not bias your beliefs--but the point of estimates is to compress the entire underlying distribution into "the useful part," and what is the useful part will depend primarily on your application's loss function, not a generalized unbiased loss function.