So, if your writing "here be dragons" on a map results in someone encountering a dragon when traveling to the mapped area (a very popular theme in SF/F), how useful is the concept of reality?
The placebo effect came up elsewhere as an example where beliefs alter reality. Similarly, self-fulfilling prophecies need not rely on magic; if I believe I'll fail at a task, I very probably alter, just by holding this belief, my odds of completing that task. The modified litany isn't "All beliefs modify reality," but "I should have accurate beliefs about what beliefs have repercussions in reality." Your dragon example is merely a demonstration of a belief which is immaterial to reality, for the purposes at least of the subject of the belief.
I believe this response suffices in answering the rest of your objections, as well.
See http://lesswrong.com/lw/h69/litany_of_instrumentarski/8qht for an example of a pretty common theme in this post. Contrary to the argument presented in the comment [ETA: I misread the comment; this argument isn't actually present. My apologies!], rationality doesn't break down, a specific and faulty idea held by some rationalists breaks down.
The Litany of Tarski (formulated by Eliezer, not Tarski) reads
If the box contains a diamond,
I desire to believe that the box contains a diamond;
If the box does not contain a diamond,
I desire to believe that the box does not contain a diamond;
Let me not become attached to beliefs I may not want.
This works for a physical realist, but I have been feeling uncomfortable with it for some time now. So I have decided to reformulate it in a more instrumental way, replacing existential statements with testable predictions. I had to find a new name for it, so I call it the Litany of Instrumentarski:
If believing that there is a diamond in the box lets me find the diamond in the box,
I desire to believe that there is a diamond in the box;
If believing that there is a diamond in the box leaves me with an empty box,
I desire to believe that there is no diamond in the box;
Let me not become attached to inaccurate beliefs.
Posting it here in a hope that someone else also finds it more palatable and unassuming than straight-up realism.
EDIT: It seems to me that this modification also guides you to straight-up one-box on Newcomb, where the original one is mired in the EDT vs CDT issues.
EDIT2: Looks like the above version resulting in people confusing desiring accurate beliefs with desiring diamonds. It's about accurate accounting, not about utility of a certain form of crystallized carbon.
Maybe the first line should be modified to something like "If I later find a diamond in the box...", or something. How about the following?
If I will find a diamond in the box,
I desire to believe that I will find a diamond in the box;
If I will find no diamond in the box,
I desire to believe that I will find no diamond in the box;
Let me not become attached to inaccurate beliefs.
For some reason the editor does not let me use the <strike> tag to cross out the previous version, not sure how to work around it.