How To Build a Computer.

Oh, interesting. So just as one should act as if one is Jesus if one seems to be Jesus, then one should act as if one cares about world-histories in proportion to their L2 measure if one seems to care about world-histories in proportion to their L2 measure and one happens to be in a world-history with relatively high L2 measure. And if probability is degree of caring, then the fact that one's world history obeys the Born rule is evidence that one cares about world-histories in proportion to their L2 measure.

I take it you would prefer option 2 in my original comment, reduce anticipation to UDT, and explain away continuity of experience.

Have I correctly characterized your point of view?

Hm, so you're saying that anticipation isn't a primitive, it's just part of one's decision-making process. But isn't there a sense in which I ought to expect the Born rule to hold in ordinary circumstances? Call it a set of preferences that all humans share — we care about futures in proportion to the square of the modulus of their amplitude (in the universal wavefunction? in the successor state to our Everett branch?). Do you have an opinion on exactly how that preference works, and what sorts of decision problems it applies to?

Hm, so you're saying that if |u> has high probability density in the subspace that contains Bob, then in the near future there must still be high probability density there, or at least nearby. But in fact |u> has very low probability density in Bob's Everett branch. Consider all the accidents of weather and history that led to Bob's birth, not to mention the quantum fluctuations that led to Bob's galaxy being created.

Conversely, any common and overused or commonly misused heuristic can also be used as a fallacy. Absurdity Fallacy, Affect Fallacy, Availability Fallacy. I probably use these far more than the original as-good-heuristic concept.

It is difficult to know which knowledge to preserve to which grade. I wish I could have a printed copy of the 'most important' Wikipedia articles.

At least I bought a Wikipedia DVD in case I'm offline but I'm not sure it will help much to restore civilitation in a real disaster.

Some things should be on paper. Some things should be on nickel like the LongNow foundation does. But which?

Actually I once wrote a proposal in the MetaPedia for that but guess what: I couldn't find it again the last time I looked for it. The whole structure had changed. We have no standardized way to tag digital documents to be more (or less) important than other documents (and such a relative classification which could be automatically resolved to a total order is much better than any fixed absolute scheme - guess why).

But a human who learns rationality already has those values, and rationality can help them understand those values better, decompartmentalize, and optimize more efficiently.

So? Let's say I value cleansing the Earth of untermenschen. Rationality can indeed help me achieve my goals and "optimize more efficiently". Once you start associating rationality with sets of values, I don't see how can you associate it with only "nice" values like altruism, but not "bad" ones like genocide.

While obviously not rigorous enough for something serious, one obvious hack is to do the "0.5 unless proven" thing, and then have a long list of special case dumb heuristics with different weights that update that without any proofs involved at all. The list of heuristics could be gotten from some unsafe source like the programer or another AI or mechanical turk, and then the weights learned by first guessing and then proving to see if it was right, with heuristics that are to bad kicked out entirely.

Xia, in anvil conversation: "What if you have the AIXI as a cartesian lump, and teach it that it's output can only influence a tiny voltage various sensitive sensors can sense, and that if the voltage to it is broken time skips forward until it's reinstated, and gives it a clock tick timeout death prior based on how long the universe has been running rather than how many bits it has outputted? The AI will predict that if it's destroyed the lump wont be found and the voltage nevrreaplied untill the universe spontaneously ceases to exist a few million years later"