At first I disbelieved. I thought A > B. Then I wrote code myself and checked, and got that B > A. I believed this result. Then I thought about it and realized why my reason for A > B was wrong. But I still didn't understand (and now I don't understand either) why the described random process is not equivalent to randomly choosing 2, 4, or 6 every roll. I thought some more and now I have some doubts. My first doubt is whether there exists some kind of standard way of describing random processes and conditioning on them, and whether the problem as stated by notfnofn. Perhaps the problem is just underspecified? Anyway, this is very interesting.
If you think you might be in a solipsist simulation, you might try to add some chaotic randomness to your decisions. For example, go outside under some trees and wait till any kind of tree leaf or seed or anything hits your left half of the face, choose one course of action. If it hits the other half of your face, choose another course of action. If you do this multiple times in your life, each of your decisions will depend on the state of the whole earth and on all your previous decisions, since weather is chaotic. And thus the simulators will be unable to get good predictions about you using a solipsist simulation. A potential counterargument is that they analyze your thinking and hardcode this binary random choice, i.e. hardcode the memory of the seed hitting your left side. But then there would need to be an intelligent process analyzing your thinking to try and isolate the randomness. But then you could make the dependence of your strategy on randomness even more complicated.
Nice. I have a suggestion how to improve the article. Put a clearly stated theorem somewhere in the middle, in its own block, like in academic math articles.
Why do you hate earworms? To me, they are mildly pleasant. The only moments when I wish I didn’t have an earworm happening at that moment is when I’m trying to remember another tune and the earworm for musicianship purposes and the earworm prevents me from being able to do that.
Instead of inspecting all programs in the UP, just inspect all programs with length less than n. As n becomes larger and larger, this covers more and more of the total probability mass in the up and the total probability mass covered this way approaches 1. What to do about the non-halting programs? Well, just run all the programs for m steps, I guess. I think this is the approximation of UP that is implied.
Well, now I'm wondering - is neural network training chaotic?
This is awesome, I would love more posts like this. Out of curiosity, how many hours have you and your colleague spent on this research.
In the very beginning of the post, I read: "Quick psychology experiment". Then, I read: "Right now, if I offered you a bet ...". Because of this, I thought about a potential real life situation, not a platonic ideal situation, that the author is offering me this bet. I declined both bets. Not because they are bad bets in an abstract world, but because I don't trust the author in the first bet and I trust them even less in the second bet.
If you rejected the first bet and accepted the second bet, just that is enough to rule you out from having any utility function consistent with your decisions.
Under this interpretation, no it doesn't.
Could you, the author, please modify the thought experiment to indicate that it is assumed that I completely trust the one who is proposing the bet to me? And, maybe discuss other caveats too. Or just say that it's Omega who's offering me the bet.
m - often used together with n to denote the height and width of a matrix