Is CFAR going to market themselves like this?

[at the workshop]:

"Look to the left of you, now to the right of you, now in 12 other directions. Only one of you will have a strong positive effect from this workshop."

1) Logical depth seems super cool to me, and is perhaps the best way I've seen for quantifying "interestingness" without mistakenly equating it with "unlikeliness" or "incompressibility".

2) Despite this, Manfred's brain-encoding-halting-times example illustrates a way a D(u/h) / D(u) optimized future could be terrible... do you think this future would not obtain, because despite being human-brain-based, would not in fact make much use of being on a human brain? That is, it would have extremely high D(u) and therefore be penalized?

I think it would be easy to rationalize/over-fit our intuitions about this formula to convince ourselves that it matches our intuitions about what is a good future. More realistically, I suspect that our favorite futures have relatively high D(u/h) / D(u) but not the highest value of D(u/h) / D(u).

I once asked a room full of about 100 neuroscientists whether willpower depletion was a thing, and there was widespread disagreement with the idea.

In which year did you do the asking?

So I found this paper by Gelman, King, and Boscodarin (1998)

The link is dead. Here's the paper.

Note that the last name of the third author is Boscardin, not Boscodarin.

Let's clear things up a little: you cannot use the category of "quantum random" to actual coin flip, because an object to be truly so it must be in a superposition of at least two different pure states, a situation that with a coin at room temperature has yet to be achieved (and will continue to be so for a very long time). So let's talk about classic randomness from a Bayesian point of view: when you have no prior information that can correlate with the outcome of an event. That's the case with the coin flip (and also with the quantum case, according to many-worlds interpretation).
Since the face landing depends not only on thumb movement but also on the exact starting position and the movement of air molecules, it's surely not possible for you to know all this informations in the beginning to a degree precise enough to deduce the side landing up. In this situation, your "throw the coin" motor impulse and the coin landing are uncorrelated, and so the coin flip is random (from your perspective).
But the degree to which the coin depends on factors you don't control is very low: if you practice enough, you can control the movement of your thumb so that it lands, say, 9 times out of 10 the side you want. In this case you have formed a better model of the coin traveling through the air and you have learned to control your thumb more precisely. In this case the correlation with your motor cortex is much higher and the coin flip is of course no more random.

I find it strange that you feel evolutionary causation is adequate to justify something, but I guess I won't question that.

Not justify: instead, explain. I understood that previously, handoflixue felt that status was dirty, but in understanding it has come to feel that it's just part of human nature (for most people, as the post points out).

If you're curious about someone else's emotions or perspective, first, remember that there are two ways to encode knowledge of how someone else feels: by having a description of their feelings, or by empathizing and actually feeling them yourself. It is more costly --- in terms of emotional energy --- to empathize with someone, but if you care enough about them to afford them that cost, I think it's the way to go. You can ask them to help you understand how they feel, or help you to see things the way they do. If you succeed, they'll appreciate having someone who can share their perspective.

My summary of this idea has been that life is a non-convex optimization problem. Hill-climbing will only get you to the top of the hill that you're on; getting to other hills requires periodic re-initializing. Existing non-convex optimization techniques are often heuristic rather than provably optimal, and when they are provable, they're slow.

skipping over the mechanisms for filtering good ideas from bad leaves me confused about the point of the post.

The point of the post is that most people, in most domains, should not trust that they are good at filtering good ideas from bad.

What if we also add a requirement that the FAI doesn't make anyone worse off in expected utility compared to no FAI? That seems reasonable, but conflicts the other axioms. For example, suppose there are two agents: A gets 1 util if 90% of the universe is converted into paperclips, 0 utils otherwise, and B gets 1 util if 90% of the universe is converted into staples, 0 utils otherwise. Without an FAI, they'll probably end up fighting each other for control of the universe, and let's say each has 30% chance of success. An FAI that doesn't make one of them worse off has to prefer a 50/50 lottery of the universe turning into either paperclips or staples to a certain outcome of either, but that violates VNM rationality.

And things get really confusing when we also consider issues of logical uncertainty and dynamical consistency.