If all it takes to ensure FAI is to instruct "henceforth, always do what humans mean, not what they say" then FAI is trivial.

Is there an effective way for a layman to get serious feedback on scientific theories?

I have a weird theory about physics. I know that my theory will most likely be wrong, but I expect that some of its ideas could be useful and it will be an interesting learning experience even in the worst case. Due to the prevalence of crackpots on the internet, nobody will spare it a glance on physics forums because it is assumed out of hand that I am one of the crazy people (to be fair, the theory does sound pretty unusual).

The most striking problem with this paper is how easy all of the tests of viability they used are to game. There are a bunch of simple tests you can do to check for viability, and it's fairly common for non-viable tissue to produce decent-looking results on at least a couple, if you do enough. (A couple of weeks ago, I was reading a paper by Fahy which described the presence of this effect in tissue slices.)

It may be worth pointing out that they only cooled the hearts to -3 C, as well.

Not relevant to cryonics. "Super-cooling" is not a neologism, it means that the water didn't freeze when they cooled the organs down to -3 degrees C. This is not extendable to lower temperatures.

Music to be resurrected to?

Assume that you are going to die, and some years later, be brought back to life. You have the opportunity to request, ahead of time, some of the details of the environment you will wake up in. What criteria would you use to select those details; and which particular details would meet those criteria?

For example, you might wish a piece of music to be played that is highly unlikely to be played in your hearing in any other circumstances, and is extremely recognizable, allowing you the opportunity to start psychologically dealing with your new circumstances before you even open your eyes. Or you may just want a favourite playlist going, to help reassure you. Or you may want to try to increase the odds that a particular piece survives until then. Or you may wish to lay the foundation for a practical joke, or a really irresistible one-liner.

Make your choice!

And in that case U(++,-) doesn't imply that forcing people on the drug increases utility.

This is only true for simple systems - with more complications you can indeed sometimes deduce causal structure!

Suppose you have three variables: Utopamine conentration, smiling, and reported happiness. And further suppose that there is an independent noise source for each of these variables - causal nodes that we put in as a catch-all for fluctuations and external forcings that are hard to model.

If Utopamine is the root cause of both smiling and reported happiness, then the variation in happiness will be independent of the variation in smiling, conditional on the variation in Utopamine. But conditional on the variation in smiling, the variation in utopamine and reported happiness will still be correlated!

The AI can now narrow down the causal structure to 2, and perhaps it can even figure out the right one if there's some time lag in the response and it assumes that causation goes forward in time.

If much effort should be invested in the initial search for hypotheses/explanations, before they are weighed against each other, then how come there are apparently so few cases where more than two major hypotheses are proposed?

I mean, I don't know much about the history of physics, but I do remember being surprised by the (relatively) many models of the Structure of the Atom we heard about in chronological order. And there used to be lots more Trees of Life, back in the XIXth century. But I cannot, on the fly, think of crazy-but-who-knows things of today (well, except for the Search for Ancestors of Angiosperms, it just goes on).

Can you explain why that's a misconception? Or at least point me to a source that explains it?

I've started working with neural networks lately and I don't know too much yet, but the idea that they recreate the generative process behind a system, at least implicitly, seems almost obvious. If I train a neural network on a simple linear function, the weights on the network will probably change to reflect the coefficients of that function. Does this not generalize?

Eliezer has recommended that one read them twice. I found this helpful.