TGGP2

Humans having this kind of tendency is a predictable result of what their design was optimized to do, and as such them having it doesn't imply much for minds from a completely different part of mind design space.
Eliezer seems to be saying his FAI will emulate his own mind, assuming it was much more knowledgeable and had heard all the arguments.

Then they don't know the true difference between the two possible lives, do they?
"True difference" gets me thinking of "no true Scotsman". Has there ever been anybody who truly knew the difference between two possible lives? Even if someone could be reincarnated and retain memories the order would likely alter their perceptions.

I'm very interested in how Eliezer gets from his meta-ethics to utilitarianism
He's not a strict utilitarian in the "happiness alone" sense. He has an aversion to wireheading, which maximizes the class... (read more)

I'm near Unknown's position. I don't trust any human being with too much power. No matter how nice they seem at first, history indicates to me that they inevitably abuse it. We've been told that a General AI will have power beyond any despot known to history. Am I supposed to have that much reliance on the essential goodness within Eliezer's heart? And in case anyone brings this up, I certainly don't trust the tyranny of the majority either. I don't recognize any moral obligation to stop it because I don't recognize any obligations at all. Also, I might no... (read more)

Hal Finney, I am reminded of Stephen Pinker's discussion of love between two individuals whose interests exactly coincide. He says that the two would come to form one organism, and they would be like multiple organs or cells within that individual organism, and so would not have to experience "love".

I was once impressed by the ability of natural selection to create incredibly complicated functioning living things that can even repair and make copies of themselves. I realized that this was the result of it having so much time and material to work with and relentlessly following an algorithm for attaining fitness that a human being with its biases would be apt to deviate from if consciously pursuing it, but I still felt impressed. I have never felt that way about beauty or emotion.

Since that's already what I believe, it wouldn't be a change at all. I must admit though that I didn't tip even when I believed in God, but I was different in a number of ways.

I think the world would change on the margin and that Voltaire was right when he warned of the servants stealing the silverware. The servants might also change their behavior in more desirable ways, but I don't know whether I'd prefer it on net and as it doesn't seem like a likely possibility in the foreseeable future I am content to be ignorant.

I would have expected that violent instability would increase the costs of resource extraction. That is explicitly part of the goal of MEND in Nigeria, which John Robb often discusses.

Sorry I'm late, but this is really a great opportunity to plug For the law, neuroscience changes nothing and everything.

Since when does science contain morality?

TGGP, are you familiar with the teachings of Jesus?
Yes, I was raised Christian and I've read the Gospels. I don't think they provide an objective standard of morality, just the jewish Pharisaic tradition filtered through a Hellenistic lens.

Matters of preference are entirely subjective, but for any evolved agent they are far from arbitrary, and subject to increasing agreement to the extent that they reflect increasingly fundamental values in common.
That is relevant to what ethics people may favor, but not to any truth or objective standard. Agreement among people is the result of subjective judgment.

Caledonian, what is an objective standard by which an ethical system can be evaluated?

Julian Morrison, William Saletan has suggested lowering the age of consent but states that people wouldn't think rationally about it. I discussed that here, and noted here a study showing that sex and pot don't screw kids up like people thought.

The Americans DID adopt mass marching tactics during the Revolutionary War. We even won battles that way!

Here is Wikipedia on the mistaken idea that the American Revolution was won by guerrilla tactics.

What did the Resistance accomplish? I already stated that it seemed to me that it was the opposing armies that got rid of the Nazis. If you disagree on that or have something else you think they did, state it.

About violence and society. What do we define by violence? Do we also define intrusion in our personal sphere, psychological re-programming, etc. as violent activities?
You should read Randall Collins.

What about the Resistance in countries that were occupied by Nazi Germany?
Did they actually accomplish anything? I think it was the violence of the opposing armies that actually ended Nazi occupation.

Should we measure the importance of the ideas discussed by citations?

I do not consider your rephrasing to be accurate. I wasn't giving the measurers choice, they are all supposed to follow the same procedure in order to obtain the same (probabilistic) results. It is the Mugger, or outside agent, that is making choices and therefore preventing the experiment from being controlled and repeatable.

What do you see as the major deficiencies in our model of reality? That the behavior of quantum particles is probabilistic rather than deterministic?

In quantum experiments the random outcomes are the same for all experimenters, so it can be repeated and the same probabilities will be observed. When you have someone else sending messages, you can't rely on them to behave the same for all experimenters. If there are a larger group of Muggers that different scientists could communicate with, than experiments might reveal statistical information about the Mugger class of entity (treating them as experimental subjects), but it's a stretch.

I don't think you've established that "You might as well consider it good", I might as well not consider it good or bad. You haven't given a reason to consider it more good than bad, just hope. I might hope my lottery ticket is a winner, but I have no reason to expect it to be.

If you want to persuade physicists to start looking for messages from beyond the space-time continuum, you'd better be able to offer them a method. I am completely at a loss for how one might go about it. I certainly don't know how you are going to do it at the blackboard. ... (read more)

Physicists have been proceeding like physicists for some time now and none of them has done anything like receiving the Old Testament from outside of our space-time. Why would you even expect a laboratory experiment to have such a result? It also seems you are postulating an extra-agent (the Mugger), which limits the amount of control experimenters have and in turn makes the experiment unrepeatable.