But if human sociopaths lack sympathy that doesn't prevent US from having sympathy for THEM at all. Likewise, it's not at all obvious that we CAN have sympathy for aliens with completely different cognitive architecture even if they have sympathy for one another. An octopus is intelligent, but if I worry about it's pain I think that I am probably purely anthropomorphizing.

"Even if 200 folks do the same sort of work in the same office, they don't do the exact same work, and usually that person wouldn't be there or be paid if no one thought their work made any difference."

Obviously Robin has never worked in a typical office environment. This is a GREAT example of the theoretical framework which he uses to model the world being grossly wrong and honestly is a great example of why no-one should be allowed a PhD in ANY social science without having spent at least 5 years in at least 3 different communities, jobs, and industries.

"I'd like to see a study confirming that. The Internet is more addictive than television and I highly suspect it drains more life-force."

On average, Americans spend far more hours watching TV than using the Internet. Obviously Eliezer's sample-set is severely biased when he makes causal statements about what's addictive. People who basically live on the internet will find that people THEY know are more addicted to internet than to TV.

"Though it's a side issue, what's even more... interesting.... is the way that our brains simply haven't updated to their diminished power in a super-Dunbarian world. We just go on debating politics, feverishly applying our valuable brain time to finding better ways to run the world, with just the same fervent intensity that would be appropriate if we were in a small tribe where we could persuade people to change things."

Actually Eliezer, this is national indoctrination. In Costa Rica people spend MUCH less time discussing better ways to run the world. In Kazakhstan they would look at you like you were crazy if you spent ANY time doing so. Things just are the way they are and no-one can know what that way is. People aren't even interested in knowing what is legal or illegal etc.

Not telling people about harmful side-effects that they don't ask about wasn't considered fraud when all the food companies failed to inform the public about Trans Fats, as far as I can tell. At the least, their management don't seem to be going to jail over it. Not even the cigarette executives are generally concerned about prison time.

Eliezer: There we totally agree, though I fear that many sub-fields of science are like philosophy in this regard. I think that these include some usual suspects like paraspychology but many others like the examples I gave such as the standard social science model or other examples like the efficient market hypothesis. Sadly, I suspect that much of medicine including some of the most important fields like cancer and AIDS research and nutrition also falls in this category.

Robin: I'm interested in why you think we should believe that sociologists know something but not that parapsychologists know something. What is your standard? Where do efficient marketers fit in? Elliot Wave theorists?

Eliezer: I'm profoundly unimpressed by most recent philosophy, but really, why is it that when we are talking about science you say "nobody knows what science knows" while in the analogous situation with philosophy you say "the mountains of philosophy are the foothills of AI"? If scientists debate group vs individual selection or the SSSM or collapse for ten times a hundred years that doesn't mean that the answers haven't been discovered. How does this differ from free will?

Yes, thanks Psy. That makes much more sense.

"With a good toolbox of nonperson predicates in hand, we could exclude all "model citizens" - all beliefs that are themselves people - from the set of hypotheses our Bayesian AI may invent to try to model its person-containing environment." After you excise a part of its hypothesis space is your AI still Bayesian?

ShardPhoenix: Yes. This is the same principle that says that credible confidentiality within a group can sometimes improve aggregate information flow and collective epistemology.

Tim Tyler: Human goals. I definitely do NOT want alien rationalists to be able to lie, but I doubt I have much choice regarding that. Also not transhuman children. There I might have some limited choice.

Eliezer: I certainly think that rationalists should practice telling truth more effectively as well as lie, and you admit that not lying enough makes people gullible, so it's mostly a matter of estimates of the magnitude of the relevant trade-offs here. I think that our disagreements are based on radically different models of social psychology. We disagree a great deal about the degree to which being known to sometimes lie reduces future credibility in the eyes of actual existent humans relative to being known to sometimes mislead without lying. I believe that being known to lie increases credibility somewhat relative to "wizards oath", while you think it greatly decreases it. I think that I know your reasons for your belief and that you don't know mine. I'm not sure whether you think that I know your reasons, and I'm not sure whether this difference in social psychological theory is the specific belief we disagree about. I'd like confirmation on whether you agree that this is our main point of disagreement. Also possibly a poll of the audience on the social psychology fact.

"I flinched away from that thought's implications, not so much because I feared superintelligent paternalism myself, but because I feared what other people would say of that position."

This is basically THE reason I always advocate increased comfort with lying. It seems to me that this fear of believing what they don't want to say if they only believe truth is the single largest seemingly removable barrier to people becoming rationalists at all, or passing that barrier, to becoming the best rationalists they can be.