handoflixue comments on By Which It May Be Judged - Less Wrong

35 Post author: Eliezer_Yudkowsky 10 December 2012 04:26AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (934)

You are viewing a single comment's thread. Show more comments above.

Comment author: handoflixue 11 December 2012 09:36:46PM 5 points [-]

Ooh, good call, thank you. I suppose it might be akin to visualization, where it actually varies from person to person. Does anyone here on LessWrong have conflicting anecdotes, though? Does anyone disagree with what I said? If not, it seems like a safe generalization for now, but it's still useful to remember I'm generalizing from one example :)

Remembering that other people have genuinely alien minds is surprisingly tricky.

Comment author: Alicorn 11 December 2012 10:34:49PM 8 points [-]

The other side of this is that I would expect my brain to NOTICE it's actual goals. If my goal is to make paperclips, I will think "I should do this because it makes paperclips", instead of "I should do this because it makes people happy". My brain doesn't have a generic "I should do this" emotion, as near as I can tell - it just has ways of signalling that an activity will accomplish my goals.

Iron deficiency feels like wanting ice. For clever, verbal reasons. Not being iron deficient doesn't feel like anything. My brain did not notice that it was trying to get iron - it didn't even notice it was trying to get ice, it made up reasons according to which ice was an instrumental value for some terminal goal or other.

Comment author: shminux 11 December 2012 11:36:28PM 4 points [-]

Remembering that other people have genuinely alien minds is surprisingly tricky.

Other people? I find my own mind quite alien below the thin layer accessible to my introspection. Heck, most of the time I cannot even tell if my introspection lies to me.

Comment author: asparisi 12 December 2012 05:32:54PM 1 point [-]

I think I have a different introspection here.

When I have a feeling such as 'doing-whats-right' there is a positive emotional response associated with it. Immediately I attach semantic content to that emotion: I identify it as being produced by the 'doing-whats-right' emotion. How do I do this? I suspect that my brain has done the work to figure out that emotional response X is associated with behavior Y, and just does the work quickly.

But this is maleable. Over time, the emotional response associated with an act can change and this does not necessarily indicate a change in semantic content. I can, for example, give to a charity that I am not convinced is good and I still will often get the 'doing-whats-right' emotion even though the semantic content isn't really there. I can also find new things I value, and occasionally I will acknowledge that I value something before I get positive emotional reinforcement. So in my experience, they aren't identical.

I strongly suspect that if you reprogrammed my brain to value counting paperclips, it would feel the same as doing what is right. At very least, this would not be inconsistent. I might learn to attach paperclippy instead of good to that emotional state, but it would feel the same.

Comment author: MugaSofer 12 December 2012 11:11:05AM 1 point [-]

Remembering that other people have genuinely alien minds is surprisingly tricky.

... they do? For what values of "alien"?

Comment author: handoflixue 14 December 2012 06:59:00PM 1 point [-]

Because I'm not sure how else to capture a "scale of alien-ness":

I once wrote a sci-fi race that was a blind, deaf ooze, but extremely intelligent and very sensitive to tactile input. Over the years, and with the help of a few other people, I've gotten a fairly good feel for their mindset and how they approach the world.

There's a distinct subset of humans which I find vastly more puzzling than these guys.

Comment author: [deleted] 14 December 2012 10:21:10PM *  3 points [-]

From Humans in Funny Suits:

But the real problem is not shape, it is mind.  "Humans in funny suits" is a well-known term in literary science-fiction fandom, and it does not refer to something with four limbs that walks upright.  An angular creature of pure crystal is a "human in a funny suit" if she thinks remarkably like a human - especially a human of an English-speaking culture of the late-20th/early-21st century.

I don't watch a lot of ancient movies.  When I was watching the movie Psycho (1960) a few years back, I was taken aback by the cultural gap between the Americans on the screen and my America.  The buttoned-shirted characters of Psycho are considerably more alien than the vast majority of so-called "aliens" I encounter on TV or the silver screen.

Comment author: handoflixue 14 December 2012 10:30:48PM 1 point [-]

The race was explicitly designed to try and avoid "humans in funny suits", and have a culture that's probably more foreign than the 1960s. But I'm only 29, and haven't traveled outside of English-speaking countries, so take that with a dash of salt!

On a 0-10 scale, with myself at 0, humans in funny suits at 1, and the 1960s at 2, I'd rate my creation as a 4, and a subset of humanity exists in the 4-5 range. Around 5, I have trouble with the idea that there's coherent intelligent reasoning happening, because the process is just completely lost on me, and I don't think I'd be able to easily assign anything more than a 5, much less even speculate on what a 10 would look like.

Trying to give a specific answer to "how alien is it" is a lot harder than it seems! :)

Comment author: Eugine_Nier 16 December 2012 04:12:55AM 3 points [-]

The race was explicitly designed to try and avoid "humans in funny suits", and have a culture that's probably more foreign than the 1960s. But I'm only 29, and haven't traveled outside of English-speaking countries, so take that with a dash of salt!

Well reading fiction (and non-fiction) for which English speakers of your generation weren't the target audience is a good way to start compensating.

Comment author: handoflixue 17 December 2012 09:14:50PM 2 points [-]

I've got a lot of exposure to "golden age" science fiction and fantasy, so going back a few decades isn't hard for me. I just don't get exposed to many other good sources. The "classics" seem to generally fail to capture that foreignness.

If you have recommendations, especially a broader method than just naming a couple authors, I'd love to hear it. Most of my favourite authors have a strong focus on foreign cultures, either exploring them or just having characters from diverse backgrounds.

Comment author: Eugine_Nier 18 December 2012 04:07:37AM 1 point [-]

I've got a lot of exposure to "golden age" science fiction and fantasy, so going back a few decades isn't hard for me.

Which time period do you mean by this? "Golden age of science fiction" typically refers to the 1940's and 1950's, "golden age of fantasy" to the late 1970's and early 1980's. If you mean the latter time period, read stuff from the former as a start. Also try going back at least a century to the foundational fantasy authors, e.g., Edgar Rice Burroughs, William Morris's The Well at the World's End. Go even further back to things like Treasure Island, or The Three Musketeers. Or even further back to the days when people believed the stuff in their "fantasy" could actually happen. Read Dante's Divine Comedy, Thomas Moore's Utopia, an actual chivalric romance (I haven't read any so I can't give recommendations).

A good rule of thumb is that you should experience values dissonance while reading them. A culture whose values don't make you feel uncomfortable isn't truly alien. Also for this reason, avoid modern adaptations as these tend to do their best clean up the politically incorrect parts and otherwise modernize the worldview.

Comment author: beoShaffer 17 December 2012 09:58:27PM 1 point [-]

Anime&Manga, particularly the older stuff is a decent source.

Comment author: handoflixue 18 December 2012 12:55:20AM 0 points [-]

... it is really sad that I completely forgot that anime and manga isn't English. I grew up around it, so it's just a natural part of my culture. Suffice to say, I've had a lot of exposure -- but not to anything older than I am.

Any recommendations for OLD anime or manga, given I don't speak/read Japanese? :)

Comment author: beoShaffer 18 December 2012 02:57:24AM 0 points [-]

You're probably best of asking on a manga/forum, but Barefoot Gen is a good, and depressing, start.

Comment author: IlyaShpitser 14 December 2012 10:36:57PM *  3 points [-]

If I may make a recommendation, if you are concerned about "alien aliens", read a few things by Stanislaw Lem. The main theme of Lem's scifi, I would say, is alien minds, and failure of first contact. "Solaris" is his most famous work (but the adaptation with Clooney is predictably terrible).

Comment author: handoflixue 15 December 2012 01:07:21AM 0 points [-]

Not sure if I've read Lem, but I'll be sure to check it out. I have a love for "truly alien" science fiction, which is why I had to try my hand at making one of my own :)

Comment author: kodos96 20 December 2012 05:30:56AM 0 points [-]

I once wrote a sci-fi race that was a blind, deaf ooze, but extremely intelligent and very sensitive to tactile input.

I'm intrigued. Do you have a link?

Comment author: handoflixue 24 December 2012 07:01:09PM 0 points [-]

Sadly not. I really should do a proper write-up, but right now they're mostly stored in the head of me and their co-creator.