Kaj_Sotala comments on Open Thread: September 2011 - Less Wrong

5 Post author: Pavitra 03 September 2011 07:50PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (441)

You are viewing a single comment's thread.

Comment author: Kaj_Sotala 04 September 2011 09:59:09AM *  16 points [-]

I'm getting increasingly pessimistic about technology.

If we don't get an AI wiping us out or some form of unpleasant brain upload evolution, we'll get hooked by superstimuli and stuff. We don't optimize for well-being, we optimize for what we (think we) want, which are two very different things. (And often, even calling it "optimization" is a stretch.)

Comment author: [deleted] 04 September 2011 10:22:20AM 8 points [-]

We don't optimize for well-being, we optimize for what we (think we) want, which are two very different things.

Natural selection does not cease operation. Say, for example, that someone invents a box that fully reproduces in every respect the subjective experience of eating and of having eaten by directly stimulating the brain. Dieters would love this device. Here's a device that implements in extreme form the very danger that you fear. In this case, the specific danger is that you will stop eating and die.

So the question is, will the device wipe out the human race? Almost certainly it will not wipe out the entire human race, simply because there are enough people around who would nevertheless choose to eat despite the availability of the device, possibly because they make a conscious decision to do so. These people will be the survivors, and they will reproduce, and their children will have both their values (transmitted culturally) and their genes, and so will probably be particularly resistant to the device.

That's an extreme case. In the actual case, there are doubtless many people who are not adapting well to technological change. They will tend to die out disproportionately, will tend to reproduce disproportionately less.

We have a model of this future in today's addictive drugs. Some people are more resistant to the lure of addictive drugs than others. Some people's lives are destroyed as they pursue the unnatural bliss of drugs, but many people manage to avoid their fate.

Many people have so far managed the trick of pursuing super stimuli without destroying their lives in the process.

Comment author: Eugine_Nier 07 September 2011 07:46:24AM 19 points [-]

Keep in mind, it's possible to evolve to extinction.

Comment author: smk 07 September 2011 02:36:02PM 0 points [-]

I wish I could upvote that more than once.

Comment author: wedrifid 07 September 2011 02:47:49PM 0 points [-]

The post or the comment? If the former then you just prompted me to vote it up for you. :)

Comment author: Will_Newsome 10 September 2011 05:09:42PM 3 points [-]

Me too. smk, your wish has been granted.

Comment author: [deleted] 12 September 2011 10:46:43PM *  4 points [-]

What struck me about the example in this post that its basically genetically equivalent to reliable easy to use contraception.

And now that I think about it humanity basically is like a giant petri dish where someone dumped some antibiotics. The demographic transition is a temporary affair, a die off of maladapted genotypes and memeplexes.

Comment author: nerzhin 05 September 2011 04:18:55PM 3 points [-]

It is not at all clear that the people resistant to addictive drugs are reproducing at a higher rate than those who aren't.

Comment author: Kaj_Sotala 04 September 2011 04:31:05PM 3 points [-]

Sure, I don't think humanity is in any danger of being destroyed by conventional technologies, and I'm pretty sure the Singularity will be happen - in one form or another - way before then. But there may very well be a lot of suffering on the way.

Comment author: Will_Newsome 10 September 2011 05:11:30PM *  0 points [-]

Have you checked out CFAI? It's like CEV but with less of an emphasis on humans. I really don't like humans and would rather only deal with them via implicit meta-level 'get information about morality from your environment' means, which is more explicit in CFAI than CEV.

Comment author: Kaj_Sotala 10 September 2011 07:17:52PM 0 points [-]

I've read part of it, though not all. (I'm a bit confused as to how your comment relates to mine.)

Comment author: Will_Newsome 10 September 2011 10:57:47PM *  0 points [-]

CEV takes more of an economic perspective where agent-extrapolations make deals with each other. The "good" agent-extrapolations might win out in the end (due to having a more-timeless discount rate, say), but there might be a lot of suffering along the way. CFAI on the other hand takes a less deal-centric perspective where the AI's more directly supposed to reason everything through from first principles, which can avoid predictably-stupid-in-retrospect agents getting much of the future's pie, so to speak. So I'm more afraid of CEV-like thinking than CFAI-like thinking, even though both are scary, because I am more afraid of humans being evil than I'm afraid of me not getting what I want. This may or may not overlap at all with your concerns.

(The difference isn't necessarily whether or not they converge on the same policy, it might also be how quickly they converge on that policy. CFAI seems like it'd converge on justifiedness more quickly, but maybe not.)

Comment author: Iabalka 05 September 2011 02:14:51PM 1 point [-]

Are you suggesting to leave everything to natural selection? Doesn't strike me as the rationalists' way.

Comment author: [deleted] 04 September 2011 02:38:24PM 3 points [-]

Are there particular technologies (or uses of) that have especially earned your pessimism?

Comment author: Kaj_Sotala 04 September 2011 04:56:10PM 12 points [-]

Lots of things, but some off the top of my head:

Communication technologies probably top the list. Sure, the Internet has given birth to lots of great communities, like the one where I'm typing this comment. But it has also created a hugely polarized environment. (See the picture on page 4 of this study.) It's ever easier to follow your biases and only read the opinions of people who agree with you, and to think that anyone who disagrees is stupid or evil or both. On one hand, it's great that people can withdraw to their own subcultures where they feel comfortable, but the groupthink that this allows...

"Television is the first truly democratic culture - the first culture available to everybody and entirely governed by what the people want. The most terrifying thing is what people do want." -- Clive Barnes. That's even more true for the Internet.

Also, it's getting easier and easier to work, study and live for weeks without talking to anyone else than the grocery store clerk. I don't think that's a particularly good thing from a mental health perspective.

Comment author: Alex_Altair 05 September 2011 07:31:06PM 3 points [-]

I gain great confidence from the principle that rational people win, on average. It is rational people that make the world, and if it gets to be something we don't want, we change it. The only real threat is rationalists with different utility functions (e.g. Quirrelmort).

(Disclaimer: please don't take this as a promotion of an "us/them" dichotomy.)

Comment author: Vaniver 04 September 2011 09:31:17PM 2 points [-]

Also, it's getting easier and easier to work, study and live for weeks without talking to anyone else than the grocery store clerk. I don't think that's a particularly good thing from a mental health perspective.

Talking with your mouth, or talking? Because it's not clear to me that talking online is significantly worse than talking in person at sustaining mental health. I suspect getting a girlfriend/boyfriend will do more for your mental health and social satisfaction than interacting with people face-to-face more.

Comment author: Kaj_Sotala 05 September 2011 12:34:04PM 4 points [-]

Personally I find that if I don't hang out with people in real life every 2-4 days I will get increasingly lethargic and incapable of getting anything done. To what degree this generalizes is another matter.

Comment author: Raemon 06 September 2011 11:08:20PM *  9 points [-]

I find the same thing as Kaj. I've started literally percieving myself as having that set of "needs" bars in the Sims. Bladder bar gets empty, and I need to use the toilet or I'll be uncomfortable. Sleep bar gets low, and I'll be tired until I get enough. Social bar (face to face time) gets low, and I'll feel bleah until I get some face to face time.

The good news is that I've noticed this, become able to distinguish between "not enough facetime Bleah" and other types of Bleah, and then make sure to get face-to-face time when I need it.

Comment author: [deleted] 07 September 2011 03:34:28AM *  2 points [-]

It's spooky how similar I am in this regard.

The good news is

What's the bad news?

Comment author: Raemon 07 September 2011 03:29:10PM 0 points [-]

That up until recently the internet (and a wide array of other neural-reward-generating things) made it very easy to NOT notice this and distinguish between various types of mental lethargy.

Comment author: [deleted] 06 September 2011 09:36:06PM 1 point [-]

Very much the same way. The internet has been a mixed blessing -- it allowed me to have the life I have at all, way back when, but now it's also a massive hook for akrasia and encourages sub-optimal use of free time. I'm still trying to get that under control.

Comment author: SilasBarta 06 September 2011 05:19:30PM 2 points [-]

If you mean a face-to-face bf/gf, you're not actually disagreeing with Kaj. Also, I concur with his points about social deprivation leading to lethargy, based on personal experience.

Comment author: luminosity 04 September 2011 11:27:35PM 1 point [-]

I've been working from home for a year now. I don't get out and see people often, my family live far away, so I don't have many opportunities to see people in person. The exception is, my brother is staying with me while he studies at University. There have been a few periods however where he's been away up with our parents, or off at a different university in a different state. I have a few friends I talk with regularly online through IM, and it helps, but the periods when my brother was away were still very difficult and I was getting very stressed towards the end, even though we don't interact all that much on a day to day basis, and even though I've always been much more tolerant and even thriving on loneliness than most people I know.

Maybe video chatting with people would be an adequate substitute? I haven't tried that, but my anecdote is that IM / talking online alleviates some of the stress, but goes nowhere near to mitigating it.

Comment author: kurokikaze 19 September 2011 04:40:19PM *  0 points [-]

Sorry, but isn't this the criticism of inappropriate use of technologies rather than technologies itself?

Comment author: Erebus 19 September 2011 05:42:26PM 2 points [-]

What would be the point of criticizing technology on the basis of its appropriate use?

Technologies do not exist in a vacuum, and even if they did, there'd be nobody around to use them. Thus restricting to only the "technology itself" is bound to miss the point of the criticism of technology. When considering the potential effects of future technology we need to take into account how the technologies will be used, and it is certainly reasonable to believe that some technologies have been and will be used to cause more harm than good. That a critical argument takes into account the relevant features of the society that uses the technology is not a flaw of the argument, but rather the opposite.

Comment author: kurokikaze 20 September 2011 01:59:12PM 0 points [-]

No, I'm not talking about the basis to criticize technology, but more about of actual target of criticism. Disclaimer: there sure are technologies that can do more harm than good. Here I will concentrate on communications, as you picked it as being one of the top problematic technologies.

For me, it all boils down to constructive side of criticism: should we change the technologies of the way we use them? Because I think in first case, new technologies will be used with the same drawbacks for humans as old ones. In the second case, successful usage patterns can be applied to new technologies as well.

For example, rather than limit the usage of communication technologies or change the comm technology itself, maybe we should focus on how the people use them. Make television more social. Or make going out with other people more easy and fun. Promote social interaction and activities using existing technologies, not relying on some magic future technology that will solve the existing problems. I think building the solution around existing technologies is a faster way than waiting for new ones.

Surely, there are technology side and social/culture side of the problem. But we cannot change any of these fast. We can only expand one to help the other. For example, on one programming site, around two years after its creation, people started to organize meetups in local places, much like LW meetups. Then, year later, other group on the site organized soccer games between different site users. The people liked it. And it doesn't take much time because they were building around existing stuff.

Also, sorry for my english. It's not my main language.

Comment author: Erebus 21 September 2011 07:40:48AM 1 point [-]

Maybe I misinterpreted your first comment. I agree almost completely with this one, especially the part

(...) not relying on some magic future technology that will solve the existing problems.

Comment author: wedrifid 04 September 2011 10:17:43AM 3 points [-]

We don't optimize for well-being, we optimize for what we (think we) want, which are two very different things. (And often, even calling it "optimization" is a stretch.)

You think we optimize for what we think we want? That's a stretch in itself. ;)

(Totally agree with what you are saying!)

Comment author: Thomas 05 September 2011 05:50:02PM 0 points [-]

I, on the contrary, remain a techno optimist, even more so.

It's a kind of sad, that so many clever people here are losing their confidence into the techno progress. Well, maybe not sad, but it certainly means, that they are not onto something big themselves.