Kaj_Sotala comments on Moloch: optimisation, "and" vs "or", information, and sacrificial ems - Less Wrong

20 Post author: Stuart_Armstrong 06 August 2014 03:57PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (58)

You are viewing a single comment's thread.

Comment author: Kaj_Sotala 07 August 2014 01:00:21PM *  7 points [-]

As I mentioned in the comments of Scott's post, I've been thinking about turning my "Technology will destroy human nature" essay into a formal paper, but I'd probably need someone more familiar with evolutionary biology as my co-author to make sure that the analogies to evolution make sense and to otherwise help develop it. TWDHN was basically talking about various physical limits that are currently stopping us from racing to the bottom but which technology will eventually overcome. Now that I've read Meditations, I might want the paper to discuss the other things holding Moloch in check that Scott talks about (excess resources, utility maximization, coordination), too.

Comment author: Stuart_Armstrong 07 August 2014 02:35:41PM 4 points [-]

We may make use of you and your ideas :-)

Comment author: atorm 07 August 2014 10:43:21PM 5 points [-]

I'm a biologist looking for co-authorships.

Comment author: Kaj_Sotala 17 August 2014 03:00:23PM 3 points [-]

Sorry for the late response! I was avoiding LW.

Here's a copy of an e-mail where I summarized the argument in TWDHN before, and suggested some directions for the paper (note that this was written before I had read the post on Moloch):

In a nutshell, the argument goes something like:

  1. Evolution adapts creatures to the regularities of their environment, with organisms evolving to use those regularities to their advantage.
  2. A special case of such regularities are constraints, things which an organism must adapt to even though it may be costly: for example, very cold weather forces an organism to spend a part of its energy reserves on growing a fur or other forms of insulation.
  3. If a constraint disappears from the environment, evolution will gradually eliminate the costly adaptations that developed in response to it. If the Arctic Circle were to become warm, polar bears would eventually lose their fur or be outcompeted by organisms that never had a thick fur in the first place.
  4. Many fundamental features of human nature are likely adaptations to various constraints: e.g. the notion of distinct individuals and personal identity may only exist because we are incapable of linking our brains directly together and merging into one vast hive mind. Conscious thought may only exist because consciousness acts as an "error handler" to deal with situations where our learned habits are incapable of doing the job right, and might become unnecessary if there was a way of pre-programming us with such good habits that they always got the job done. Etc.
  5. The process of technological development acts to remove various constraints in our environment: for example, it may one day become possible to actually link minds together directly.
  6. If technology does remove previous constraints from our environment, the things that we consider fundamental human values would actually become costly and unnecessary, and be gradually eliminated as organisms without those "burdens" would do better.

What I'd like to do in the paper would be to state the above argument more rigorously and clearly, provide evidence in favor of it, clarify things that I'm uncertain about (Does it make sense to distinguish constraints from just regularities in general? Should one make a distinction between constraints in the environment and constraints from what evolution can do with biological cells?), discuss various possible constraints as well as what might eliminate them and how much of an advantage that would give to entities that didn't need to take them into account, raise the possibility of some of this actually being a good thing, etc. Stuff like that. :-)

Does that argument sound sensible (rather than something that represents a total misunderstanding of evolutionary biology) and something that you'd like to work on? Thoughts on how to expand it to take Moloch into account?

Also, could you say a little more about your background and amount of experience in the field?

Comment author: atorm 22 August 2014 03:02:27AM 2 points [-]

This argument seems like something I would need to think long and hard about, which I see as a good thing: it seems rare to me that non-trivial things are simple and apparent. I don't see any glaring misinterpretation of natural selection. I would be interested in working on it in a "dialogue intellectually and hammer out more complete and concrete ideas" sense. I'm answering this quickly in a tired state because I'm not on LW as much as I used to be and I don't want to forget.

I'm getting a PhD in a biological field that is not Evolution. Both this and my undergraduate education covered evolution because it underlies all the biological fields. I have one publication out that discusses evolution but is not actually specifically relevant to this topic. I'll happily share more detail in private communications if you can't find an explicitly evolutionary biologist.

Comment author: Kaj_Sotala 23 August 2014 05:04:28PM 2 points [-]

That sounds good to me. :-)

E-mail me at xuenay@gmail.com and we can talk about things in more detail?