TheAncientGeek comments on Natural selection defeats the orthogonality thesis - Less Wrong

-13 Post author: aberglas 29 September 2014 08:52AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (71)

You are viewing a single comment's thread. Show more comments above.

Comment author: Gunnar_Zarncke 29 September 2014 09:27:04PM *  4 points [-]

I don't understand why this post is so clearly down-voted. I thinks its main point

Instead this post argues that there is one and only one super goal for any agent, and that goal is simply to exist in a competitive world. Our human sense of other purposes is just an illusion created by our evolutionary origins.

is quite valid if steelmaned by

1) not assuming that every AGI automatically prevents value drift and mutation and

2) goal is not taken literally but in the same sense as genes have the main function of reproduction (existence of gene copies).

My understanding of AGI mechanics is that in general AGIs are subject to evolution. Could be that drift-preventing AGI win in the long run. But maybe there are just too few of them.

Note: I think one should consider the main part of the post only and not the provided and lengthy extracts; these should better have been linked to.

Comment author: TheAncientGeek 29 September 2014 11:08:38PM 0 points [-]

I don't understand why this post is so clearly down-voted.

It's vaguely anti MIRI?

Comment author: aberglas 30 September 2014 07:52:40AM *  1 point [-]

The post was not meant to be anti-anything. But it is a different point of view from that posted by several others in this space. I hope many of the down voters take the time to comment here.

One thing that I would say is that while it may not be the best post ever posted to less wrong, it is certainly not a troll. Yet one has to go back over 100 posts to find another article voted down so strongly!

Comment author: ChristianKl 01 October 2014 01:12:18PM 0 points [-]

The most upvoted post on LW is anti MIRI. You don't get down-voted on LW just because you are contrarian.

Comment author: TheAncientGeek 01 October 2014 03:14:32PM 1 point [-]

Indeed not, pre existing status is important as well.

Comment author: RobbBB 30 November 2014 02:04:47AM 1 point [-]

Suppose Holden Karnofsky had written the exact post above ("Natural selection defeats the orthogonality thesis") and some unknown had written the substantive points from Holden's critique (minus the GiveWell-specific stuff). What karma values would you expect for those two posts?