Vaniver comments on The Value Learning Problem - LessWrong

16 Post author: So8res 29 January 2015 06:23PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Article Navigation

Comments (37)
Tags: miri

Comments (37)

You are viewing a single comment's thread. Show more comments above.

Comment author: Vaniver 30 January 2015 01:02:01AM *  1 point [-]

Such humans are Natural_Selection!Broken, but the point is that that's not Human!Broken.

Do you have a better example of what the algorithm feels like from the inside? (Pointing out that an example could be problematic seems less useful than supplying a fix also.)

Comment author: fubarobfusco 30 January 2015 03:44:40AM *  0 points [-]

Well, how about just generalizing it away from the politically pointy example?

Even a system smart enough to figure out what its creators intended is not thereby motivated to act accordingly: an individual human, upon learning that humans arose through evolution by natural selection, doesn't thereby decide to abandon any other personal values he or she might have had and adopt "maximize the number of descendants I have" as the only goal in life.

Although just as the original example is prone to political criticism, this one may be prone to the critique that, as a matter of fact, in the generations after the discovery of evolution, quite a few humans did attempt to adopt their interpretation of evolution's goals as the proper goals of humanity.

Comment author: MarkusRamikin 30 January 2015 09:44:29AM *  1 point [-]

The sex example is more concrete. This new one blurs the point.

For instance, it may be misread as saying that humans who don't draw that conclusion are somehow broken.

Your revised example is just as prone to that, isn't it?

Which makes me guess (I know, guessing is rude, sorry) that this isn't your real objection, and you're just reacting to the keyword "contraceptives".

Comment author: gjm 30 January 2015 12:45:50PM 4 points [-]

It doesn't look to me as if fubarobfusco's example is as prone to that problem.

With the original example:

  • There actually are people -- quite a lot of them -- who believe that (1) sex's natural purpose is reproduction, and that (2) because of #1 it is wrong to use contraception when having sex.
    • (They generally believe #1 on the grounds that "God made it so" rather than "natural selection made it so".)
  • If such a person reads the paper as currently written, they are likely to find the statement that "human beings [...] do not thereby conclude that contraceptives are abhorrent" as an attack on their reasoning; after all, they draw just such a deduction.
    • (Although not quite the same deduction, and actually a more defensible one given their premises: God is suppose to have wise and benevolent intentions, whereas natural selection is not.)
  • If someone who isn't in that category but does disapprove of contraception for politicoreligious reasons reads the paper as currently written, they may go through roughly the process of reasoning described above on behalf of their political/religious allies and get offended.
  • This would be a shame.

With fubarobfusco's modified version:

  • There are way way fewer people who consider themselves obliged to maximize their number of descendants.
    • (Whether you imagine them basing that on natural selection, or divine design, or whatever else.)
  • Those people, as well as being few in number, are not a group with much political influence or social status.
  • There is accordingly much less danger that readers will either be offended on their own behalf, or take offence on others' behalf.

Perhaps fubarobfusco is "reacting to the keyword 'contraceptives'" in the following sense: he sees that word, recognizes that there is a whole lot of political/religious controversy around it, and feels that it would be best avoided. I'm not sure there's anything wrong with that.

Comment author: MarkusRamikin 30 January 2015 02:18:13PM 3 points [-]

Hm. Yeah, point taken, though I'd probably have to be American to be able to take this seriously on a gut level.

Still, the original example was clearer. It had a clear opposition, Bad according to genes, Good according to humans (even if not all of them). The modified example would lose that, as people generally do leave, and want to leave, descendants. It doesn't convey that sense of a sharp break with the "original intention".

Can't seem to think of an equally strong example that would be less likely to be objectionable...

Comment author: [deleted] 31 January 2015 01:36:48PM *  0 points [-]

Although just as the original example is prone to political criticism, this one may be prone to the critique that, as a matter of fact, in the generations after the discovery of evolution, quite a few humans did attempt to adopt their interpretation of evolution's goals as the proper goals of humanity.

Yes, but that's because most human beings interpret normative force as being a command coming from an authority figure, and vice-versa. Let them hallucinate an authority figure and they'll think there's reason to do what It says.

Powered by Reddit Powered by Reddit