ChristianKl comments on MIRI strategy - Less Wrong

5 Post author: ColonelMustard 28 October 2013 03:33PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (94)

You are viewing a single comment's thread. Show more comments above.

Comment author: BaconServ 28 October 2013 08:53:46PM -2 points [-]

Letting plants grow their own pesticides for killing of things that eat the plants sounds to me like a bad strategy if you want healthy food.

Is there reason to believe someone in the field of genetic engineering would make such a mistake? Shouldn't someone in the field be more aware of that and other potential dangers, despite the GE FUD they've no doubt encountered outside of academia? It seems like the FUD should just be motivating them to understand the risks even more—if for no other reason than simply to correct people's misconceptions on the issue.

Your reasoning for why the "bad" publicity would have severe (or any notable) repercussions isn't apparent.

If you have a lot of people making bad arguments for why UFAI is a danger, smart MIT people might just say, hey those people are wrong I'm smart enough to program an AGI that does what I want.

This just doesn't seem very realistic when you consider all the variables.

Comment author: ChristianKl 28 October 2013 09:23:36PM -1 points [-]

Is there reason to believe someone in the field of genetic engineering would make such a mistake?

Because those people do engineer plants to produce pesticides? Bt Potato was the first which was approved by the FDA in 1995.

The commerical incentives that exist encourage the development of such products. A customer in a store doesn't see whether a potato is engineered to have more vitamins. He doesn't see whether it's engineered to produce pesticides.

He buys a potato. It's cheaper to grow potatos that produce their own pesticides than it is to grow potatos that don't.

In the case of potatos it might be harmless. We don't eat the green of the potatos anyway, so why bother if the green has additional poison? But you can slip up. Biology is complicated. You could have changed something that also gets the poison to be produced in the edible parts.

It seems like the FUD should just be motivating them to understand the risks even more

It's not a question of motivation. Politics is the mindkiller. If a topic gets political people on all sides of the debate get stupid.

This just doesn't seem very realistic when you consider all the variables.

According to Eliezer it takes strong math skills to see how an AGI can overtake their own utility function and is therefore dangerous. Eliezer made the point that it's very difficult to explain to people who are invested into their AGI design that it's dangerous because that part needs complicated math.

It easy to say in abstract that some AGI might become UFAI, but it's hard to do the assessment for any individual proposal.