eli_sennesh comments on Siren worlds and the perils of over-optimised search - Less Wrong

27 Post author: Stuart_Armstrong 07 April 2014 11:00AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Article Navigation

Comments (411)
Tags: constraints optimisation unconstrained deceit manipulation optimization concealment ai risk search

Comments (411)

You are viewing a single comment's thread. Show more comments above.

Comment author: [deleted] 01 May 2014 07:44:42AM -2 points [-]

That is assuming that we are capable of programming a strong AI to do any one thing instead of another, but if we cannot do that then the entire discussion seems to me to be moot.

And therein lies the rub. Current research-grade AGI formalisms don't actually allow us to specifically program the agent for anything, not even paperclips.

Comment author: PhilosophyTutor 01 May 2014 11:49:29AM 0 points [-]

If I was unclear, I was intending that remark to apply to the original hypothetical scenario where we do have a strong AI and are trying to use it to find a critical path to a highly optimal world. In the real world we obviously have no such capability. I will edit my earlier remark for clarity.

Powered by Reddit Powered by Reddit