Stuart_Armstrong comments on Siren worlds and the perils of over-optimised search - Less Wrong

27 Post author: Stuart_Armstrong 07 April 2014 11:00AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Article Navigation

Comments (411)
Tags: constraints optimisation unconstrained deceit manipulation optimization concealment ai risk search

Comments (411)

You are viewing a single comment's thread. Show more comments above.

Comment author: Stuart_Armstrong 17 April 2014 11:19:08AM 0 points [-]

It won't intentionally hide anything dark under there, but it will spend way too much effort on things that make the world look good.

That's a marketing world, not a sire world.

Comment author: DanielLC 17 April 2014 06:09:04PM 0 points [-]

What's the difference?

Comment author: Stuart_Armstrong 18 April 2014 08:51:30AM 0 points [-]

Siren worlds are optimised to be bad and hide this fact. Marketing worlds are optimised to appear good, and the badness is an indirect consequence of this.

Powered by Reddit Powered by Reddit