You're looking at Less Wrong's discussion board. This includes all posts, including those that haven't been promoted to the front page yet. For more information, see About Less Wrong.

KatjaGrace comments on Superintelligence 20: The value-loading problem - Less Wrong Discussion

4 Post author: KatjaGrace 27 January 2015 02:00AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Article Navigation

Comments (21)

Comments (21)

You are viewing a single comment's thread.

Comment author: KatjaGrace 27 January 2015 01:55:09AM 2 points [-]

I'm actually not clear on what exactly falls under the 'value loading problem'. These seem like somewhat separate issues:

  • Figuring out what we want in any sense (e.g. utilitarianism with lots of details nailed down)
  • Translating 'any sense' into being able to write down what we want in a formal way
  • Causing the values to be the motivations of an AI

Is the 'value loading problem' some subset of these?

Comment author: [deleted] 27 January 2015 02:35:39PM 5 points [-]

Figuring out what we want in any sense (e.g. utilitarianism with lots of details nailed down)

That's the "value problem"

Translating 'any sense' into being able to write down what we want in a formal way

That's the "formalization problem"

Causing the values to be the motivations of an AI

This is the "value loading problem"

Comment author: Luke_A_Somers 27 January 2015 11:20:19PM 2 points [-]

Well, then, it seems like almost all the difficulty is in the value and formalization problems. Once we've really formalized it, it's 99% of the way to machine code from where it started as human intuition.

Comment author: torekp 02 February 2015 08:38:43PM 2 points [-]

Doesn't that mean the value loading strategy is an alternative to the (direct) formalization strategy?

Comment author: Luke_A_Somers 03 February 2015 03:01:04PM 1 point [-]

Via, say, doubly-indirect meta-ethics? Well, we need to decide that that's really the decision algorithm that's going to result in the right answer, both that it's ethically correct and predictably converges on that ethically correct result.

Comment author: Houshalter 05 February 2015 10:50:53AM 0 points [-]

Explicitly figuring out what our values are and formalizing them, is only one possible sequence of steps to get AI with our values.

It seems like most people don't think that this approach will work. So there are a number of proposals to use AI itself to assist in this process. E.g. "motivational scaffolding" sounds like it solves the second step (formalizing the values.)

Powered by Reddit Powered by Reddit