hairyfigment comments on An overall schema for the friendly AI problems: self-referential convergence criteria - Less Wrong

17 Post author: Stuart_Armstrong 13 July 2015 03:34PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (110)

You are viewing a single comment's thread. Show more comments above.

Comment author: [deleted] 16 July 2015 05:11:29AM *  2 points [-]

What do I mean by that? Well, imagine you're trying to reach reflective equilibrium in your morality. You do this by using good meta-ethical rules, zooming up and down at various moral levels, making decisions on how to resolve inconsistencies, etc... But how do you know when to stop? Well, you stop when your morality is perfectly self-consistent, when you no longer have any urge to change your moral or meta-moral setup.

Wait... what? No.

You don't solve the value-alignment problem by trying to write down your confusions about the foundations of moral philosophy, because writing down confusion still leaves you fundamentally confused. No amount of intelligence can solve an ill-posed problem in some way other than pointing out that the problem is ill-posed.

You solve it by removing the need to do moral philosophy and instead specifying a computation that corresponds to your moral psychology and its real, actually-existing, specifiable properties.

And then telling metaphysics to take a running jump to boot, and crunching down on Strong Naturalism brand crackers, which come in neat little bullet shapes.

Comment author: hairyfigment 19 July 2015 05:32:38PM 0 points [-]

Near as I can tell, you're proposing some "good meta-ethical rules," though you may have skipped the difficult parts. And I think the claim, "you stop when your morality is perfectly self-consistent," was more a factual prediction than an imperative.

Comment author: [deleted] 20 July 2015 01:19:03PM 0 points [-]

I didn't skip the difficult bits, because I didn't propose a full solution. I stated an approach to dissolving the problem.

Comment author: hairyfigment 22 July 2015 06:00:14AM 0 points [-]

And do you think that approach differs from the one you quoted?

Comment author: [deleted] 22 July 2015 12:43:21PM 0 points [-]

It involves reasoning about facts rather than metaphysics.