Warrigal comments on Open Thread: March 2010, part 2 - Less Wrong

4 Post author: RobinZ 11 March 2010 05:25PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (334)

You are viewing a single comment's thread. Show more comments above.

Comment author: Vladimir_Nesov 11 March 2010 06:30:08PM *  4 points [-]

No. FAI is about figuring out how to implement precise preference, not an approximation of it appropriate for non-magical environments. Requires completely different tools.

It seems that to work on FAI, one has to become mathematician and theoretical computer scientist (whatever the actual career).

Comment author: [deleted] 11 March 2010 06:52:45PM 1 point [-]

What do you mean by "non-magical environments"?

Comment author: Vladimir_Nesov 11 March 2010 08:52:33PM *  2 points [-]

I gave a link! A non-magical environment gives limited expressive power, so there are few surprising situations that given heuristics don't capture. With enough testing and debugging, you may get your weakly intelligent robot to behave. Where more possibilities are open, you have to get preference exactly, or the decisions will be obviously wrong (see The Hidden Complexity of Wishes).

Comment author: RobinZ 11 March 2010 11:23:29PM 1 point [-]

Your terminology was unclear but this definition is not - I would tend to call it an "organic" environment.