Comment on Magical Categories

Thu, 23 Dec 2010 18:02:12 GMT

Hmmm, I can forsee many problems with guessing what humans "ought" to prefer. Even humans have got that one wrong pretty much every time they've tried.

I'd say a "better" goal might be cased as "increasing the options available to most humans (not at the expense of the options of other humans)"

This goal seems compatible with allowing humans to choose happier lifestyles - but without forcing them into any particular lifestyle that they may not consider to be "better".

It would "work" by concentrating on things like extending human lifespans and finding better medical treatments for things that limit human endeavour.

However, this is just a guess... and I am still only a novice here... which means I am in no way capable of figuring out how I'd actually go about training an AI to accept the above goal.

All I know is that I agree with Eliezer's post that the lab-rat method would be sub-optimal as it has a high propensity to fall into pathological configurations.

Discuss