Question: Regardless of the degree to which this is true, if everyone collectively assumed that Valence Utilitarianism (every conscious experience has value (positive or negative, depending on pleasantness/unpleasantness), each action's utility is the sum of all value it causes / changes / prevents) was universally true, how much would that change about Friendly AI research?
It would surely create new failure modes for Friendliness: e.g. kill all humans to spawn a trillion barely sentients copy of an AI.
But on the other hand I would say: not much. You'll still have the hard problems of instilling a very fragile ontology (what is a conscious being, from the bottom up?), analyzing the consequences of the actions performed by the AI, judging carefully which values it is fulfilling or breaking and how much, etc.
If it's worth saying, but not worth its own post, then it goes here.
Notes for future OT posters:
1. Please add the 'open_thread' tag.
2. Check if there is an active Open Thread before posting a new one. (Immediately before; refresh the list-of-threads page before posting.)
3. Open Threads should start on Monday, and end on Sunday.
4. Unflag the two options "Notify me of new top level comments on this article" and "