You're looking at Less Wrong's discussion board. This includes all posts, including those that haven't been promoted to the front page yet. For more information, see About Less Wrong.

Viliam_Bur comments on A Somewhat Vague Proposal for Grounding Ethics in Physics - Less Wrong Discussion

-3 Post author: capybaralet 27 January 2015 05:45AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (15)

You are viewing a single comment's thread.

Comment author: Viliam_Bur 27 January 2015 08:19:51AM *  0 points [-]

In AI research, intelligent agents typically have a clear-cut and well-defined final goal, e.g., win the chess game or drive the car to the destination legally. The same holds for most tasks that we assign to humans, because the time horizon and context is known and limited. (...) a truly well-defined goal would specify how all particles in our Universe should be arranged at the end of time.

We typically only care about the arrangement of particles at the end of the task, because that is the nature of the simple tasks we usually use machines for today. Actually, even that is not true: when "driving the car to destination legally" we care not only about the arrangement of the particles of the car at the end of the trip, but also about what happened on the way -- that's what "legally" means here. (Unless we also count "police sending us tickets" as particles. But I guess the car is supposed to follow the laws even when the police does not look.)

We can define "journey" goals e.g. by calculating score at each time interval, and trying to maximize the sum or the average (or some other function) of all the intervals. This can make sense even if we don't know how long the task will last.

treat experience as inherently positive and not try to distinguish between positive and negative experiences.

This sounds wrong. But I am not even sure what exactly would we measure here, if both positive and negative experience count the same. Is it the intensity of the experience (in either direction) which counts? (That is, would you rather be tortured than bored? Would you rather be tortured really painfully than enjoying a mild pleasure?) Or is it duration of the experience? (That is, we want to maximize the subjective time of sentient beings, regardless of what happens during the time? Would you rather live 1001 years in hell than 1000 years in heaven?)

Comment author: capybaralet 28 January 2015 02:59:44AM *  0 points [-]

This sounds wrong.

Of course. That's why I proposed refining it.

But I am not even sure what exactly would we measure here

I thought it was obvious. It is the integral of total experience (suitably defined) through time that counts.

Comment author: 9eB1 27 January 2015 09:09:13AM 0 points [-]

During the end of a drive there would either be/not be a configuration of particles in the shape of a paper ticket memorializing your transgression of the law. And if not that, there is a configuration of particles in the heads of the law enforcement officials recalling your transgression and planning on writing you a ticket or whatever. Any universe-wide configuration of particles contains the history of all of the events preceding it, even if they are opaque to us, because the possibility space of particle configurations is (probably?) larger than utility relevant histories.

Comment author: capybaralet 28 January 2015 03:01:56AM 0 points [-]

Any universe-wide configuration of particles contains the history of all of the events preceding it

So there is no way that we can arrive at the same state from different starting points? That seems ridiculous to me.

Comment author: 9eB1 28 January 2015 05:46:44AM 0 points [-]

I'm talking about particles at the quantum level here. Subatomic particles are ridiculously small. The amount of empty space in the universe is incomprehensibly vaster than the amount of particles that inhabit it, so it wouldn't be surprising to me if it were impossible to arrive at the same universe-wide state from different starting points, but I don't know that that's true as a matter of fact. But if there were a perfectly symmetrical perturbation, by definition it would be unobservable to us, since we would end up in the exact same state along either pathway.