I guess the difference is that disutility has a lower bound: 0. So there's a point where an expected disutility minimizer can actually stop, and it is always trying to get to that point.
It seems to me expected utility maximizers can never logically stop because utility has no upper bound. Am I wrong?
This inability to stop is a big part of why expected utility maximizers are creepy. Am I wrong?
I'm not even sure two utility maximizers can coexist peacefully at all. Two disutility minimizers could certainly get along unless their disutility functions overlap in specific ways.
Of course minimizing disutility, like maximizing utility, is extremely broad and most tasks could probably be described as either - including ones that go spectacularly wrong.
My stupid question is whether I'm overlooking something here. Because this inherent drive towards a point where inaction is okay seems like a great trait for future AIs to have and yet everybody keeps talking of maximizing expected utility.
Edit: clarity.
Why is 0 a lower bound for disutility? Suppose I make a machine that makes one person one Standard Happiness Unit happier than they'd otherwise have been and then stops and self-destructs; isn't that a disutility of -1 units?
If what you mean by minimizing disutility is that the machine tries not to cause harm on balance and doesn't care about any good it does, then I agree with Lumifer (and don't understand why he's got all those downvotes for saying it): the trivial zero-risk solution is to shut down immediately without doing anything, and nothing else yo...
This thread is for asking any questions that might seem obvious, tangential, silly or what-have-you. Don't be shy, everyone has holes in their knowledge, though the fewer and the smaller we can make them, the better.
Please be respectful of other people's admitting ignorance and don't mock them for it, as they're doing a noble thing.
To any future monthly posters of SQ threads, please remember to add the "stupid_questions" tag.