I'm sorry my post was so ambiguous. I'll try to put the idea in clearer words.
Disutility, like utility, is a learning machine's internal representation of it valuation of the environment. The machine observes its environment (virutal or physical) and runs a disutility function on its observations to establish how "desirable" the environment is according to its disutility function.
Example: A pest control drone patrols the are it is programmed to patrol. Its disutility function is "number of small insects that are not butterflies, and spiders + ((humans harmed by my actions)*1000000)".
It decides how to act by modelling the possible worlds that result from implementing things it can do and choosing the one with the lowest expected disutility, as long as at least one is above some arbitrary "shutdown threshold" where nothing the drone can "imagine" is good enough.
In the example: The drone might model what would happen if it pointed a mosquito-zapping laser at a bug that it sees. If the world with a zapped bug has less disutility, it'll do that. If that wouldn't work because, say, the bug is sitting on some human's lap, it will not do that but instead try one of its other options, like wait until the bug presents a better target or move on to a different part of the area.
And if there is no disutility to reduce - because the calculated disutility is 0 and nothing the system could do would reduce it further - the system does nothing. This is the difference from a utility maximizer, because utility seems to always be (at least implicitly) unbounded.
Of course this still presents the obvious failure mode where the system is prone hack itself and change the internal representation. In the bug-zapping drone, the drone might find the best way to have 0 disutility is to turn off its cameras and simply not see any bugs. This remains a serious problem. But at least at that point the system shuts down "satisfied", rather than turn its future light cone into computronium in order to represent ever higher internal representations of utility.
Suppose we are considering an agent with a more "positive" mission than that of your pest control drone (whose purpose is best expressed negatively: get rid of small pests). For instance, perhaps the agent is working for a hedge fund and trying to increase the value of its holdings, or perhaps it's trying to improve human health and give people longer healthier lives.
How do you express that in terms of "disutility"?
I think what is doing the work here is not using "disutility" rather than "utility", but having a utili...
This thread is for asking any questions that might seem obvious, tangential, silly or what-have-you. Don't be shy, everyone has holes in their knowledge, though the fewer and the smaller we can make them, the better.
Please be respectful of other people's admitting ignorance and don't mock them for it, as they're doing a noble thing.
To any future monthly posters of SQ threads, please remember to add the "stupid_questions" tag.