= 27d567bc2d48d9f40e9ccc20ea370b15)
timtyler comments on Hedging our Bets: The Case for Pursuing Whole Brain Emulation to Safeguard Humanity's Future - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
Comments (244)
Leaving aside the other reasons why this scenario is unrealistic, one of the big flaws in it is the assumption that a mind decomposes into an engine plus a utility function. In reality, this decomposition is a mathematical abstraction we use in certain limited domains because it makes analysis more tractable. It fails completely when you try to apply it to life as a whole, which is why no humans even try to be pure utilitarians. Of course if you postulate building a superintelligent AGI like that, it doesn't look good. How would it? You've postulated starting off with a sociopath that considers itself licensed to commit any crime whatsoever if doing so will serve its utility function, and then trying to cram the whole of morality into that mathematical function. It shouldn't be any surprise that this leads to absurd results and impossible research agendas. That's the consequence of trying to apply a mathematical abstraction outside the domain in which it is applicable.
Humans regularly use utilitly-based agents - to do things like play the stockmarket. They seem to work OK to me. Nor do I agree with you about utility-based models of humans. Basically, most of your objections seem irrelevant to me.
When studying the stock market, we use the convenient approximation that people are utility maximizers (where the utility function is expected profit). But this is only an approximation, useful in this limited domain. Would you commit murder for money? No? Then your utility function isn't really expected profit. Nor, as it turns out, is it anything else that can be written down - other than "the sum total of all my preferences", at which point we have to acknowledge that we are not utility maximizers in any useful sense of the term.
"We" don't have to acknowledge that.
I've gone over my views on this issue before - e.g. here:
http://lesswrong.com/lw/1qk/applying_utility_functions_to_humans_considered/1kfj
If you reject utility-based frameworks in this context, then fine - but I am not planning to rephrase my point for you.
Right, I hadn't read your comments in the other thread, but they are perfectly clear, and I'm not asking you to rephrase. But the key term in my last comment is in any useful sense. I do reject utility-based frameworks in this context because their usefulness has been left far behind.
Personally, I think a utilitarian approach is very useful for understanding behaviour. One can model most organisms pretty well as expected fitness maximisers with limited resources. That idea is the foundation of much evolutionary psychology.
The question isn't whether the model is predictively useful with respect to most organisms, it's whether it is predictively useful with respect to a hypothetical algorithm which replicates salient human powers such as epistemic hunger, model building, hierarchical goal seeking, and so on.
Say we're looking to explain the process of inferring regularities (such as physical laws) by observing one's environment - what does modeling this as "maximizing a utility function" buy us?
In comparison with what?
The main virtues of utility-based models are that they are general - and so allow comparisons across agents - and that they abstract goal-seeking behaviour away from the implementation details of finite memories, processing speed, etc - which helps if you are interested in focusing on either of those areas.