= 27d567bc2d48d9f40e9ccc20ea370b15)
Vladimir_Nesov comments on The utility curve of the human population - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (31)
In the light of the correction you've made just now, do you retract this comment as well? (It looks to be based on the same mistake, but if you don't think so, I'd like to argue.)
No, it's a different point, and one I'd be happy to argue. Here I talk about encoding actual human preferences over all possible futures, not designing an algorithm that will yield one good future. For example, an algorithm that gives one good future may never actually have to worry about torture vs dust specks. So it's not clear that we should worry about it either.
I suspect you are not talking about neurons in the brain, but have no idea what you do mean...
By Church-Turing thesis, you can construct an artifact behaviorally indistinguishable from a human based even on expected utility maximization (even though it's an inadequate thing to do). Whatever you can expect of a real human, including answering hypothetical questions, you can expect from this construction.
Algorithms are strategies, they are designed to work depending on observations. When you design an algorithm, you design behaviors for all possible futures. Other than giving this remark, I don't know what to do with your comment...
Nodes in the graph are hypothetical situations, and arrows are preferences.
Preference as order on situations? Make that order on histories, or better order on games to be provably won, but you should already know that, so again I don't see what you are saying.
Oh, okay, on possible histories. I really don't understand what's unclear to you. It's not obvious to me that there's a unique canonical way to build a complete acyclic graph (utility-based preference) from an incomplete graph with cycles (actual human preference). Yes, expected utility optimization can mimic any behavior, but I don't want to mimic behavior, I want to represent the data structure of preferences.
By C-T, you can represent any data, right? The utility-surrogate can have a detailed scan of a human in its virtual utility-maximizing pocket, or even run a simulation of human brain, just on a different substrate.
For histories: you argue that people have cyclic preference over world histories as well, because you consider preference to be the same thing as choice, that is prone to whim? That's not what I mean by preference (which you should also know), but it explains your comments in this thread.
Whims are all we can observe. We disagree on whether whims can be canonically regularized into something coherent. I don't think Eliezer knows that either (it's kind of similar to the question whether humanity's volition coheres). Yeah, he's trying to regularize his whims, and you may strive for that too, but what about the rest of us?
You can consider a person as a system that gives various counterfactual reactions to interaction -- most of these reactions won't be observed in the history of what actually happened to that person in the past. While it e.g. makes sense to talk about what a person (actually) answered to a question asked in English, you are not working with concepts themselves in this setting: just as the interpretation of words is a little iffy, deeper understanding of the meaning of the words (by the person who answers the questions) is even more iffy.
What you need to talk about preference is to compare huge formal strategies or games (not even snapshots of the history of the world), while what you get in the naive settings is asking "yes/no" questions in English.
Unavailability of adequate formalization of what it means to ask the actual question about consequences doesn't justify jumping to identification of preference with "yes/no" utterances resulting from questions obtained in unspecified manner.
I don't see how going from yes/no questions to simulated games helps. People will still exhibit preference reversals in their actions, or just melt down.