= 27d567bc2d48d9f40e9ccc20ea370b15)
andreas comments on Hedging our Bets: The Case for Pursuing Whole Brain Emulation to Safeguard Humanity's Future - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
Comments (244)
Here is a potentially more productive way of seeing this situation: We do want our current preferences to be made reality (because that's what the term preference describes), but we do not know what our preferences look like, part of the reason being that we are not smart enough and do not have enough time to think about what they are. In this view, our preferences are not necessarily going to drift if we figure out how to refer to human preference as a formal object and if we build machines that use this object to choose what to do — and in this view, we certainly don't want our preferences to drift.
On the other hand, WBE does not "simply take the guess work out". It may be the case that the human mind is built such that "making people smarter" is feasible without changing preference much, but we don't know that this is the case. As long as we do not have a formal theory of preference, we cannot strongly believe this about any given intervention – and if we do have such a theory, then there exist better uses for this knowledge.
Yes, but one of our preferences may well be that we are open to an evolution of our preferences. And, whether or not that is one of our preferences, it certainly is the cases that preferences do evolve over time, and that many consider that a fundamental aspect of the human condition.
I agree we don't know that is the case, and would assume that it isn't.
Any notion of progress (what we want is certainly not evolution) can be captured as a deterministic criterion.
Obviously I meant 'evolution' in the sense of change over time, not change specifically induced by natural selection.
As to a deterministic criterion, I agree that such a thing is probably possible. But... so what? I'm not arguing that FAI isn't possible. The topic at hand is FAI research relative to WBE. I'm assuming a priori that both are possible. The question is which basket should get more eggs.
You said:
This is misuse of the term "preference". "Preference", in the context of this discussion, refers specifically to that which isn't to be changed, ever. This point isn't supposed to be related to WBE vs. FAI discussion, it's about a tool (the term "preference") used in leading this discussion.
Your definition is too narrow for me to accept. Humans are complicated. I doubt we have a core set of "preferences" (by your definition) which can be found with adequate introspection. The very act of introspection itself changes the human and potentially their deepest preferences (normal definition)!
I have some preferences which satisfy your definition, but I wouldn't consider them my core, underlying preferences. The vast majority of preferences I hold do not qualify. I'm perfectly OK with them changing over time, even the ones that guide the overarching path of my life. Yes, the change in preferences is often caused by other preferences, but to think that this causal chain can be traced back to a core preference is unjustified, in my opinion. There could just as well be closed loops in the causal tree.
You are disputing definitions! Of course, there are other natural ways to give meaning to the word "preference", but they are not as useful in discussing FAI as the comprehensive unchanging preference. It's not supposed to have much in common with likes or wants, and with their changes, though it needs to, in particular, describe what they should be, and how they should change. Think of your preference as that particular formal goal system that it is optimal, from your point of view (on reflection, if you knew more, etc.), to give to a Strong AI.
Your dislike for application of the label "preference" to this concept, and ambiguity that might introduce, needs to be separated from consideration of the concept itself.
I specifically dispute the usefulness of your definition. It may be a useful definition in the context of FAI theory. We aren't discussing FAI theory.
And, to be fair, you were originally the one disputing definitions. In my post I used the standard definition of 'preference', which you decided was 'wrong', saying
rather than accepting the implied (normal!) definition I had obviously used.
Regardless, it seems unlikely we'll be making any progress on the on-topic discussion even if we resolve this quibble.
But we do. Whether a particular action is going to end well for humanity is a core consideration in Friendliness. When you say
if it's read as implying that this road is OK, it is a factual claim about how preferable (in my sense) the outcome is going to be. The concept of preference (in my sense) is central to evaluating the correctness of your factual claim.
Your concept of preference is one way of evaluating the correctness of my claim, I agree. If you can resolve the complex web of human preferences (in my sense) into a clean, non-contradictory, static preference system (your sense) then you can use that system to judge the value of the hypothetical future in which WBE research overran FAI research.
It's not clear to me that this is the only way to evaluate my claim, or that it is even a reasonable way. My understanding of FAI is that arriving at such a resolution of human preferences is a central ingredient to building an FAI, hence using your method to evaluate my claim would require more progress on FAI. But the entire point of this discussion is to decide if we should be pushing harder for progress on FAI or WBE. I'll grant that this is a point in favor for FAI -- that it allows for a clearer evaluation of the very problem we're discussing -- but, beyond that, I think we must rely on the actual preferences we have access to now (in my sense: the messy, human ones) to further our evaluations of FAI and WBE.
You are right, I was wrong to claim authority over the meaning of the term as you used it. The actual problem was in you misinterpreting its use in andreas's comment, where it was used in my sense: