= 27d567bc2d48d9f40e9ccc20ea370b15)
Kawoomba comments on [Link] Your Elusive Future Self - Less Wrong Discussion
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (37)
There is a curious disparity between the ardor with which we tackle the issue of defending against being reprogrammed by an AI, and with which we accept major life changes which, while we sometimes chose them, often result in severe and unpredictable (as per the article) changes of your utility function, your terminal values and whatever else constitutes your identity.
The difference may lie partly in the scope of the transformation, but then again, think of the moron you (yes, you) probably consider your 10 years younger self. Would it be so strange to have reached your present self when hypothetically exchanging your past self with some friend/peer? That's how ~~much~~/little is preserved. However, I'd expect most of the difference being due to an "agenty" process changing you invoking more self-defense reflexes than the supposedly more "random" process of just living your life.
EDIT: For a more entertaining if hyperbolic example, it's like saying "Never change a thing about my like for Beethoven, or for this particular brand of Irish Whiskey. These are irreducible, terminal values! Wait, there's a button which is gonna randomize me from head to toe, it says "marry, get children". Alright, here we gooooo!"
Is that just an LW-specific cultural attitude? Ben Goertzel doesn't seem to think that value drift from a non-CEV AI would be that terrible. It occurred to me that the path humanity has taken already through technological and cultural development is a very poor approximation of CEV, and yet we're likely happier today than we were in the ancestral environment.
Well, if my goal were to be replaced by something different from me that is happier doing whatever it ends up doing than I am doing what I do, that's relatively simple. But it doesn't actually seem to be an adequate description of my goal.
Do you think a typical hunter-gatherer would be happy to be transported to the modern world?
Fair question. I'm not sure.
Immediately upon being transported, certainly not.
I suppose I'm weakly confident that they would experience more happiness over the course of their lives than they would have had they not been so transported, mostly by virtue of avoiding or deferring various happiness-reducing conditions (e.g. disease, death, pain, malnutrition, death-of-loved-ones, etc.).
This is largely an expression of my belief that there hasn't in fact been that much value drift between then and now, and my willingness to treat happiness as a rough measure of compliance-with-values (aka utility). If I believed there were a lot of value drift, my confidence would decrease.
If at time T there's a fully-compliant-with-my-values system controlling my environment, I similarly expect less utility at some later time if that system experiences value drift than if it doesn't.
Generally agree, but I don't think the situation is quite as bad as this, for two reasons. First, I agree that stated preferences can change drastically over time, but I suspect that actual preferences are more stable. Second, introspection is difficult and humans are not automatically strategic, so people are both unlikely to have a good grasp of their own preferences and unlikely to reliably take actions to satisfy even what they believe their preferences to be. Life changes that look like they're changing your preferences might just be life changes that help you get a better grasp of your existing preferences or get better ideas about how to satisfy them.
My guess is that the life changes that are most likely to change your actual preferences are those associated with biological mechanisms, e.g. puberty and the biological clock. The ones most likely to change your stated preferences might be those associated with substantial shifts in your peer group, e.g. going to college. Either way, these kinds of shifts both have the property that everyone else is going through them too, so at least they're not weird shifts.
Kawoomba, judging from my conversations with him, believes that stated preferences are all there is. When you convince someone they were wrong, their terminal values change. So naturally he views this prospect as rather more common, and undervalued.
Agree.
On the other hand, I don't think of my 10 years younger self as a moron, just naive. I don't think I've changed all that much, even, though sometimes I'd like to think I have. It's possible that part of the reason for the bias described in the study is that we overestimate the changes we've gone through in the last 10 years.
Well, yes, in that we'd have to explain how my present self ended up with memories of what are now my friend/peer's experiences, among other things. Any explanation of that is bound to be pretty damned strange.
That said, I do agree with the general principle that the differences between me at various different times are potentially radical enough that definitions of "my identity" based on similarities will pick out a different set of person-moments than those based on continuity. Personally, I'm content using a pretty broad similarity-based definition that includes lots of discontinuous person-moments, but others' mileage varies.
I don't think that your declarative/explicit memory is all that important to your identity, the influence those experiences had on your cognitive algorithms, certainly, but the explicit memories themselves? Of course any path that would take a 10-years-younger peer of yours to your current identity would be strange, even somewhat stranger than the semi-random walk you yourself took. But it is conceivable, i.e. your past and long gone personae were part of your path through time and space to your current self, but they aren't strictly necessary to get to where you are now. Mostly, they're just gone.
I'm no longer sure what you mean by "your current identity" in that sentence, so let me try to taboo it.
Consider person X, my friend of ten years ago. I distinctly remember that ten years ago, X called himself George and described himself as having been born in Atlanta, whereas I called myself Dave and described myself as having been born in New Jersey.
Right now, I call myself Dave and describe myself as having been born in New Jersey.
Any path that takes X from where he was ten years ago to exactly the state I'm in right now, which includes calling myself Dave and describing myself as having been born in New Jersey, is very very strange... far stranger than the path I myself took, and stranger than .999999999 of the paths that were available for me to take.
Any path that takes X from where he was ten years ago to some state that shares some elements with the state I'm in right now, but not others, may be less strange than that. It may even be less strange than the path I myself took. It depends on which shared elements we're talking about.
I agree that many elements of the state I'm in right now don't matter very much.
"What's in a name? That which we call a rose
By any other name would smell as sweet."
Yea, e.g. lots of variables share the name "x". It's just denoting that which you talk about, and while a name can influence your upbringing (say it's a foreign name), i.e. while it'll help take you to your current utility function, at least I would not consider it crucial for identity purposes having reached the present time (yet it is essential for daily life, of course). Hopefully that makes my example clearer, if not steelman it as you see fit :), I think even without it my point is comprehensible.
As I said, I wasn't sure what you meant by "current identity", which is why I provided two versions of what you might mean.
I was hoping you might endorse one or the other of them, but oh well.
Anyway, yes, if we generalize your point about my name to also include where I was born and what my native language was and what religion I was raised in and the million other demographic data that distinguish me from my ten-years-ago peer, then I agree with you.