= 27d567bc2d48d9f40e9ccc20ea370b15)
PhilGoetz comments on The scourge of perverse-mindedness - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (249)
I'm breaking this out into a separate reply, because it's its own sub-thread:
If no utility function, and hence no world state, is objectively better than any other, then all utility functions are wireheading. Because the only distinction between wireheading, and not wireheading, is that the wirehead only cares about his/her own qualia, not about states of the world. If the only reason you care about states of the world is because of how your utility function evaluates them - that is to say, what qualia they generate in you - you are a wirehead.
You have it backwards. I do not care about things because of how my utility function evaluates them. Rather, my utility function evaluates things the way it does because of how I care about it. My utility function is a description of my preferences, not the source of them.
I don't think the order of execution matters here. If there's no objective preference over states of the world, then there's no objective reason to prefer "not wireheading" (caring about states of the world) over "wireheading" (caring only about your percepts).
There is no "objective" reason to do anything. Knowing that, what are you going to do anyways? Myself, I am still going to things for my subjective reasons.
Okay; but then don't diss wireheading.
You appear to have an overexpansive definition of wireheading. Having an arbitrary utility function is not the same as wireheading. Wireheading is a very specific sort of alteration of utility functions that we (i.e. most humans, with our current, subjective utility functions, nearly universally) see as very dangerous, because it throws away what we currently care about. Wireheading is a "parochial" definition, not universal. But that's OK.
What's your definition of wireheading?
I didn't define it as having an arbitrary utility function. I defined it as a utility function that depends only on your qualia.
What else can the utility function as implemented by your hardware depend on besides your qualia, and computations derived from your qualia?
Calling utility functions "wireheading" is a category error. Wireheading is either:
1. Directly acting on the machinery that implements one's utility function to trivially satisfy this hardware, i.e. by directly injecting qualia rather than providing the qualia via what they are normally correlated with.
2. More broadly, altering one's utility function to one that is trivial to broadly satisfy, such as by reinforcement via 1.
If you read my original comment, it's clear that I meant wireheading is having a utility function that depends only on your qualia. Or maybe "choosing to have".
Huh? So you think there's nothing inside your head except qualia?
Beliefs aren't qualia. Subconscious information isn't qualia.
This sounds like a potentially good definition. But I'm unclear then why anyone using utility theory, and that definition, would object to wireheading. If you've got a utility function, and you can satisfy it, that's the thing to do, right? Why does it matter how you satisfy it? You seem to be saying that the hardware implementation isn't your real utility function, it's just an implementation of it. As if the utility function stood somewhere outside you.
Beliefs and subconcious information are derived from qualia and the information about the external world that they correlate with, no?
Utility functions are a convenient mathematical description to describe preferences of entities in game theory and some decision theories, when these preferences are consistent. It's useful as a metaphor for "what we want", but when used loosely like this, there are troubles.
As applied to humans, this flat-out doesn't work. Empirically and as a general rule, we're not consistent, and most of us can readily be money-pumped. We do not have a nice clean module that weighs outcomes and assigns real numbers to them. Nor do we feed outcome weights into a probability weighting module, and then choose the maximum utility. Our values change on reflection. Heck, we're not even unitary entities. Our consciousness is multi-faceted. There are the left and right brains communicating and negotiating through the corpus callosum. The information immediately accessible to the consciousness, what we identify with, is rather different than the information our subconscious uses. We are a gigantic hack of an intelligence built upon the shifting sands of stimulus-response and reinforcement conditioning. These joints in our selves make it easier to wirehead, and essentially kill our current selves, leaving only animal-level instincts, if that.
There are multiple utility functions running around here. The basic point was that what I consider important now matters to what choices I make now. The fact that I can make the future me have a new utility function, satisfied by wireheading, does not register positively on my current utility function. In fact, because it throws away almost everything I now care about, I am unlikely to do it now. My goals are "satisfy my current utility function", and are always that, because that's what we mean by the abstraction of utility function. My goals are not to satisfy what preferences I may later have. My goals are not to change my preferences to be easier to satisfy, because that means my current goals are less likely to be satisfied. If my goals change, than they will have changed, and only then will I choose differently. It's not that my utility function stands outside of me: my utility function is part of me. Changing it changes me. It so happens that my utility function would be easily changed if I started directly stimulating my reward center. The reward center is not my utility function, though it is part of the implementation of my decision function (which if it were coherent, could be summarized in a utility function, and sets of probabilities). If we wish to identify the reward circuitry of my brain with a utility function, we've also got to put a few other utility functions in, and entities having these utility functions that are in a non-zero sum game with the reward circuitry.
I think I see your point: a wireheading utility function would value (1) for providing the reward with less effort, while a nonwireheading utility function would disvalue (1) for providing the reward without the desideratum.
You should define 'qualia,' then, in such a way that makes it clear how they're causally isolated from the rest of the universe.
I didn't say they were causally isolated.
If you think that the notion of "qualia" requires them to be causally isolated from the universe (which is my guess at why you even bring the idea up), then the burden is on you to explain why everyone who discusses consciousness except Daniel Dennett is silly.
In that case, nothing can be said to depend only on the qualia, because anything that depends on them is also indirectly influenced by whatever the qualia themselves depend on.