Where “pain” and “suffering” are defined, respectively, as... what?
Roughly, pain is a sensation typically associated with damage to the body, suffering is an experience of stimuli as intrinsically unpleasant.
I do not suffer if my room is painted a color I do not like, but I still may care about the color my room is painted.
So pain asymbolia means something else than “being able to feel pain but not caring about it”?
It means "being able to feel pain but not suffering from it."
Suppose an AI were to design and implement more efficient algorithms for processing sensory stimuli? Or add a "face recognition" module when it determines that this would be useful for interacting with humans?
The ancient Greeks have developed methods of improved memorization. It has been shown that human-trained dogs and chimps are more capable of human-face recognition than others of their kind. None of them were artificial (discounting selective breeding in dogs and Greeks).
It seems that you should be able to write a simple program that overwrites its own code with an arbitrary value. Wouldn't that be a counterexample?
Would you consider such a machine an artificial intelligent agent? Isn't it just a glorified printing press?
I'm not saying that some configurations of memory are physically impossible. I'm saying that intelligent agency entails typicality, and therefore, for any intelligent agent, there are some things it is extremely unlikely to do, to the point of practical impossibility.
Do we agree, then, that humans and artificial agents are both subject to laws forbidding logical contradictions and the like, but that artificial agents are not in principle necessarily bound by the same additional restrictions as humans?
I would actually argue the opposite.
Are you familiar with the claim that people are getting less intelligent since modern technology allows less intelligent people and their children to survive? (I never saw this claim discussed seriously, so I don't know how factual it is; but the logic of it is what I'm getting at.) The idea is that people today are less constrained in their required intelligence, and therefore the typical human is becoming less intelligent.
Other claims are that activities such as browsing the internet and video gaming are changing the set of mental skills which humans are good at. We improve in tasks which we need to be good at, and give up skills which are less useful. You gave yet another example in your comment regarding face recognition.
The elasticity of biological agents is (quantitatively) limited, and improvement by evolution takes time. This is where artificial agents step in. They can be better than humans, but the typical agent will only actually be better if it has to. Generally, more intelligent agents are those which are forced to comply to tighter constraints, not looser ones.
Suppose an AI were to design and implement more efficient algorithms for processing sensory stimuli? Or add a "face recognition" module when it determines that this would be useful for interacting with humans?
The ancient Greeks have developed methods of improved memorization. It has been shown that human-trained dogs and chimps are more capable of human-face recognition than others of their kind. None of them were artificial (discounting selective breeding in dogs and Greeks).
It seems that you should be able to write a simple program that overwrites its own code with an arbitrary value. Wouldn't that be a counterexample?
Would you consider such a machine an artificial intelligent agent? Isn't it just a glorified printing press?
I'm not saying that some configurations of memory are physically impossible. I'm saying that intelligent agency entails typicality, and therefore, for any intelligent agent, there are some things it is extremely unlikely to do, to the point of practical impossibility.
Certainly that doesn't count as an intelligent agent - but a GAI with that as its only goal, for example, why would that be impossible? An AI doesn't need to value survival.
I'd be interested in the conclusions derived about "typical" intelligences and the "forbidden actions", but I don't see how you have derived them.
Do we agree, then, that humans and artificial agents are both subject to laws forbidding logical contradictions and the like, but that artificial agents are not in principle necessarily bound by the same additional restrictions as humans?
I would actually argue the opposite.
Are you familiar with the claim that people are getting less intelligent since modern technology allows less intelligent people and their children to survive? (I never saw this claim discussed seriously, so I don't know how factual it is; but the logic of it is what I'm getting at.) The idea is that people today are less constrained in their required intelligence, and therefore the typical human is becoming less intelligent.
Other claims are that activities such as browsing the internet and video gaming are changing the set of mental skills which humans are good at. We improve in tasks which we need to be good at, and give up skills which are less useful. You gave yet another example in your comment regarding face recognition.
The elasticity of biological agents is (quantitatively) limited, and improvement by evolution takes time. This is where artificial agents step in. They can be better than humans, but the typical agent will only actually be better if it has to. Generally, more intelligent agents are those which are forced to comply to tighter constraints, not looser ones.
I think we have our quantifiers mixed up? I'm saying an AI is not in principle bound by these restrictions - that is, it's not true that all AIs must necessarily have the same restrictions on their behavior as a human. This seems fairly uncontroversial to me. I suppose the disconnect, then, is that you expect a GAI will be of a type bound by these same restrictions. But then I thought the restrictions you were talking about were "laws forbidding logical contradictions and the like"? I'm a little confused - could you clarify your position, please?
Depending on the rest of your utility distribution, that is probably true. Note, however, that an additional 10^6 utility in the right half of the utility function will change the median outcome of your "life": If 10^6 is larger than all the other utility you could ever receive, and you add a 49 % chance of receiving it, the 50th percentile utility after that should look like the 98th percentile utility before.
Could you rephrase this somehow? I'm not understanding it. If you actually won the bet and got the extra utility, your median expected utility would be higher, but you wouldn't take the bet, because your median expected utility is lower if you do.
I want that it is possible to have a very bad outcome: If I can play a lottery that has 1 utilium cost, 10^7 payoff and a winning chance of 10^-6, and if I can play this lottery enough times, I want to play it.
"Enough times" to make it >50% likely that you will win, yes? Why is this the correct cutoff point?
Ah, it appears that I'm mixing up identities as well. Apologies.
Yes, I retract the "variance greater than 5". I think it would have to be variance of at least 10,000 for this method to work properly. I do suspect that this method is similar to decision-making processes real humans use (optimizing the median outcome of their lives), but when you have one or two very important decisions instead of many routine decisions, methods that work for many small decisions don't work so well.
If, instead of optimizing for the median outcome, you optimized for the average of outcomes within 3 standard deviations of the median, I suspect you would come up with a decision outcome quite close to what people actually use (ignoring very small chances of very high risk or reward).
This all seems very sensible and plausible!
Thanks for challenging my position. This discussion is very stimulating for me!
Sure, but we could imagine an AI deciding something like "I do not want to enjoy frozen yogurt", and then altering its code in such a way that it is no longer appropriate to describe it as enjoying frozen yogurt, yeah?
I'm actually having trouble imagining this without anthropomorphizing (or at least zoomorphizing) the agent. When is it appropriate to describe an artificial agent as enjoying something? Surely not when it secretes serotonin into its bloodstream and synapses?
This seems trivially false - if an AI is instantiated as a bunch of zeros and ones in some substrate, how could Godel or similar concerns stop it from altering any subset of those bits?
It's not a question of stopping it. Gödel is not giving it a stern look, saying: "you can't alter your own code until you've done your homework". It's more that these considerations prevent the agent from being in a state where it will, in fact, alter its own code in certain ways. This claim can and should be proved mathematically, but I don't have the resources to do that at the moment. In the meanwhile, I'd agree if you wanted to disagree.
You see reasons to believe that any artificial intelligence is limited to altering its motivations and desires in a way that is qualitatively similar to humans? This seems like a pretty extreme claim - what are the salient features of human self-rewriting that you think must be preserved?
I believe that this is likely, yes. The "salient feature" is being subject to the laws of nature, which in turn seem to be consistent with particular theories of logic and probability. The problem with such a claim is that these theories are still not fully understood.
Thanks for challenging my position. This discussion is very stimulating for me!
It's a pleasure!
Sure, but we could imagine an AI deciding something like "I do not want to enjoy frozen yogurt", and then altering its code in such a way that it is no longer appropriate to describe it as enjoying frozen yogurt, yeah?
I'm actually having trouble imagining this without anthropomorphizing (or at least zoomorphizing) the agent. When is it appropriate to describe an artificial agent as enjoying something? Surely not when it secretes serotonin into its bloodstream and synapses?
Yeah, that was sloppy of me. Leaving aside the question of when something is enjoying something, let's take a more straightforward example: Suppose an AI were to design and implement more efficient algorithms for processing sensory stimuli? Or add a "face recognition" module when it determines that this would be useful for interacting with humans?
This seems trivially false - if an AI is instantiated as a bunch of zeros and ones in some substrate, how could Godel or similar concerns stop it from altering any subset of those bits?
It's not a question of stopping it. Gödel is not giving it a stern look, saying: "you can't alter your own code until you've done your homework". It's more that these considerations prevent the agent from being in a state where it will, in fact, alter its own code in certain ways. This claim can and should be proved mathematically, but I don't have the resources to do that at the moment. In the meanwhile, I'd agree if you wanted to disagree.
Hm. It seems that you should be able to write a simple program that overwrites its own code with an arbitrary value. Wouldn't that be a counterexample?
You see reasons to believe that any artificial intelligence is limited to altering its motivations and desires in a way that is qualitatively similar to humans? This seems like a pretty extreme claim - what are the salient features of human self-rewriting that you think must be preserved?
I believe that this is likely, yes. The "salient feature" is being subject to the laws of nature, which in turn seem to be consistent with particular theories of logic and probability. The problem with such a claim is that these theories are still not fully understood.
This sounds unjustifiably broad. Certainly, human behavior is subject to these restrictions, but it is also subject to much more stringent ones - we are not able to do everything that is logically possible. Do we agree, then, that humans and artificial agents are both subject to laws forbidding logical contradictions and the like, but that artificial agents are not in principle necessarily bound by the same additional restrictions as humans?
And what do you mean by "the possibility of getting tortured will manifest itself only very slightly at the 50th percentile"? I thought you were restricting yourself to median outcomes, not distributions? How do you determine the median distribution?
I don't. I didn't write that.
Your formulation requires that there be a single, high probability event that contributes most of the utility an agent has the opportunity to get over its lifespan. In situations where this is not the case (e.g. real life), the decision agent in question would choose to take all opportunities like that.
The closest real-world analogy I can draw to this is the decision of whether or not to start a business. If you fail (which there is a slightly more than 50% chance you will), you are likely to be in debt for quite some time. If you succeed, you will be very rich. This is not quite a perfect analogy, because you will have more than one chance in your life to start a business, and the outcomes of business ownership are not orders of magnitude larger than the outcomes in real life. However, it is much closer than the "51% chance to lose $5, 49% chance to win $10000" that your example intuitively brings to mind.
Ah! Sorry for the mixed-up identities. Likewise, I didn't come up with that "51% chance to lose $5, 49% chance to win $10000" example.
But, ah, are you retracting your prior claim about a variance of greater than 5? Clearly this system doesn't work on its own, though it still looks like we don't know A) how decisions are made using it or B) under what conditions it works. Or in fact C) why this is a good idea.
Certainly for some distributions of utility, if the agent knows the distribution of utility across many agents, it won't make the wrong decision on that particular example by following this algorithm. I need more than that to be convinced!
For instance, it looks like it'll make the wrong decision on questions like "I can choose to 1) die here quietly, or 2) go get help, which has a 1/3 chance of saving my life but will be a little uncomfortable." The utility of surviving presumably swamps the rest of the utility function, right?
Yes that is a good insight. I'll rephrase it to perhaps make it clear to a somewhat different set of people. If your strategy is to have a good median outcome of your life, you will still get to bet on longshots with high payoffs, as long as you expect to be offered a lot of those bets. The fewer bets you expect to be offered of a certain type, the more likely winning must be for you to take it, even if the "expected" pay out on these is very high.
A quantification of this concept in somewhat simple cases was done by Jim Kelly and is called the Kelly Criterion. Kelly asked a question: given you have finite wealth, how do you decide how much to bet on a given offered bet in order to maximize the rate at whcih your expected wealth grows? Kelly's criterion, if followed, also has the side-effect of insuring you never go completely broke, but in a world of minimum bet sizes, you might go broke enough to not be allowed to play anymore.
Of course, all betting strategies, where you are betting against other presumed rational actors, require you to be smarter, or at least more correct thant the people you are betting against, in order to allow you to win. In Kelly's calculation, the size of your bet depends on both the offered odds and the "true" odds. So how do you determine the true odds? Well that is left as an exercise for the reader!
And so it goes with Pascal's muggings. As far as my study has taken me, I know of no way to reliably estimate whether the outcomes in offered in Pascal's muggings are one in a million, one in a google, one in a googleplex, or one in 3^^^3. And yet the "correct" amount to bet using the Kelly criterion will vary by as big a factor as those probability estimates vary one from the other.
There is also the result that well-known cognitive biases will cause you to get infinitesimal probabilities wrong by many orders of magnitude, without properly estimating your probable error on them. For any given problem, there is some probability estimate below which all further attempts to refine the estimate are in the noise: the probability is "essentially zero." But all the bang in constantly revisiting these scenarios comes from the human biases that allow us to think that because we can state a number like 1 in a million or 1 in a google or 1 in 3^^^3 that we must be able to use it meaningfully in some probabilistic calculation.
If you are of the bent that hypotheses such as the utility of small probabilities should be empirically checked before you start believing the results of these calculations, it may take a few lifetimes of the universe (or perhaps a google lifetimes of the universe) before you have enough evidence to determine whether a calculation involving a number like 1 in a google means anything at all.
Googol. Likewise, googolplex.
Subscribe to RSS Feed
= f037147d6e6c911a85753b9abdedda8d)
Agree, with the assumption that "stimuli" as relevant to suffering includes internal stimuli generated from one's own thoughts.
Yeah, that's certainly a fair clarification. It'd probably take a lot more space to give a really robust definition of "suffering", but that's close enough for gummint work.