You're looking at Less Wrong's discussion board. This includes all posts, including those that haven't been promoted to the front page yet. For more information, see About Less Wrong.
Comment author:DanielVarga
06 April 2012 11:43:14PM
4 points
[-]
Felix means happy (or lucky), and is the origin of the word felicity. It took me a while to realize this, so I thought I would note it. Is it obvious for all native English speakers?
Comment author:Desrtopa
08 April 2012 12:39:02PM
2 points
[-]
Not obvious to me. I did know the meaning of Felix, but it's deep enough in the unused drawers of my memory that I might never have made the connection without someone pointing it out.
Comment author:Dmytry
06 April 2012 10:22:09AM
*
8 points
[-]
if it was a total utility maximizing AI it would clone the utility monster (or start cloning everyone else if the utility monster is super linear) edit: on the other hand, if it was average utility maximizing AI it would kill everyone else leaving just the utility monster. In any case there'd be some serious population 'adjustment'.
Comment author:Dmytry
06 April 2012 03:15:46PM
*
3 points
[-]
It doesn't have to tell the monster. (this btw is one wireheading-related issue; i do quite hate the lingo here though; calling it wireheaded makes it sound like there isn't a couple thousands years of moral philosophy about the issue and related issues)
Comment author:Emile
06 April 2012 03:56:31PM
2 points
[-]
this btw is one wireheading-related issue; i do quite hate the lingo here though; calling it wireheaded makes it sound like there isn't a couple thousands years of moral philosophy about the issue and related issues
I'm not aware of an alternative to "wireheading" with the same meaning.
That's the ancient greeks writing about hypothetical wireheads. (the 'moral philosophy' is perhaps a bad choice of word for search for greek stuff; ethics is the greek word)
Comment author:Emile
06 April 2012 08:56:44PM
*
1 point
[-]
A bit of search around that showed nearly no reference to lotus eating/lotus eater in moral philosophy.
Something much closer to "wireheading" would be hedonism, and more specifically Nozick's Experience Machine, which is pretty much wireheading, but isn't thousands of years old, and has been referenced here.
(And the term "wirehead" as used here probably comes from the Known Space stories, so probably predates Nozick's 1974 book)
Comment author:Rhwawn
06 April 2012 09:14:41PM
2 points
[-]
I don't think you looked very hard - I turned up a few books apparently on moral philosophy by searching in Google Books for 'moral ("lotus eating" OR "lotus-eating" OR "lotus eater" OR "lotus-eater")'.
And yes, I'm pretty sure the wirehead term comes from Niven's Known Space. I've never seen any other origin discussed.
Comment author:Dmytry
06 April 2012 09:03:19PM
*
2 points
[-]
Well, for one thing, it ought to be obvious that Mohammed would have banned a wire into the pleasure centre, but lacking the wires, he just banned the alcohol and other intoxicants. The concept of 'wrong' ways of seeking the pleasure is very, very old.
Comment author:Desrtopa
06 April 2012 03:18:45PM
0 points
[-]
It would be awfully hard to hide.
Sure, it could lock the monster in an illusory world of optimal happiness, or just stimulate his pleasure centers directly, etc. But unless we assume that the AI is working under constraints that prevent that sort of thing, the comic doesn't make much sense.
Comment author:Dmytry
06 April 2012 04:03:14PM
*
0 points
[-]
There's no clear line between 'hiding' and 'not showing'. You can leave just a million people or so, to be put around the monster, and simply not show him the rest. It is not like the AI is making every wall into the screen displaying the suffering on the construction of pyramids. Or you can kill those people and show it in such a way that the monster derives pleasure from it. At any rate, anyone whose death would go unnoticed by the monster, or whose death does not sufficiently distress the monster, would die, if the AI is to focus on average pleasure.
edit: I think those solutions really easily come to mind when you know of what a soviet factory would do to exceed the five year plan.
Comment author:Desrtopa
06 April 2012 06:33:35PM
1 point
[-]
At any rate, anyone whose death would go unnoticed by the monster, or whose death does not sufficiently distress the monster, would die, if the AI is to focus on average pleasure.
The AI explicitly wasn't focused on average pleasure, but on total pleasure, as measured by average pleasure times the population.
Comment author:nonplussed
14 April 2013 10:08:34PM
0 points
[-]
You're all wrong — if the happiness of the utility monster compounds as the comic says, then you get greater happiness out of lumping it all into one monster rather than cloning.
Comment author:[deleted]
06 April 2012 07:07:03PM
*
2 points
[-]
It is a good thing that you are thinking good things about Felix. This means he is happier if you aren't in corn field since you are a good person with no bad thoughts.
Comment author:Thomas
06 April 2012 12:40:05PM
-8 points
[-]
Yeah, yeah.
I am very glad, that the people who advocate the Felix morality with the "dust speck" sophism have virtually no chance to really accomplish something in the AI field.
Comment author:Incorrect
06 April 2012 12:46:52PM
*
3 points
[-]
To be fair, one could argue for dust specks without Felix morality by weighing increases in individual's happiness with diminishing returns such as to asymptotically approach some limit. (But then you would sacrifice one individuals arbitrarily unimaginable happiness just to bring someone else an arbitrarily small sliver towards baseline)
Since it's meaningless to call dust specks "right", just consider it true if you want to. I don't want to so I don't.
Downvoted: You should let someone actually advocate the Felix argument, before bashing them for supposedly advocating it.
So far, generalizing from my own example, there's atleast one person who agrees with the dust speck argument, but opposes the Felix argument. I know as yet no person who agree with the Felix argument. So, I find it obnoxious that you effectively pretend I advocate the Felix argument when I don't.
You may think I'm inconsistent in supporting the one but not the other, but don't pretend I support both, okay?
So far, generalizing from my own example, there's atleast one person who agrees with the dust speck argument, but opposes the Felix argument.
Make that two.
Thomas, your treatment of this is reductio ad absurdum of what I feel like is what at least 33% of LW believes. Worse, when we're (and by we, I mean everyone else, since I'm not going to bother getting involved in this further) calling you on it and actually trying to have a dialogue, you're dismissing us and insulting us.
Comment author:wedrifid
06 April 2012 09:54:14AM
5 points
[-]
What is then the torture of all the Humanity, against the super happy Felix with 3^^^3 pyramids. Nothing. By the same line of reasoning.
It's hedonistic total-utilitarianism vs preference based consequentialism. That's a big difference. Not only would the 'sequence' you reject not advocate preferring to torture humanity for the sake of making Felix superhappy, even in the absence of negative externalities it would still consider that sort of 'happiness' production a bad thing even for Felix.
The problem is the line of reasoning, where a "50 years of torture" is better than 3^^^3 years with a dust speck in the eye every so often.
That's not even the dilemma you linked to. The dilemma you linked to "one person be horribly tortured for fifty years without hope or rest, or that 3^^^3 people get dust specks in their eyes".
What is then the torture of all the Humanity, against the super happy Felix with 3^^^3 pyramids. Nothing. By the same line of reasoning.
It's probably bad practice to say two lines of reasoning are the same line of reasoning, if you don't believe in either of them.
For starters I don't need to have a positive factor for Felix's further happiness in my utilty function. That alone is a significant difference.
Comment author:Thomas
06 April 2012 10:15:42AM
*
7 points
[-]
Look. You have one person, under terrible torture for 50 years on one side and a gazillion of people with a slight discomfort every year or so on the other side.
It is claimed that the first is better.
Now, you have a small humanity as is, only enslaved for pyramid building for Felix. He has eons of subjective time to enjoy this pyramids and he is unbelievably happy. More happy than any man, woman or child could ever be. The amount of happiness of Felix outweights the misery of billion of people by a factor of a million.
What's the fundamental difference between those two cases? I don't see it, do you?
Comment author:Nornagest
06 April 2012 07:43:36PM
*
5 points
[-]
Felix is essentially a Utility Monster: a thought experiment that's been addressed here before. As that family of examples shows, happiness-maximization breaks down rather spectacularly when you start considering self- or other-modification or any seriously unusual agents. You can bite that bullet, if you want, but not many people here do; fortunately, there are a few other ways you can tackle this if you're interested in a formalization of humanlike ethics. The "Value Stability and Aggregation" post linked above touches on the problem, for example, as does Eliezer's Fun Theory sequence.
You don't need any self-modifying or non-humanlike agents to run into problems related to "Torture vs. Dust Specks", though; all you need is to be maximizing over the welfare of a lot of ordinary agents. 3^^^3 is an absurdly huge number and leads you to a correspondingly counterintuitive conclusion (one which, incidentally, I'd estimate has led to more angry debate than anything else on this site), but lesser versions of the same tradeoff are quite realistic; unless you start invoking sacred vs. profane values or otherwise define the problem away, it differs only in scale from the same utilitarian calculations you make when, say, assigning chores.
Comment author:FAWS
06 April 2012 10:37:42AM
*
12 points
[-]
The only similarity between those cases is that they involve utility calculations you disagree with. Otherwise every single detail is completely different. (e. g. the sort of utility considered, two negative utilities being traded against each other vs. trading utility elsewhere (positive and negative) for positive utility, which side of the trade the single person with the large individual utility difference is on, the presence of perverse incentives, etc, etc).
If anything it would be more logical to equate Felix with the tortured person and treat this as a reductio ad absurdum of your position on the dust speck problem. (But that would be wrong too, since the numbers aren't actually the problem with Felix, the fact that there's an incentive to manipulate your own utility function that way is (among other things).)
Comment author:Dmytry
06 April 2012 11:22:26AM
*
0 points
[-]
You aren't seeing forest for the trees... the thing that is identical is that you are trading utilities across people, which is fundamentally problematic and leads to either tortured child or utility monster, or both.
Comment author:WrongBot
06 April 2012 05:22:09PM
4 points
[-]
Omelas is a goddamned paradise. Omelas without the tortured child would be better, yeah, but Omelas as described is still better than any human civilization that has ever existed. (For one thing, it only contains one miserable child.)
Comment author:Dmytry
06 April 2012 05:23:28PM
*
-2 points
[-]
Well it seems to me they are trading N dust specks vs torture in Omelas. edit: Actually, I don't like Omelas [as example]. I think that miserable child would only make the society way worse, with the people just opting to e.g. kill someone when it ever so slightly results in increase in their personal expected utility. This child in Omelas puts them straight on the slippery slope, and making everyone aware of slippage makes people slide down for fun and profit.
Our 'civilization' though, of course, is a god damn jungle and so its pretty damn bad. It's pretty hard to beat on the moral wrongness scale, from first principles; you have to take our current status quo and modify it to get to something worse (or take our earlier status quo).
Comment author:WrongBot
06 April 2012 07:26:06PM
*
1 point
[-]
Your edit demonstrates that you really don't get consequentialism at all. Why would making a good tradeoff (one miserable child in exchange for paradise for everyone else) lead to making a terrible one (a tiny bit of happiness for one person in exchange for death for someone else)?
Comment author:FAWS
06 April 2012 11:41:37AM
0 points
[-]
the thing that is identical is that you are trading utilities across people,
This is either wrong (the utility functions of the people involved aren't queried in the dust speck problem) or so generic as to be encompassed in the concept of "utility calculation".
Aggregating utility functions across different people is an unsolved problem, but not necessarily an unsolvable one. One way of avoiding utility monsters would be to normalize utility functions. The obvious way to do that leads to problems such as arachnophobes getting less cake even if they like cake equally much, but IMO that's better than utility monsters.
Comment author:Dmytry
06 April 2012 11:49:13AM
*
2 points
[-]
This is either wrong (the utility functions of the people involved aren't queried in the dust speck problem) or so generic as to be encompassed in the concept of "utility calculation".
The utilities of many people are a vector, you are to map it to a scalar value, that loses a lot of information in process, and it seems to me however you do it, leads to some sort of objectionable outcomes. edit: I have a feeling one could define it reasonably with some sort of Kolmogorov complexity like metric that would grow incredibly slowly for the dust specks and would never equate what ever hideously clever thing does our brain do to most of the neurons when we suffer; the suffering beating the dust specks on the complexity (you'd have to write down the largest number you can write down in as many bits as the bits being tortured in the brain; then that number of dust specks starts getting to the torture level). We need to understand how pain works before we can start comparing pain vs dust specks.
Comment author:billswift
06 April 2012 05:22:41PM
0 points
[-]
but not necessarily an unsolvable one.
Really? Every use of utilities I have seen either uses a real world measure (such as money) with a notation that it isn't really utilities or they go directly for the unfalsifiable handwaving. So far I haven't seen anything to suggest "aggregating utility functions" is even theoretically possible. For that matter most of what I have read suggests that even an individual's "utility function" is usually unmanageably fuzzy, or even unfalsifiable, itself.
Comment author:[deleted]
06 April 2012 01:04:17PM
*
3 points
[-]
In one case, (Torture to avoid the specks) the larger portion of people is better off if you pick the single person. In the other case, (Build pyramids to please Felix) the larger portion of people is worse off if you pick the single person.
So if my position was "The majority should win" It would be right to torture the person and it would be right to depose Felix.
I'm not sure if it's a fundamental difference or a good difference, but I think that means I can lay out the following 4 distinct answer pairs:
Depose Felix, Torture Man: Majority wins.
Adore Felix, Speck people: Minority wins.
Adore Felix, Torture Man: Mean Happiness wins.
Depose Felix, Speck People: Minimum happiness wins. (Assuming either Felix is happier about being deposed than an average person with a dust speck in their eye, or dead, and no longer counted for minimum happiness.)
So I think I can see all 4 distinct positions, if I'm not missing something.
Comment author:Thomas
06 April 2012 01:57:47PM
*
1 point
[-]
In one case, (Torture to avoid the specks) the larger portion of people is better off if you pick the single person. In the other case, (Build pyramids to please Felix) the larger portion of people is worse off if you pick the single person.
Imagine that there is one tortured for 50 years and then free of any dust speck for the next 3^^^3 years.
Then we don't have "the larger portion of people" anymore. Is anything different in such a case?
Imagine that there is one tortured for 50 years and then free of any dust speck for the next 3^^^3 years.
If I understand the dilemma, in your most recent phrasing, it's this: A person who lives 3^^^3 years either:
a) has to suffer a dustspeck per year
b) has to suffer 50 years of torture at some point in that time, then I assume gets the memory of that torture deleted from his mind and his mind's state restored to what it was before the torture (so that he doesn't suffer further disutility from that memory or the broken mind-state, he only has to suffer the torture itself), He lives the remaining 3^^^3 years dustspeck-free.
If we don't know what his own preferences are, and have no way of asking him, what should we choose on his behalf?
Can we have one dilemma at a time, please, Thomas? You said something about 3^^^3 years -- therefore you're not talking about the dilemma as stated in the original sequence, as that dilemma doesn't say anything about 3^^^3 years.
Which preferences are in question now?
The preferences relating to the original dilemma, are the preferences of the person who presumably prefers not to get tortured, vs the preferences of 3^^^3 people who presumably prefer not to get a dust speck in the eye.
Comment author:[deleted]
06 April 2012 02:50:20PM
1 point
[-]
Well, first of all, I'm assuming that you're doing that to both groupings (since otherwise I could say "Well, one has only one person and one has a massive number of people, which is a difference." but that seems like a trivial point)
So if you apply it to both, then it's just one person considering tradeoff A, (pay torture to go speck free for eons)
And another person considering tradeoff B(personally build pyramids for eons to get to live in your own collection of pyramids for some years.)
I could say that in once case the pain is relatively dense (torture, condensed to 50 years) and the pleasure is relatively sparse,(speck free, over 3^^^3 years) and that in the other case the pain relatively sparse (slave labor, spread out over a long time) and the pleasure is relatively dense (Incomprehensible pyramidgasm.).
I'm not sure if that matters or in what ways that difference matters. I'm really not up to date on how your brain handles that specifically and would probably need to look it up further.
Comment author:Thomas
06 April 2012 03:08:16PM
1 point
[-]
personally build pyramids for eons to get to live in your own collection of pyramids for some years.)
No. Building pyramids as humans. And enjoying them much, much longer as they stand there, for Felix. Enjoyed by Felix.
Maybe the amount of our pleasure with Giza pyramids already exceeded the pain invested to build them. I don't know.
Can all the pains of a slave be justified by all the pleasures of the tourist, visiting the hole in the rock, he was forced carving for 50 years?
Or can a large group of sick sadists are entitled to slowly torture someone, since their pleasure sum will be greater than the pain of the unlucky one?
Comment author:Rhwawn
06 April 2012 08:12:17PM
*
1 point
[-]
Maybe the amount of our pleasure with Giza pyramids already exceeded the pain invested to build them. I don't know. Can all the pains of a slave be justified by all the pleasures of the tourist, visiting the hole in the rock, he was forced carving for 50 years?
Was it that much pain? I read in National Geographic, IIRC, that the modern archaeological conception was that the pyramids were mostly or entirely built by paid labor - Nile farmers killing time during the dry season. This may even be a good thing, depending on whether it diverted imperial tax revenue from foreign adventurism into monument/tomb-building.
Being very very outraged isn't really an argument.
Give us your own (non-utilitarian I assume) decision theory that you consider encapsulating all that is good and moral, if you please.
If you can't, please stop being outraged as those of us who try to solve the problem, even if you feel we've taken wrong turns in the path towards the solution.
I don't know, 3^^^3 is a pretty long time to fix brain trauma. Or are you offering complete restoration after the torture? In that case, I might just take it.
Comment author:Thomas
06 April 2012 07:39:23PM
-1 points
[-]
I am not offering anything at all. I strongly advice you NOT to substitute the slight discomfort over long time period with a horrible torture for a shorter period.
What's the fundamental difference between those two cases? I don't see it, do you?
One fundamental difference is that I don't care about Felix's further happiness. After some point, I may even resent it, which would make his additional happiness of negative utility to me.
Another difference is that happiness may be best represented as a percentage with an upper bound of e.g. 100% happy, rather than be an integer you can keep adding to without end.
I think Felix's case may be an interesting additional scenario to consider, in order to be sure that AIs don't fall victims to it (e.g. by creating a superintelligence and making it super-happy, to the expense of normal human happiness). But it's not the same scenario as the specks.
Comment author:Dmytry
06 April 2012 11:20:28AM
*
1 point
[-]
The FAI should make a drug which will make you happy for Felix. edit: to clarify. The two choices here are not happy naturally vs happy via wireheading. The two choices are intense AI-induced 'natural' unhappiness, vs drug induced happiness. It's similar to having your hand amputated, with or without 'wireheading', err, painkillers. I think it is pretty clear that if you have someone's hand amputated, it is better if they can't feel it and see it. Be careful with non-wireheading FAIs, 'less all surgery will be without anaesthesia (perhaps with only the muscle relaxant).
Comment author:Dmytry
06 April 2012 11:25:40AM
*
1 point
[-]
Well, in some sense, achieving happiness by anything other than reproduction, is already wireheading. Doesn't need to be with a wire; what if I make a video which evokes intense feeling of pleasure? How far you can go before it is a mind hack?
edit: actually, I think the AI could raise people to be very empathetic for Felix, and very happy for him. Is it not good to raise your kids so that they can be happy in the world the way it is (when they can't change anything anyway) ?
Comment author:roystgnr
06 April 2012 04:10:16PM
2 points
[-]
"achieving happiness by anything other than [subgoals of] reproduction" is wireheading from the perspective of my genes, and if they want to object I'm not stopping them. Happiness via drugs is wireheading from the perspective of me, and I object myself.
Well, in some sense, achieving happiness by anything other than reproduction, is already wireheading.
No. This reduces the words to the point of meaninglessness. Human beings have values other than reproduction, values that make them happy when satisfied - art, pride, personal achievement, understanding, etc. Wireheading is about being made happy directly, regardless of the satisfaction of the various values.
The scenario previously discussed about Felix is that he was happy and everyone else suffered.
Now you're posing a scenario where everyone is happy, but they're made happy by having their values rewritten to place extremelty value on Felix's happiness instead.
At this point, I hope we're not pretending it's the same scenario with only minor modifications, right? Your scenario is about the AI rewriting our values, it's not about trading our collective suffering for Felix's happiness.
Your scenario can effectively remove the person of Felix from the situation altogether, and the AI could just make us all very happy that the laws of physics keep on working.
Comment author:dugancm
06 April 2012 11:51:34PM
*
0 points
[-]
Happiness, as a state of mind in humans, seems less to me about how strong the "orgasms" are than how frequently they occur without lessening the probability they will continue to occur. So what problems might there be with maximizing total future happy seconds experienced in humans, including emulations thereof (other than describing with sufficient accuracy the concepts of 'human' and 'happiness' to a computer)?
I think doing so would extrapolate to increasing population and longevity to within resource constraints and diminishing returns on improving average happiness uptime and existential risk mitigation, which seem to me to be the crux of people's intuitions about the Felix and Wireheading problems.
Comment author:Dmytry
06 April 2012 10:18:49AM
*
-7 points
[-]
Hahaha.
Seriously though, either the Moral Universalism (and absolutism) is correct, in which case we could make an AI that would by itself develop very agreeable universal moral code, similar to how you can do it for mathematics or laws of physics (instead of us trying to implement our customs into AI), or it is incorrect, there's no way to absolute moral code, and any FAI is going to be a straitjacket of humanity, at best implementing (some of) our customs and locking those in, and at worst implementing and enforcing something else like in that comic.
Comment author:Incorrect
06 April 2012 12:31:23PM
4 points
[-]
Saying no FAI exists in design space that could satisfy us is equivalent to saying nothing can satisfy us. In other words, if you are correct then the AI isn't the problem and humanity would be "straitjacketed" anyway.
Saying we could never build an AI that would satisfy us because of the technical difficulty is plausible, but I don't think that's what you are saying.
Comment author:Dmytry
06 April 2012 12:42:52PM
*
0 points
[-]
Saying no FAI exists in design space that could satisfy us is equivalent to saying nothing can satisfy us. In other words, if you are correct then the AI isn't the problem and humanity would be "straitjacketed" anyway.
I don't see how not being fully satisfied is a straitjacket. I'm saying that our (the mankind) maximum satisfaction may be when straitjacketed, because mankind isn't sane (and if there isn't any truly sane morality system edit: to clarify. if there is truly sane morality system, then mankind can be cured of insanity).
Comment author:Incorrect
06 April 2012 12:59:16PM
1 point
[-]
I was using the term "satisfied" to include all human preferences, including the desire to not be "straitjacketed".
If human preferences are inconsistent then humans still can't do any better than an AI for there is an AI in design space that does nothing in our world but would make similar worlds look exactly like ours.
Comment author:Dmytry
06 April 2012 01:09:27PM
*
0 points
[-]
You assume that the utility of two different worlds can not be exactly equal. edit: or maybe you don't. In any case, this AI which does absolutely nothing in our world is no more useful than AI that does nothing in all possible worlds, or just a brick.
Also, the desire for mankind (and life) not to be straitjacketed, is my view, i'm not sure it is coherently shared by mankind, and in fact i'm not even sure i like the way it is going if it is not straitjacketed in some way. edit: to clarify. I like the heuristics of maximizing the future choices for me. It is part of my values, that i don't want removed. I don't like [consequences of] this heuristic for mankind. Mankind is a meta-organism that is dumb and potentially self destructive.
edit: To clarify. What I am saying, is that there's conflict between two values whose product matters. Survival vs freedom. Survival without freedom is bad. Freedom without survival is nonsense.
Comment author:Incorrect
06 April 2012 01:19:23PM
0 points
[-]
this AI which does absolutely nothing in our world is no more useful than AI that does nothing in all possible worlds, or just a brick.
Sorry, I wasn't being clear. The point was saying that no AI can do better than humanity implies that our world is optimal out of all similar worlds. (I believe there are much stronger arguments than this against what you are saying, but this one should suffice)
Comment author:Dmytry
06 April 2012 01:26:09PM
*
0 points
[-]
It only implies so if your AI is totally omniscient.
edit: Anyhow, I can of course think of AI that can do better than humanity: the AI sits inside Jupiter, and nudges away any incoming comets and asteroids, and that's it (then as sun burns up then burns out, moves Earth around). The problem starts when you make the AI discriminate between very similar worlds. edit: and even that asteroid stopping AI may be a straitjacket to intelligent life as it may be that the mankind is a wrong thing entirely, and should be permitted to kill itself, and then the meteorite impacts should be allowed so that ants get a chance.
Comment author:Incorrect
06 April 2012 01:44:58PM
0 points
[-]
as it may be that the manking is a wrong thing entirely, and should be permitted to kill itself, and then the meteorite impacts should be allowed so that ants get a chance.
I don't know much about my own extrapolated preferences but I can reason that as my preferences are the product of noise in the evolutionary process, reality is unlikely to align with them naturally. It's possible that my preferences consider "mankind a wrong thing entirely"; but that they would align with whatever the universe happens to produce next on earth (assuming the rise of another dominant species is even plausible) is incredibly unlikely. Anything that happens without a causal line of descent from human values is unlikely to align with human values.
Comment author:Dmytry
06 April 2012 01:53:53PM
*
0 points
[-]
Anything that happens without a causal line of descent from human values is unlikely to align with human values.
Unlikely to align how exactly? There's also the common causes, you know; A and B can be correlated when A causes B, when B causes A, or when C causes A and B.
It seems to me that you can require arbitrary degree of alignment to arrive at arbitrary unlikehood, but some alignment via common cause is nonetheless probable.
Comment author:Dmytry
06 April 2012 11:09:09AM
*
0 points
[-]
Moral Universalism could be true in some sense, but not automatically compelling, and the AI would need to be programmed to find and/or follow it.
My original post had this possibility. Where you make the AI that develops much of the morality (which it would really have to). edit: note that the AI in question may be just a theorem prover which tries to find some universal moral axioms, but is not itself moral or compelled to implement anything in real world.
There could be a uniquely specified human morality that fulfills much of the same purpose Moral Universalism does for humans.
What's in 10 millions years? 100 millions? A straitjacket for intelligent life.
It might be possible to specify what we want in a more dynamic way than freezing in current customs.
We would still want some limits from our values right now, e.g. so that the society wouldn't steer itself to suicide somehow. Even rules like 'it is good if 99% of people agree with it" can steer us into some really nasty futures over the time. Other issue is the possibility of de-evolution of human intelligence. We would not want to lock in all the customs, but some of the values of the today, would get frozen in.
Comment author:Dmytry
06 April 2012 10:49:44AM
*
-2 points
[-]
Name 1 then.
edit: and it's not even a dichotomy. There's the hypothetical AIs which implement some moral absolute that is good for all cultures, possible cultures, and everyone, which we would invent, aliens would invent, whatever we evolve into could invent, etc. If those do not exist, then what exists that isn't to some extent culturally specific to h. Sapiens circa today?
Comment author:wedrifid
07 April 2012 02:21:44AM
0 points
[-]
Name 1 then.
The Unobtrusive Guardian. An FAI that concludes that humanity's aversion to being 'straightjacketed' is such that it is never ok for it to interfere with what humans do themselves. It proceeds to navigate itself out of the way and wait until it spots an external threat like a comet or hostile aliens. It then destroys those threats.
(The above is not a recommended FAI design. It is a refutation by example of an absolute claim that would exclude the above.)
Comment author:Dmytry
07 April 2012 04:48:22AM
-1 points
[-]
didn't i myself describe it and outline how this one also limits opportunities normally available to evolution for instance? It's to very little extent a straitjacket to life, as it does very little.
Comment author:Bugle
12 July 2012 08:48:47PM
*
1 point
[-]
Everyone's talking about this as if it was a hypothetical, but as far as I can tell it describes pretty accurately how hierarchical human civilizations tend to organize themselves once they hit a certain size. Isn't a divine ruler precisely someone who is more deserving and more able to absorb resources? Aren't the lower orders people who would not appreciate luxuries and indeed have fully internalized such a fact ("Not for the likes of me")
If you skip the equality requirement, it seems history is full of utilitarian societies.
= 27d567bc2d48d9f40e9ccc20ea370b15)
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
= f037147d6e6c911a85753b9abdedda8d)
Comments (113)
In case anyone is unfamiliar with the concept: Utility Monster.
Felix means happy (or lucky), and is the origin of the word felicity. It took me a while to realize this, so I thought I would note it. Is it obvious for all native English speakers?
Not obvious to me. I did know the meaning of Felix, but it's deep enough in the unused drawers of my memory that I might never have made the connection without someone pointing it out.
It was obvious to me, I'm not a native English speaker. Anyone knowing a bit of Latin is probably going to catch it.
I am a native English speaker, but, yeah, without the Latin I probably wouldn't have noticed.
Not obvious to me. I honestly would never have made the connection.
Obvious to me. Native speaker.
It's a total-utility maximising AI.
if it was a total utility maximizing AI it would clone the utility monster (or start cloning everyone else if the utility monster is super linear) edit: on the other hand, if it was average utility maximizing AI it would kill everyone else leaving just the utility monster. In any case there'd be some serious population 'adjustment'.
Not if that made the utility monster unhappy.
It doesn't have to tell the monster. (this btw is one wireheading-related issue; i do quite hate the lingo here though; calling it wireheaded makes it sound like there isn't a couple thousands years of moral philosophy about the issue and related issues)
I'm not aware of an alternative to "wireheading" with the same meaning.
Go classical - 'lotus-eating'.
Good one.
http://en.wikipedia.org/wiki/Lotus-eaters
That's the ancient greeks writing about hypothetical wireheads. (the 'moral philosophy' is perhaps a bad choice of word for search for greek stuff; ethics is the greek word)
A bit of search around that showed nearly no reference to lotus eating/lotus eater in moral philosophy.
Something much closer to "wireheading" would be hedonism, and more specifically Nozick's Experience Machine, which is pretty much wireheading, but isn't thousands of years old, and has been referenced here.
(And the term "wirehead" as used here probably comes from the Known Space stories, so probably predates Nozick's 1974 book)
I don't think you looked very hard - I turned up a few books apparently on moral philosophy by searching in Google Books for 'moral ("lotus eating" OR "lotus-eating" OR "lotus eater" OR "lotus-eater")'.
And yes, I'm pretty sure the wirehead term comes from Niven's Known Space. I've never seen any other origin discussed.
Well, for one thing, it ought to be obvious that Mohammed would have banned a wire into the pleasure centre, but lacking the wires, he just banned the alcohol and other intoxicants. The concept of 'wrong' ways of seeking the pleasure is very, very old.
It would be awfully hard to hide.
Sure, it could lock the monster in an illusory world of optimal happiness, or just stimulate his pleasure centers directly, etc. But unless we assume that the AI is working under constraints that prevent that sort of thing, the comic doesn't make much sense.
There's no clear line between 'hiding' and 'not showing'. You can leave just a million people or so, to be put around the monster, and simply not show him the rest. It is not like the AI is making every wall into the screen displaying the suffering on the construction of pyramids. Or you can kill those people and show it in such a way that the monster derives pleasure from it. At any rate, anyone whose death would go unnoticed by the monster, or whose death does not sufficiently distress the monster, would die, if the AI is to focus on average pleasure.
edit: I think those solutions really easily come to mind when you know of what a soviet factory would do to exceed the five year plan.
The AI explicitly wasn't focused on average pleasure, but on total pleasure, as measured by average pleasure times the population.
Yep. I was just posting on what average pleasure maximizing AI would do, that isn't part of the story.
You're all wrong — if the happiness of the utility monster compounds as the comic says, then you get greater happiness out of lumping it all into one monster rather than cloning.
Whoops. Panel 3 (y axis caption) and 6 (suicide not allowed) indeed make that clear.
Why are you wasting your time on-line? Felix wants more pyramids.
Chain gangs strike me as sub-optimal for building pyramids or total happiness.
Clearly, Felix prefers pyramids built by chain-gangs.
The latest SMBC comic is now an illustrated children's story which more or less brings up parallel thoughts to Cynical about Cynicism.
It's a GOOD life.
It is a good thing that you are thinking good things about Felix. This means he is happier if you aren't in corn field since you are a good person with no bad thoughts.
I'm not sure why the down vote.
If it helps, Konkavistador and I are referring to a classic horror story called "It's a Good Life".
Felix is 3^^^3 units happy. And no dust speck in his eyes. What is torturing millions for this noble goal?
I, of course, reject that "sequence" which preaches exactly this.
That's because your brain doesn't have the ability to imagine just how happy Felix is and fails to weigh his actual happiness against humanity's.
And since I don't want that ability I think we are still fine. At the end of the day I'm perfectly ok with not caring about Felix that much.
BTW, would anyone have a one on one chat with me about the dust speck argument?
Which sequence is that?
This one.
I am not sure if it counts into "The Sequence", I guess it does.
The problem is the line of reasoning, where a "50 years of torture" is better than 3^^^3 years with a dust speck in the eye every so often.
What is then the torture of all the Humanity, against the super happy Felix with 3^^^3 pyramids. Nothing. By the same line of reasoning.
It's hedonistic total-utilitarianism vs preference based consequentialism. That's a big difference. Not only would the 'sequence' you reject not advocate preferring to torture humanity for the sake of making Felix superhappy, even in the absence of negative externalities it would still consider that sort of 'happiness' production a bad thing even for Felix.
That's not even the dilemma you linked to. The dilemma you linked to "one person be horribly tortured for fifty years without hope or rest, or that 3^^^3 people get dust specks in their eyes".
It's probably bad practice to say two lines of reasoning are the same line of reasoning, if you don't believe in either of them.
For starters I don't need to have a positive factor for Felix's further happiness in my utilty function. That alone is a significant difference.
Look. You have one person, under terrible torture for 50 years on one side and a gazillion of people with a slight discomfort every year or so on the other side.
It is claimed that the first is better.
Now, you have a small humanity as is, only enslaved for pyramid building for Felix. He has eons of subjective time to enjoy this pyramids and he is unbelievably happy. More happy than any man, woman or child could ever be. The amount of happiness of Felix outweights the misery of billion of people by a factor of a million.
What's the fundamental difference between those two cases? I don't see it, do you?
Felix is essentially a Utility Monster: a thought experiment that's been addressed here before. As that family of examples shows, happiness-maximization breaks down rather spectacularly when you start considering self- or other-modification or any seriously unusual agents. You can bite that bullet, if you want, but not many people here do; fortunately, there are a few other ways you can tackle this if you're interested in a formalization of humanlike ethics. The "Value Stability and Aggregation" post linked above touches on the problem, for example, as does Eliezer's Fun Theory sequence.
You don't need any self-modifying or non-humanlike agents to run into problems related to "Torture vs. Dust Specks", though; all you need is to be maximizing over the welfare of a lot of ordinary agents. 3^^^3 is an absurdly huge number and leads you to a correspondingly counterintuitive conclusion (one which, incidentally, I'd estimate has led to more angry debate than anything else on this site), but lesser versions of the same tradeoff are quite realistic; unless you start invoking sacred vs. profane values or otherwise define the problem away, it differs only in scale from the same utilitarian calculations you make when, say, assigning chores.
The only similarity between those cases is that they involve utility calculations you disagree with. Otherwise every single detail is completely different. (e. g. the sort of utility considered, two negative utilities being traded against each other vs. trading utility elsewhere (positive and negative) for positive utility, which side of the trade the single person with the large individual utility difference is on, the presence of perverse incentives, etc, etc).
If anything it would be more logical to equate Felix with the tortured person and treat this as a reductio ad absurdum of your position on the dust speck problem. (But that would be wrong too, since the numbers aren't actually the problem with Felix, the fact that there's an incentive to manipulate your own utility function that way is (among other things).)
You aren't seeing forest for the trees... the thing that is identical is that you are trading utilities across people, which is fundamentally problematic and leads to either tortured child or utility monster, or both.
Omelas is a goddamned paradise. Omelas without the tortured child would be better, yeah, but Omelas as described is still better than any human civilization that has ever existed. (For one thing, it only contains one miserable child.)
Well it seems to me they are trading N dust specks vs torture in Omelas. edit: Actually, I don't like Omelas [as example]. I think that miserable child would only make the society way worse, with the people just opting to e.g. kill someone when it ever so slightly results in increase in their personal expected utility. This child in Omelas puts them straight on the slippery slope, and making everyone aware of slippage makes people slide down for fun and profit.
Our 'civilization' though, of course, is a god damn jungle and so its pretty damn bad. It's pretty hard to beat on the moral wrongness scale, from first principles; you have to take our current status quo and modify it to get to something worse (or take our earlier status quo).
Your edit demonstrates that you really don't get consequentialism at all. Why would making a good tradeoff (one miserable child in exchange for paradise for everyone else) lead to making a terrible one (a tiny bit of happiness for one person in exchange for death for someone else)?
This is either wrong (the utility functions of the people involved aren't queried in the dust speck problem) or so generic as to be encompassed in the concept of "utility calculation".
Aggregating utility functions across different people is an unsolved problem, but not necessarily an unsolvable one. One way of avoiding utility monsters would be to normalize utility functions. The obvious way to do that leads to problems such as arachnophobes getting less cake even if they like cake equally much, but IMO that's better than utility monsters.
The utilities of many people are a vector, you are to map it to a scalar value, that loses a lot of information in process, and it seems to me however you do it, leads to some sort of objectionable outcomes. edit: I have a feeling one could define it reasonably with some sort of Kolmogorov complexity like metric that would grow incredibly slowly for the dust specks and would never equate what ever hideously clever thing does our brain do to most of the neurons when we suffer; the suffering beating the dust specks on the complexity (you'd have to write down the largest number you can write down in as many bits as the bits being tortured in the brain; then that number of dust specks starts getting to the torture level). We need to understand how pain works before we can start comparing pain vs dust specks.
Really? Every use of utilities I have seen either uses a real world measure (such as money) with a notation that it isn't really utilities or they go directly for the unfalsifiable handwaving. So far I haven't seen anything to suggest "aggregating utility functions" is even theoretically possible. For that matter most of what I have read suggests that even an individual's "utility function" is usually unmanageably fuzzy, or even unfalsifiable, itself.
In one case, (Torture to avoid the specks) the larger portion of people is better off if you pick the single person. In the other case, (Build pyramids to please Felix) the larger portion of people is worse off if you pick the single person.
So if my position was "The majority should win" It would be right to torture the person and it would be right to depose Felix.
I'm not sure if it's a fundamental difference or a good difference, but I think that means I can lay out the following 4 distinct answer pairs:
Depose Felix, Torture Man: Majority wins.
Adore Felix, Speck people: Minority wins.
Adore Felix, Torture Man: Mean Happiness wins.
Depose Felix, Speck People: Minimum happiness wins. (Assuming either Felix is happier about being deposed than an average person with a dust speck in their eye, or dead, and no longer counted for minimum happiness.)
So I think I can see all 4 distinct positions, if I'm not missing something.
Edit: Fixed spacing.
Imagine that there is one tortured for 50 years and then free of any dust speck for the next 3^^^3 years.
Then we don't have "the larger portion of people" anymore. Is anything different in such a case?
If I understand the dilemma, in your most recent phrasing, it's this: A person who lives 3^^^3 years either:
a) has to suffer a dustspeck per year
b) has to suffer 50 years of torture at some point in that time, then I assume gets the memory of that torture deleted from his mind and his mind's state restored to what it was before the torture (so that he doesn't suffer further disutility from that memory or the broken mind-state, he only has to suffer the torture itself), He lives the remaining 3^^^3 years dustspeck-free.
If we don't know what his own preferences are, and have no way of asking him, what should we choose on his behalf?
But what does this have to do with Felix?
It is argued in the said sequence, how much better is to have 1 tortured for 50 years, than 3^^^3 people having slight discomfort.
Which preferences are in question now?
Can we have one dilemma at a time, please, Thomas? You said something about 3^^^3 years -- therefore you're not talking about the dilemma as stated in the original sequence, as that dilemma doesn't say anything about 3^^^3 years.
The preferences relating to the original dilemma, are the preferences of the person who presumably prefers not to get tortured, vs the preferences of 3^^^3 people who presumably prefer not to get a dust speck in the eye.
Well, first of all, I'm assuming that you're doing that to both groupings (since otherwise I could say "Well, one has only one person and one has a massive number of people, which is a difference." but that seems like a trivial point)
So if you apply it to both, then it's just one person considering tradeoff A, (pay torture to go speck free for eons)
And another person considering tradeoff B(personally build pyramids for eons to get to live in your own collection of pyramids for some years.)
I could say that in once case the pain is relatively dense (torture, condensed to 50 years) and the pleasure is relatively sparse,(speck free, over 3^^^3 years) and that in the other case the pain relatively sparse (slave labor, spread out over a long time) and the pleasure is relatively dense (Incomprehensible pyramidgasm.).
I'm not sure if that matters or in what ways that difference matters. I'm really not up to date on how your brain handles that specifically and would probably need to look it up further.
No. Building pyramids as humans. And enjoying them much, much longer as they stand there, for Felix. Enjoyed by Felix.
Maybe the amount of our pleasure with Giza pyramids already exceeded the pain invested to build them. I don't know.
Can all the pains of a slave be justified by all the pleasures of the tourist, visiting the hole in the rock, he was forced carving for 50 years?
Or can a large group of sick sadists are entitled to slowly torture someone, since their pleasure sum will be greater than the pain of the unlucky one?
I don't think so.
Was it that much pain? I read in National Geographic, IIRC, that the modern archaeological conception was that the pyramids were mostly or entirely built by paid labor - Nile farmers killing time during the dry season. This may even be a good thing, depending on whether it diverted imperial tax revenue from foreign adventurism into monument/tomb-building.
I've heard that the labourers who made the pyramids were actually quite well paid.
Being very very outraged isn't really an argument.
Give us your own (non-utilitarian I assume) decision theory that you consider encapsulating all that is good and moral, if you please.
If you can't, please stop being outraged as those of us who try to solve the problem, even if you feel we've taken wrong turns in the path towards the solution.
I don't know, 3^^^3 is a pretty long time to fix brain trauma. Or are you offering complete restoration after the torture? In that case, I might just take it.
I am not offering anything at all. I strongly advice you NOT to substitute the slight discomfort over long time period with a horrible torture for a shorter period.
One fundamental difference is that I don't care about Felix's further happiness. After some point, I may even resent it, which would make his additional happiness of negative utility to me.
Another difference is that happiness may be best represented as a percentage with an upper bound of e.g. 100% happy, rather than be an integer you can keep adding to without end.
I think Felix's case may be an interesting additional scenario to consider, in order to be sure that AIs don't fall victims to it (e.g. by creating a superintelligence and making it super-happy, to the expense of normal human happiness). But it's not the same scenario as the specks.
The FAI should make a drug which will make you happy for Felix. edit: to clarify. The two choices here are not happy naturally vs happy via wireheading. The two choices are intense AI-induced 'natural' unhappiness, vs drug induced happiness. It's similar to having your hand amputated, with or without 'wireheading', err, painkillers. I think it is pretty clear that if you have someone's hand amputated, it is better if they can't feel it and see it. Be careful with non-wireheading FAIs, 'less all surgery will be without anaesthesia (perhaps with only the muscle relaxant).
Cute, but that's effectively the well-known scenario of Wireheading where the complexity of human value is replaced by mere 'happiness'.
Well, in some sense, achieving happiness by anything other than reproduction, is already wireheading. Doesn't need to be with a wire; what if I make a video which evokes intense feeling of pleasure? How far you can go before it is a mind hack?
edit: actually, I think the AI could raise people to be very empathetic for Felix, and very happy for him. Is it not good to raise your kids so that they can be happy in the world the way it is (when they can't change anything anyway) ?
"achieving happiness by anything other than [subgoals of] reproduction" is wireheading from the perspective of my genes, and if they want to object I'm not stopping them. Happiness via drugs is wireheading from the perspective of me, and I object myself.
No. This reduces the words to the point of meaninglessness. Human beings have values other than reproduction, values that make them happy when satisfied - art, pride, personal achievement, understanding, etc. Wireheading is about being made happy directly, regardless of the satisfaction of the various values.
The scenario previously discussed about Felix is that he was happy and everyone else suffered. Now you're posing a scenario where everyone is happy, but they're made happy by having their values rewritten to place extremelty value on Felix's happiness instead.
At this point, I hope we're not pretending it's the same scenario with only minor modifications, right? Your scenario is about the AI rewriting our values, it's not about trading our collective suffering for Felix's happiness.
Your scenario can effectively remove the person of Felix from the situation altogether, and the AI could just make us all very happy that the laws of physics keep on working.
Happiness, as a state of mind in humans, seems less to me about how strong the "orgasms" are than how frequently they occur without lessening the probability they will continue to occur. So what problems might there be with maximizing total future happy seconds experienced in humans, including emulations thereof (other than describing with sufficient accuracy the concepts of 'human' and 'happiness' to a computer)?
I think doing so would extrapolate to increasing population and longevity to within resource constraints and diminishing returns on improving average happiness uptime and existential risk mitigation, which seem to me to be the crux of people's intuitions about the Felix and Wireheading problems.
Everyone's talking about this as if it was a hypothetical, but as far as I can tell it describes pretty accurately how hierarchical human civilizations tend to organize themselves once they hit a certain size. Isn't a divine ruler precisely someone who is more deserving and more able to absorb resources? Aren't the lower orders people who would not appreciate luxuries and indeed have fully internalized such a fact ("Not for the likes of me")
If you skip the equality requirement, it seems history is full of utilitarian societies.
Another good one on Ethics