= 27d567bc2d48d9f40e9ccc20ea370b15)
enoonsti comments on Cryonics Questions - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (165)
I apologize for the confusion and I understand if you're frustrated; I experience that frustration quite often once I realize I'm talking past someone. For whatever it's worth, I left it open because the curious side of me didn't want to limit Yvain; that curious side wanted to hear his thoughts in general. So... I guess both #2 and #3 (I'm not sure how #1 and #4 could be deduced from my posts, but my opinion is irrelevant to this situation). Anyways, I didn't mean to push this too much, because I felt it was minor. Perhaps I should not have asked it in the first place.
Also, thank you for being honest (admittedly, I was tempted to say, "So you weren't being honest with your other posts?" but I decided to present that temptation passively inside these parentheses)
:)
Ok, we're cool. Regarding my own opinions/postings, I said I'm not signing up, but my opinions on FAI or UFAI had nothing to do with it. Well, maybe I did implicitly express skepticism that FAI will create a utopia. What the hell! I'll express that skepticism explicitly right now, since I'm thinking of it. There is nothing an FAI can do to eliminate human misery without first changing human nature. An FAI that tries to change human nature is an UFAI.
But I would like my nature changed in some ways. If an AI does that for me, does that make it unFriendly?
No, that is your business. But if you or the AI would like my nature changed, or the nature of all yet-to-be-born children ...
If you have moral objections to altering the nature of potential future persons that have not yet come into being, then you had better avoid becoming a teacher, or interacting at all with children, or saying or writing anything that a child might at some point encounter, or in fact communicating with any person under any circumstances whatsoever.
I have no moral objection to any person of limited power doing whatever they can to influence future human nature. I do have an objection to that power being monopolized by anyone or anything. It is not so much that I consider it immoral, it is that I consider it dangerous and unfriendly. My objections are, in a sense, political rather than moral.
What threshold of power difference do you consider immoral? Do you have a moral objection to pickup artists? Advertisers? Politicians? Attractive people? Toastmasters?
Where do you imagine that I said I found something immoral? I thought I had said explicitly that morality is not involved here. Where do I mention power differences? I mentioned only the distinction between limited power and monopoly power.
When did I become the enemy?
Sorry, I shouldn't have said immoral, especially considering the last sentence in which you explicitly disclaimed moral objection. I read "unfriendly" as "unFriendly" as "incompatible with our moral value systems".
Please read my comment as follows:
I simply don't understand why the question is being asked. I didn't object to power differences. I objected to monopoly power. Monopolies are dangerous. That is a political judgment. Your list of potentially objectionable people has no conceivable relationship with the subject matter we are talking about, which is an all-powerful agent setting out to modify future human nature toward its own chosen view of the desirable human nature. How do things like pickup artists even compare? I'm not discussing short term manipulations of people here. Why do you mention attractive people? I seem to be in some kind of surreal wonderland here.
Although you're right (except for the last sentence, which seems out of place), you didn't actually answer the question, and I suspect that's why you're being downvoted here. Sub out "immoral" in Pavitra's post for "dangerous and unfriendly" and I think you'll get the gist of it.
To be honest, no, I don't get the gist of it. I am mystified. I consider none of them existentially dangerous or unfriendly. I do consider a powerful AI, claiming to be our friend, who sets of to modify human nature for our own good, to be both dangerous (because it is dangerous) and unfriendly (because it is doing something to people which people could well do to themselves, but have chosen not to).
We may assume that an FAI will create the best of all possible worlds. Your argument seems to be that the criteria of a perfect utopia do not correspond to a possible world; very well then, an FAI will give us an outcome that is, at worst, no less desirable than any outcome achievable without one.
The phrase "the best of all possible worlds" ought to be the canonical example of the Mind Projection Fallacy.
It would be unreasonably burdensome to append "with respect to a given mind" to every statement that involves subjectivity in any way.
ETA: For comparison, imagine if you had to say "with respect to a given reference frame" every time you talked about velocity.
I'm not saying that you didn't express yourself precisely enough. I am saying that there is no such thing as "best (full stop)" There is "best for me", there is "best for you", but there is not "best for both of us". No more than there is an objective (or intersubjective) probability that I am wearing a red shirt as I type.
Your argument above only works if "best" is interpreted as "best for every mind". If that is what you meant, then your implicit definition of FAI proves that FAI is impossible.
ETA: What given frame do you have in mind??????
The usual assumption in this context would be CEV. Are you saying you strongly expect humanity's extrapolated volition not to cohere?
Perhaps you should explain, by providing a link, what is meant by CEV. The only text I know of describing it is dated 2004, and, ... how shall I put this ..., it doesn't seem to cohere.
But, I have to say, based on what I can infer, that I see no reason to expect coherence, and the concept of "extrapolation" scares the sh.t out of me.
"Coherence" seems a bit like the human genome project. Yes there are many individual differences - but if you throw them all away, you are still left with something.
So we are going to build a giant AI to help us discover and distill that residue of humanity which is there after you discard the differences?
And here I thought that was the easy part, the part we had already figured out pretty well by ourselves.
And I'm not sure I care for the metaphor of "throwing away" the differences. Shouldn't we instead be looking for practices and mechanisms that make use of those differences, that weave them into a fabric of resilience and mutual support rather than a hodgepodge of weakness and conflict?
But would that something form a utility function that wouldn't be deeply horrifying to the vast majority of humanity?
I'm looking at the same document you are, and I actually agree that EV almost certainly ~C. I just wanted to make sure the assumption was explicit.