= 27d567bc2d48d9f40e9ccc20ea370b15)
Will_Newsome comments on Our Phyg Is Not Exclusive Enough - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (513)
From my favorite post and comments section on Less Wrong thus far:
Yes, it looks like Eliezer is mistaken there (or speaking hyperbole).
I agree with:
... but would weaken the claim drastically to "Take metaethics, a clearly reducible problem with many technical details to be ironed out". I suspect you would disagree with even that, given that you advocate meta-ethical sentiments that I would negatively label "Deeply Mysterious". This places me approximately equidistant from your respective positions.
I only weakly advocate certain (not formally justified) ideas about meta-ethics, and remain deeply confused about certain meta-ethical questions that I wouldn't characterize as mere technical details. One simple example: Eliezer equates reflective consistency (a la CEV) with alignment with the big blob of computation he calls "right"; I still don't know what argument, technical or non-technical, could justify such an intuition, and I don't know how Eliezer would make tradeoffs if the two did in fact have different referents. This strikes me as a significant problem in itself, and there are many more problems like it.
(Mildly inebriated, apologies for errors.)
Are you sure Eliezer does equate reflective consistency with alignment with what-he-calls-"right"? Because my recollection is that he doesn't claim either (1) that a reflectively consistent alien mind need have values at all like what he calls right, or (2) that any individual human being, if made reflectively consistent, would necessarily end up with values much like what he calls right.
(Unless I'm awfully confused, denial of (1) is an important element in his thinking.)
I think he is defining "right" to mean something along the lines of "in line with the CEV of present-day humanity". Maybe that's a sensible way to use the word, maybe not (for what it's worth, I incline towards "not") but it isn't the same thing as identifying "right" with "reflectively consistent", and it doesn't lead to a risk of confusion if the two turn out to have different referents (because they can't).
He most certainly does not.
Relevant quote from Morality as Fixed Computation:
Thanks - I hope you're providing that as evidence for my point.
Sort of. It certainly means he doesn't define morality as extrapolated volition. (But maybe "equate" meant something looser than that?)
Aghhhh this is so confusing. Now I'm left thinking both you and Wei Dai have furnished quotes supporting my position, User:thomblake has interpreted your quote as supporting his position, and neither User:thomblake nor User:gjm have replied to Wei Dai's quote so I don't know if they'd interpret it as evidence of their position too! I guess I'll just assume I'm wrong in the meantime.
Now two people have said the exact opposite things both of which disagree with me. :( Now I don't know how to update. I plan on re-reading the relevant stuff anyway.
If you mean me and thomblake, I don't see how we're saying exact opposite things, or even slightly opposite things. We do both disagree with you, though.
I guess I can interpret User:thomblake two ways, but apparently my preferred way isn't correct. Let me rephrase what you said from memory. It was like, "right is defined as the output of something like CEV, but that doesn't mean that individuals won't upon reflection differ substantially". User:thomblake seemed to be saying "Eliezer doesn't try to equate those two or define one as the other", not "Eliezer defines right as CEV, he doesn't equate it with CEV". But you think User:thomblake intended the latter? Also, have I fairly characterized your position?
I definitely meant the latter, and I might be persuaded of the former.
Though "define" still seems like the wrong word. More like, " 'right' is defined as *point at big blob of poetry*, and I expect it will be correctly found via the process of CEV." - but that's still off-the-cuff.
Thanks much; I'll keep your opinion in mind while re-reading the meta-ethics sequence/CEV/CFAI. I might be being unduly uncharitable to Eliezer as a reaction to noticing that I was unduly (objectively-unjustifiably) trusting him. (This would have been a year or two ago.) (I notice that many people seem to unjustifiably disparage Eliezer's ideas, but then again I notice that many people seem to unjustifiably anti-disparage (praise, re-confirm, spread) Eliezer's ideas;—so I might be biased.)
(Really freaking drunk, apologies for errors, e.g. poltiically unmotivated adulation/anti-adulation, or excessive self-divulgation. (E.g., I suspect "divulgation" isn't a word.))
Not to worry, it means "The act of divulging" or else "public awareness of science" (oddly).
I don't know whether thomblake intended the latter, but he certainly didn't say the former. I think you said "Eliezer said A and B", thomblake said "No he didn't", and you are now saying he meant "Eliezer said neither A nor B". I suggest that he said, or at least implied, something rather like A, and would fiercely repudiate B.
If I understand you correctly then this particular example I don't think I have a problem with, at least not when I assume the kind of disclaimers and limitations of scope that I would include if I were to attempt to formally specify such a thing.
I suspect I agree with some of your objections to various degrees.
Part of my concern about Eliezer trying to build FAI also stems from his treatment of metaethics. Here's a caricature of how his solution looks to me:
Alice: Hey, what is the value of X?
Bob: Hmm, I don't know. Actually I'm not even sure what it means to answer that question. What's the definition of X?
Alice: I don't know how to define it either.
Bob: Ok... I don't know how to answer your question, but what if we simulate a bunch of really smart people and ask them what the value of X is?
Alice: Great idea! But what about the definition of X? I feel like we ought to be able to at least answer that now...
Bob: Oh that's easy. Let's just define it as the output of that computation I just mentioned.
I thought the upshot of Eliezer's metaethics sequence was just that "right" is a fixed abstract computation, not that it's (the output of) some particular computation that involves simulating really smart people. CEV is not even mentioned in the sequence (EDIT: whoops it is.).
(Indeed just saying that it's a fixed abstract computation is at the right level of abstraction to qualify as metaethics; saying that it's some particular computation would be more like just plain ethics. The upshot does feel kind of underwhelming and obvious. This might be because I just don't remember how confusing the issue looked before I read those posts. It could also mean that Eliezer claiming that metaethics is a solved problem is not as questionable as it might seem. And it could also mean that metaethics being solved doesn't consitute as massive progress as it might seem.)
BTW, I've had numerous "wow" moments with philosophical insights, some of which made me spend years considering their implications. For example:
I expect that a correct solution to metaethics would produce a similar "wow" reaction. That is, it would be obvious in retrospect, but in an overwhelming instead of underwhelming way.
Is the insight about free will and logical facts part of the sequences? or is it something you or others discuss in a post somewhere? I'd like to learn about it, but my searches failed.
I never wrote a post on it specifically, but it's sort of implicit in my UDT post (see also this comment). Eliezer also has a free will sequence which is somewhat similar/related but I'm not sure if he would agree with my formulation.
What is "you"? And what is "deciding"? Personally I haven't been able to come to any redefinition of free will that makes more sense than this one.
I haven't read the free will sequence. And I haven't read up on decision theory because I wasn't sure if my math education is good enough yet. But I doubt that if I was going to read it I would learn that you can salvage the notion of "deciding" from causality and logical facts. The best you can do is look at an agent and treat it is as a transformation. But then you'd still be left with the problem of identity.
(Agreed; I also think meta-ethics and ethics are tied into each other in a way that would require that a solution to meta-ethics would at least theoretically solve any ethical problems. Given that I can think of hundreds or thousands of object level ethical problems, and given that I don't think my inability to answer at least some of them is purely due to boundedness, fallibility, self-delusion, or ignorance as such, I don't think I have a solution to meta-ethics. (But I would characterize my belief in God as at least a belief that meta-ethics and ethical problems do at least have some unique (meta-level) solution. This might be optimistic bias, though.))
Wei Dai, have you read the Sermon on the Mount, particularly with superintelligences, Tegmark, (epistemic or moral) credit assignment, and decision theory in mind? If not I suggest it, if only for spiritual benefits. (I suggest the Douay-Rheims translation, but that might be due to a bias towards Catholics as opposed to Protestants.)
(Pretty damn drunk for the third day in a row, apologies for errors.)
Are you planning on starting a rationalist's drinking club? A byob lesswrong meetup with one sober note-taker? You usually do things purposefully, even if they're unusual purposes, so consistent drunkenness seems uncharacteristic unless it's part of a plan.
Will_Newsome isn't a rationalist. (He has described himself as a 'post-rationalist', which seems as good a term as any.)
In vino veritas et sanitas!
It's mentioned here:
ETA: Just in case you're right and Eliezer somehow meant for that paragraph not to be part of his metaethics, and that his actual metaethics is just "morality is a fixed abstract computation", then I'd ask, "If morality is a fixed abstract computation, then it seems that rationality must also be a fixed abstract computation. But don't you think a complete "solved" metaethics should explain how morality differs from rationality?"
Rationality computation outputs statements about the world, morality evaluates them. Rationality is universal and objective, so it is unique as an abstract computation, not just fixed. Morality is arbitrary.
How so? Every argument I've heard for why morality is arbitrary applies just as well to rationality.
If we assume some kind of mathematical realism (which seems to be necessary for "abstract computation" and "uniqueness" to have any meaning) then there exist objectively true statements and computations that generate them. At some point there are Goedelian problems, but at least all of the computations agree on the primitive-recursive truths, which are therefore universal, objective, unique, and true.
Any rational agent (optimization process) in any world with some regularities would exploit these regularities, which means use math. A reflective self-optimizing rational agent would arrive to the same math as us, because the math is unique.
Of course, all these points are made by a fallible human brain and so may be wrong.
But there is nothing even like that for morality. In fact, when a moral statement seems universal under sufficient reflection, it stops being a moral statement and becomes simply rational, like cooperating in the Prisoner's Dilemma when playing against the right opponents.
Yeah: CEV appears to just move the hard bit. Adding another layer of indirection.
To take Eliezer's statement one meta-level down:
What did he mean by "I tried that..."?
I'm not at all sure, but I think he means CFAI.
Possibly he means this.
He may have soleved it, but if only he or someone else could say what the solution was.