= 27d567bc2d48d9f40e9ccc20ea370b15)
Zack_M_Davis comments on David Chalmers' "The Singularity: A Philosophical Analysis" - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (202)
You're responding to an interpretation of what I said that assumes I'm stupid, not the thing I was actually trying to say. Do you seriously think I've spent a year at SIAI without understanding such basic arguments? I'm not retarded. I just don't have the energy to think through all the ways that people could interpret what I'm saying as something dumb because it pattern matches to things dumb people say. I'm going to start disclaiming this at the top of every comment, as suggested by Steve Rayhawk.
Specifically, in this case, in the comment you replied to and elsewhere in this thread, I said: "this doesn't apply to AIs that are bad at that kind of philosophical reflection". I'm making a claim that all well-designed AIs will converge to universal 'morality' that we'd like upon reflection even if it wasn't explicitly coded to approximate human values. I'm not saying your average AI programmer can make an AI that does this, though I am suggesting it is plausible.
This is stupid. I'm suggesting a hypothesis with low probability that is contrary to standard opinion. If you want to dismiss it via absurdity heuristic go ahead, but that doesn't mean that there aren't other people who might actually think about what I might mean while assuming that I've actually thought about the things I'm trying to say. This same annoying thing happened with Jef Allbright, who had interesting things to say but no one had the ontology to understand him so they just assumed he was speaking nonsense. Including Eliezer. LW inherited Eliezer's weakness in this regard, though admittedly the strength of narrowness and precision was probably bolstered in its absence.
If what I am saying sounds mysterious, that is a fact about your unwillingness to be charitable as much as it is about my unwillingness to be precise. (And if you disagree with that, see it as an example.) That we are both apparently unwilling doesn't mean that either of us is stupid. It just means that we are not each others' intended audience.
(Downvoted.)
No one said you were stupid.
People are responding to the text of your comments as written. If you write something that seems to ignore a standard argument, then it's not surprising that people will point out the standard argument.
As a parable, imagine an engineer proposing a design for a perpetual motion device. An onlooker objects: "But what about conservation of energy?" The engineer says: "Do you seriously think I spent four years at University without understanding such basic arguments?" An uncharitable onlooker might say "Yes." A better answer, I think, is: "Your personal credentials are not at issue, but the objection to your design remains."
(Upvoted.)
I suppose I mostly meant 'irrational', not stupid. I just expected people to expect me to understand basic SIAI arguments like "value is fragile" and "there's no moral equivalent of a ghost in the machine" et cetera. If I didn't understand these arguments after having spent so much time looking at them... I may not be stupid, but there'd definitely be some kind of gross cognitive impairment going on in software if not in hardware.
There were a few cues where I acknowledged that I agreed with the standard argument (AGI won't automatically converge to Eliezer's "good"), but was interested in a different argument about philosophically-sound AIs that didn't necessarily even look at humanity as a source of value but still managed to converge to Eliezer's good, because extrapolated volitions for all evolved agents cohere. (I realize that your intuition is interestingly perhaps somewhat opposite mine here, in that you fear more than I do that there won't be much coherence even among human values. I think that we might just be looking at different stages of extrapolation... if human near mode provincial hyperbolic discounting algorithms make deals with human far mode universal exponential discounting algorithms, the universal (pro-coherence) algorithms will win out in the end (by taking advantage of near mode's hyperbolic discounting). If this idea is too vague or you're interested I could expand on this elsewhere.)
Your parable makes sense, it's just that I don't think I was proposing a perpetual motion device, just something that could sound like a perpetual motion device if I'm not clear enough in my exposition, which it looks like I wasn't. I was just afraid of italicizing and bolding the disclaimers because I thought it'd appear obnoxious, but it's probably less obnoxious than failing to emphasize really important parts of what I'm saying.
What does time discounting have to do with coherence? Of course exponential discounting is "universal" in the sense that if you're going to time-discount at all (and I don't think we should), you need to use an exponential in order to avoid preference reversals. But this doesn't tell us anything about what exponential discounters are optimizing for.
I think your comments would be better received if you just directly talked about your ideas and reasoning, rather than first mentioning your shocking conclusions ("theism might be correct," "volitions of evolved agents cohere") while disclaiming that it's not how it looks. If you make a good argument that just so happens to result in a shocking conclusion, then great, but make sure the focus is on the reasons rather than the conclusion.
vs.
It really really seems like these two statements contradict each other; I think this is the source of the confusion. Can you go into more detail about the second statement?
In particular, why would two agents which both evolved but under two different fitness functions be expected to have the same volition?