= 27d567bc2d48d9f40e9ccc20ea370b15)
TheAncientGeek comments on Debunking Fallacies in the Theory of AI Motivation - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (343)
Alright, I'll take you up on it:
Failure Mode I: The AI doesn't do anything useful, because there's no way of satisfying every contextual constraint.
Predicting your response: "That's not what I meant."
Failure Mode II: The AI weighs contextual constraints incorrectly and sterilizes all humans to satisfy the sort of person who believes in Voluntary Human Extinction.
Predicting your response: "It would (somehow) figure out the correct weighting for all the contextual constraints."
Failure Mode III: The AI weighs contextual constraints correctly (for a given value of "correctly") and sterilizes everybody of below-average intelligence or any genetic abnormalities that could impose costs on offspring, and in the process, sterilizes all humans.
Predicting your response: "It wouldn't do something so dumb."
Failure Mode IV: The AI weighs contextual constraints correctly and puts all people of minority ethical positions into mind-rewriting machines so that there's no disagreement anymore.
Predicting your response: "It wouldn't do something so dumb."
We could keep going, but the issue is that so far, you've defined -any- failure mode as "dumb"ness, and have argued that the AI wouldn't do anything so "dumb", because you've already defined that it is superintelligent.
I don't think you know what intelligence -is-. Intelligence does not confer immunity to "dumb" behaviors.
It's got to confer some degree of dumbness avoidance.
In any case, MIRI has already conceded that superintelligent AIs won't misbehave through stupidity. They maintain the problem is motivation ... the Genie KNOWS but doesn't CARE.
Does it? On what grounds?
That's putting an alien intelligence in human terms; the very phrasing inappropriately anthropomorphizes the genie.
We probably won't go anywhere without an example.
Market economics ("capitalism") is an intelligence system which is very similar to the intelligence system Richard is proposing. Very, very similar; it's composed entirely of independent nodes (seven billion of them) which each provide their own set of constraints, and promote or demote information as it passes through them based on those constraints. It's an alien intelligence which follows Richard's model which we are very familiar with. Does the market "know" anything? Does it even make sense to suggest that market economics -could- care?
Does the market always arrive at the correct conclusions? Does it even consistently avoid stupid conclusions?
How difficult is it to program the market to behave in specific ways?
Is the market "friendly"?
Does it make sense to say that the market is "stupid"? Does the concept "stupid" -mean- anything when talking about the market?
On the grounds of the opposite meanings of dumbness and intelligence.
Take it up with the author,
Economic systems affect us because wrong are part of them. How is an some neither-intelligent-nor-stupid-system in a box supposed to effect us?
And if AIs are neither-intelligent-nor-stupid, why are they called AIs?
And if AIs are alien, why are they able to do comprehensible and useful thing like winning jeopardy and guiding us to our destinations.
Dumbness isn't merely the opposite of intelligence.
I don't need to.
Not really relevant to the discussion at hand.
Every AI we've created so far has resulted in the definition of "AI" being changed to not include what we just created. So I guess the answer is a combination of optimism and the word "AI" having poor descriptive power.
What makes you think an alien intelligence should be useless?
What makes you think that a thing designed by humans to be useful to humans, which is useful to humans would be alien?
Because "human" is a tiny piece of a potential mindspace whose dimensions we mostly haven't even identified yet.
That's about a quarter of an argument. You need to show that AI research is some kind of random shot into mind space, and not anthropomorphically biased for the reasons given.
The relevant part of the argument is this: "whose dimensions we mostly haven't even identified yet."
If we created an AI mind which was 100% human, as far as we've yet defined the human mind, we have absolutely no idea how human that AI mind would actually behave. The unknown unknowns dominate.
Alien isnt the most transparent term to use fir human unknowns.