= 27d567bc2d48d9f40e9ccc20ea370b15)
OrphanWilde comments on Debunking Fallacies in the Theory of AI Motivation - LessWrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (343)
Well, the problem here is a misunderstanding of my claim.
(If I really were claiming the things you describe in your above comment, your points would be reasonable. But there is such a strong misunderstanding the your points are hitting a target that, alas, is not there.)
There are several things that I could address, but I will only have time to focus on one. You say:
No. A hundred times no :-). My claim is not even slightly that "a logically consistent AI won't do anything bad because bad is logically inconsistent".
The claim is this:
1) The entire class of bad things that these hypothetical AIs are supposed to be doing are a result of the AI systematically (and massively) ignoring contextual information.
(Aside: I am not addressing any particular bad things, on a case-by-case basis, I am dealing with the entire class. As a result, my argument is not vulnerable to charges that I might not be smart enough to guess some really-really-REALLY subtle cases that might come up in the future.)
2) The people who propose these hypothetical AIs have made it absolutely clear that (a) the AI is supposed to be fully cognizant of the fact that the contextual information exists (so the AI is not just plain ignorant), but at the same time (b) the AI does not or cannot take that context into account, but instead executes the plan and does the bad thing.
3) My contribution to this whole debate is to point out that the DESIGN of the AI is incoherent, because the AI is supposed to be able to hold two logically inconsistent ideas (implicit belief in its infallibility and knowledge of its fallibility).
If you look carefully at that argument you will see that it does not make the claim that
I never said that. The logical inconsistency was not in the 'bad things' part of the argument. Completely unrelated.
Your other comments are equally as confused.
Not acting upon contextual information isn't the same as ignoring it.
The AI knows, for example, that certain people believe that plants are morally relevant entities - is it possible for it to pick strawberries at all? What contextual information is relevant, and what contextual information is irrelevant? You accuse the "infallible" AI of ignoring contextual information - but you're ignoring the magical leap of inference you're taking when you elevate the concerns of the chef over the concerns of the bioethicist who thinks we shouldn't rip reproductive organs off plants in the first place.
The issue is that fallibility doesn't -imply- anything. I think this is the best course of action. I'm fallible. I still think this is the best course of action. The fallibility is an unnecessary and pointless step - it doesn't change my behavior. Either the AI depends upon somebody else, who is treated as an infallible agent - or it doesn't.
Then we're in agreement that insane-from-an-outside-perspective behaviors don't require logical inconsistency?
Sorry, I cannot put any more effort into this. Your comments show no sign of responding to the points actually made (either in the paper itself, or in my attempts to clarify by responding to you).
Maybe, given the number of times you feel you've had to repeat yourself, you're not making yourself as clear as you think you are.
I find that when I talk about this issue with people who clearly have expert knowledge of AI (including the people who came to the AAAI symposium at Stanford last year, and all of the other practising AI builders who are my colleagues), the points I make are not only understood but understood so clearly that they tell me things like "This is just obvious, really, so all you are doing is wasting your time trying to convince a community that is essentially comprised of amateurs" (That is a direct quote from someone at the symposium).
I always want to make myself as clear as I can. I have invested a lot of my time trying to address the concerns of many people who responded to the paper. I am absolutely sure I could do better.
We're all amateurs in the field of AI, it's just that some of us actually know it. Seriously, don't pull the credentials card. I'm not impressed. I know exactly how "hard" it is to pay the AAAI a hundred and fifty dollars a year for membership, and three hundred dollars to attend their conference. Does claiming to have spent four hundred and fifty dollars make you an expert? What about bringing up that it's in "Stanford"? What about insulting everybody you're arguing with?
I'm a "practicing AI builder" - what a nonsense term - although my little heuristics engine is actually running in the real world, processing business data and automating hypothesis elevation work for humans (who have the choice of agreeing with its best hypothesis, selecting among its other hypotheses, or entering their own) - that is, it's actually picking strawberries.
Moving past tit-for-tat on your hostile introduction paragraph, I don't doubt your desire to be clear. But you have a conclusion you're very obviously trying to reach, and you leave huge gaps on your way to get there. The fact that others who want to reach the same conclusion overlook the gaps doesn't demonstrate anything. And what's your conclusion? That we don't have to worry about poorly-designed AI being dangerous, because... contextual information, or something. Honestly, I'm not even sure anymore.
Then you propose a model, which you suggest has been modeled after the single most dangerous brain on the planet - as proof that it's safe! Seriously.
As for whether you could do better? No, not in your current state of mind. Your hubris prevents you from doing better. You're convinced you know better than any of the people you're talking with, and they're ignorant amateurs.
When someone repeatedly distorts and misrepresents what is said in a paper, then blames the author of the paper for being unclear ... then hears the author carefully explain the distortions and misrepresentations, and still repeats them without understanding ....
Well, there is a limit.
Not to suggest that you are implying it, but rather as a reminder - nobody is deliberately misunderstanding you here.
But at any rate, I don't think we're accomplishing anything here except driving your karma score lower, so by your leave, I'm tapping out.
Why not raise his karma score instead?
Because that was the practical result, not the problem itself, which is that the conversation wasn't going anywhere, and he didn't seem interested in it going anywhere.