= 27d567bc2d48d9f40e9ccc20ea370b15)
private_messaging comments on Wasted life - Less Wrong Discussion
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (39)
The issue is not merely that you don't have ground up definitions which respect the time constraints. The issue is that you don't seem to have any ground-up definitions at all, i.e. not even for something like AIXI. The goals themselves lack any bottom up definitions.
Worst of all you build stuff from the dubious concepts like that monolithic "intelligence".
Say, we want to make better microchips. We the engineers have to build it from bottom up, so we make some partially implemented intelligence to achieve such a goal, omitting the definition of what exactly is 'best microchip', omitting the real world goals, focussing the search (heuristics are about where you search!) and instead making it design smaller logical gates and then route up the chip, and perhaps figure out manufacturing. All doable with same methods, all to the point, strongly superhuman performance on subhuman hardware.
You build Oracle AI out of that monolithic "intelligence" concept, and tell it - I want a better microchip. This monolithic intelligence figures out how to take over the world to do so. You think, how do we prevent this monolithic intelligence concept from thinking about taking over the world? That looks like an incredibly difficult problem.
Or the orthogonality thesis. You think - are the goals of that monolithic intelligence arbitrary?
Meanwhile if you try to build bottom up or at least from the concepts with known bottom up definitions, well, something like number of paperclips in the universe is clearly more difficult than f(x,n) where x is the output on the screen at the step n and f is 1 if the operator responds with the reward button, 0 otherwise (note that the state if the computer is unplugged has to be explicitly modelled, and its not trivial to build bottom-up the concept of mathematical function 'disappearing', may actually be impossible).
The reason for that is that the AIs that are worrying are those of human-like levels of ability. And humans have shown skill in becoming intelligent in many different domains and the ability to build machine intelligence in the domains we have little skill in. So whatever its design, a AGI (artificial general intelligence) will have a broad pallet of abilities, and probably the ability to acquire others - hence the details of its design are less important than meta considerations. This is not the case for non-AGI AIs.
I think "how to convince philosophers that high intelligence will not automatically imply certain goals" - ie that they are being incorrectly meta.
Moor's law is a better way of predicting the future than knowing the exact details of the current research into microprocessors. Since we don't have any idea how the first AGI will be built (assuming it can be built), why bother focusing down on the current details when we're pretty certain they won't be relevant?
The AIs that are worrying have to beat the (potentially much simpler) partial AIs, that can be of autistic savant - like levels of ability and beyond, in the fields very relevant to being actually powerful. You can't focus just on the human-level AGIs when you consider the risks. The AGIs have to be able to technologically outperform contemporary human civilization to significant extent. Which would not happen if both AGIs and humans are substantially bottlenecked on running essentially same highly optimized (possibly self optimized) non general purpose algorithms to solve domain specific problems.
The meta considerations in question look identical to the least effective branches of philosophy.
I think so too, albeit in different way: I do not think that high intelligence will automatically imply that the goals are within the class of "goals which we have no clue how to define mathematically but which are really easy to imagine".
I have trouble parsing logical structure of this argument. Since we don't have any idea how the first AGI will be built, wouldn't make reasoning that employs faulty concepts relevant. Furthermore, being certain that something is irrelevant without having studied it, is a very Dunning-Kruger prone form of thought.
Furthermore, I can't see how in the world can you be certain that it is irrelevant that (for example) the AI has to work in peer to peer topology with very substantial lag, efficiently (i.e. no needless simulation of other nodes of itself, significant ignorance of content of other nodes, the local nodes lacking sight of the global picture, etc), when it comes to how it will interact with the other intelligences. We truly do not know that the hyper-morality won't fall out of this as a technological solution, considering that our own morality was produced as the solution for cooperation.
Also, I can't see how it can be irrelevant that (a good guess) AGI is ultimately a mathematical function that calculates outputs from inputs using elementary operations, and the particular instance of the AGI is a machine that's computing this function. That's a meta consideration built from ground up rather than from concept of monolithic 'intelligence'. The 'symbol grounding' may be a logical impossibility (and the feeling that symbols are grounded may well be a delusion that works via fallacies); in any case we don't see how it can be solved. Like free will; a lot of people feel very sure that they have something in their mind, that's clearly not compatible with reductionism. Well, I think there can be a lot of other things that we feel very sure we have, which are not compatible with reductionism in less obvious ways.
edit: to summarize, my opinion is that everything is far, far, far too speculative to warrant investigation. It's like trying to prevent Hindenburg disaster, bombing of Dresden, atomic bombing of Hiroshima and Nagasaki, and the risk of nuclear war, by thinking of the flying carpet as the flying vehicle (because bird-morphizing is not cool).
Yes. The SIAI world view doesn't seem to pay much particular attention to how morality necessarily evolved as the cooperative glue necessary for the social super-organism meta-transition.
Well, my opinion is that this is far too dangerous (as compared with other risks to humanity) to not investigate it. Philosophical tools are weak, but they've yet to prove weak enough that we should shelve the ongoing project.
It seems to me that you are a: grossly over estimating the productivity of symbolic manipulation on a significant number of symbols with highly speculative meanings, and b: there is the issue that you do not seem to dedicate due effort to investigating existing software or trying to verify relevance of the symbols or improve it. The symbolic manipulation is only as relevant as the symbols being manipulated.
Contradicts:
If you don't have any idea how AGI will be built, how can you be so confident about the distribution of its goals?
Ignorance widens the space of possible outcomes, it doesn't narrow it.
i.e. it makes no sense to make arguments like "we know nothing about the mind of god, but he doesn't like gay sex"
If you are ignorant about the nature of superintelligence, then you don't know whether or not it entails certain goals.
Ignorance does not allow you to hold confidence in the proposition that "high intelligence will not automatically imply certain goals".
Adopting this argument from ignorance puts you in the unfortunate position of being like the uninformed layman attempting to convince particle physicists of the grave dangers of supercolliders destroying the earth.
For in fact there is knowledge to be had about intelligence and the nature of future AI, and recognized experts in the field (Norvig, Kurzweil, Hawkins etc) are not dismissing the SIA position out of ignorance.
Just to make sure I understand you: does it make sense to be confident that the roll of a hundred-sided die won't be 12?
EDIT: I meant to say "Yes more or less, but the closer mapping of that analogy is . .."
No, but the closer mapping of that analogy is: does it make sense to be confident that the "roll" of an unknown object will be 12 when you don't even know that it's a die?
OK, I think I understand you now. Thanks.
To answer your question as I understand it: it does not make sense for me to be confident that the result of some unspecified operation performed on some unknown object will be 12.
It does make sense to be confident that it won't be 12. (I might, of course, be confident and wrong. It's just unlikely.)
I consider the latter a more apposite analogy for the argument you challenge here. Being confident that an unspecified process (e.g., AGI) won't value paperclips makes more sense than being confident that it will, in the same way that being confident that it won't return "12" makes more sense than being confident than it will.
Perhaps we are using a different notion of 'confidence'. The uses of that term that I am familiar with have a separate specific meaning apart from probability. Confidence to me implies meta-level knowledge about the potential error in one's probability estimate, intervals, or something of that nature.
So in your analogy, I can't be confident about any properties of an unspecified process. I can of course assign a low probability estimate to the proposition that this unspecified process won't value paperclips purely from priors, but that will not be a high confidence estimate. The 'process' could be a paperclip factory for all I know.
If you then map this analogy back to the original SIAI frame, I assume that the 'die' maps loosely to AGI development, and the 12 is AGI's values being human values. And then no, it does not make sense to be confident that the roll won't be 12, given that we supposedly don't know what kind of die it is. It very well could be a 'die' with only 12s.
Priors are really just evidence you have already accumulated, so in reality one is never in a state of complete ignorance.
For example, I know that AGI will be created by humans (high confidence), humans create things which they value or things which help fulfill their values, and that humans for reasons both anthropocentric and economic are more likely to thus create AGI that shares human values.
I don't think it's useful to talk about whether we can have confidence in statements about the outcome of an AGI process while we still disagree about whether we can have confidence in statements about the outcome of rolling a hundred-sided die.
So, OK.
Given two statements, P1 ("my next roll of this hundred-sided die will not be 12") and P2 ("my next roll of this hundred-sided die will be 12"), I consider it sensible to be confident of P1 but not P2, you don't consider it sensible to be confident of either statement. This may be because of different uses of the term "confident", or it might be something more substantive.
Would you agree that there's a 99% chance of P1 being true, and a 99% chance of P2 being false, given a fair die toss?
If so, can you say more about the class of statements like P1, where I estimate a 99% chance of it being true but it's inappropriate for me to be confident in it?