I think that 'awesome' loses a lot of value when you are forced to make the statement "Watching lot of people die was the most awesome choice I had, because any intervention would have added victims without saving anyone."
I propose 'lame' and 'bummer' as antonyms for 'awesome'. Instead of trying to figure out the most awesome of a series of bad options, we can discuss the least lame.
Sucks less sucks less.
Right, SI's basic idea is correct.
However, given that WBE's will in any case be developed (and we can mention IA as well) , I'd like to see more consideration of how to keep brain-based AI's as safe as possible before they enter their Intelligence Explosion -- even though we understand that after an Explosion, there is little you can do.
One trouble is that that is essentailly tacking mind enslavement on to the WBE proposition. Nobody wants that. Uploads wouldn't volunteer. Even if a customer paid enough of a premium for an employee with loyalty modifications, that only rolls us back to relying on the good intent of the customer.
This comes down to the exact same arms race between friendly and 'just do it' . With extra ethical and reverse-engineering hurdles. (I think we're pretty much stuck with testing and filtering based on behavior. And some modification will only be testable after uploading is available)
Mind you I'm not saying don't do work on this, I'm saying not much work will be done on it.
I think we're going to get WBE's before AGI.
If we viewed this as a form of heuristic AI, it follows from your argument that we should look for ways to ensure friendliness of WBE's. (Ignoring the ethical issues here.)
Now, maye this is becasue most real approaches would consider ethical issues, but it seems like figuring out how to modify a human brain so that it doesn't act against your interests even if is powerful and without hampering its intellect, is a big 'intractable' problem.
I suspect no one is working on it and no one is going to, even though we have working models of these intellects today. A new design might be easier to work with, but it will still be a lot harder than it wil seem to be worth - as long as the AI's are doing near human level work.
Aim for an AI design that's easy enough to work on saftey that people actually will work on safety... and it will start to look a lot like SIAI ideas.
Just because Harry's learned to keep his claws sheathed doesn't mean he's not still engaging in dominance contests.
Moody set it as a condition for being able to speak as an equal.
Signs of leglimency can be detected, so if Quirrell is leglimizing Harry, he may be risking getting caught.
The fact that Harry has an occlumency teacher who uses leglimency on him in an instructional setting might serve to disguise other people using leglimency on him though. We don't know what kind of evidence checking for leglimency can provide about recency and whether the mind has been accessed by different people.
However, I doubt that Quirrell has been using leglimency on Harry, because there's a point where you're really just giving the antagonist too great an advantage, and if Quirrell has been accessing the contents of Harry's mind undetected, I would have an incredibly hard time buying the protagonists not being completely fucked (tvtropes link.)
There is some time resolution.
Albus said heavily, "A person who looked like Madam McJorgenson told us that a single Legilimens had lightly touched Miss Granger's mind some months ago. That is from January, Harry, when I communicated with Miss Granger about the matter of a certain Dementor. That was expected; but what I did not expect was the rest of what Sophie found."
Essentially this is about cutting your enemy, and some procedural pitfalls in his examples.
I didn't spot any novel principles in the talk, but I want to learn more about how he thinks.
how likely an AGI project is to doom us
Isn't it more like "how likely a formally proven FAI design is to doom us", since this is what Holden seems to be arguing (see his quote below)?
Suppose that it is successful in the "AGI" part of its goal, i.e., it has successfully created an intelligence vastly superior to human intelligence and extraordinarily powerful from our perspective. Suppose that it has also done its best on the "Friendly" part of the goal: it has developed a formal argument for why its AGI's utility function will be Friendly, it believes this argument to be airtight, and it has had this argument checked over by 100 of the world's most intelligent and relevantly experienced people. .. What will be the outcome?
"When a distinguished but elderly scientist states that something is possible, he is almost certainly right. When he states that something is impossible, he is very probably wrong."
Is a human equipped with Google an optimization process powerful beyond the human's ability to steer?
Tell me from China.
AGI will only be Friendly if its goals are the kinds of goals that we would want it to have
At the risk of losing my precious karma, I'll play the devil's advocate and say I disagree.
First some definitions: "Friendly" (AI), according to Wikipedia, is one that is beneficial to humanity (not a human buddy or pet). "General" in AGI means not problem-specific (narrow AI).
My counterexample is an AI system that lacks any motivations, goals or actuators. Think of an AIXI system (or, realistically, a system that approximates it), and subtract any reward mechanisms. It just models its world (looking for short programs that describe its input). You could use it to make (super-intelligent) predictions about the future. This seems clearly beneficial to humanity (until it falls into malicious human hands, but that's besides the argument you are making)
That would make (human[s] + predictor) in to an optimization process that was powerful beyond the human[s]'s ability to steer. You might see a nice looking prediction, but you won't understand the value of the details, or the value of the means used to achieve it. (Which would be called trade-offs in a goal directed mind, but nothing weighs them here.)
It also won't be reliable to look for models in which you are predicted to not hit the Emergency Regret Button As that may just find models in which your regret evaluator is modified.
View more: Next
= f037147d6e6c911a85753b9abdedda8d){kind=logo}
= 27d567bc2d48d9f40e9ccc20ea370b15)
= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Write a bullet-point summary for each sequence and tell me that one would not be tempted to "dismiss them out of hand, even lacking a chain of arguments leading up to them", unless one is already familiar with the arguments.
Anecdote: I think I've had better responses summarizing LW articles in a few paragraphs without linking, than linking to them with short explanations.
It does take a lot to crosss those inferential distances, but I don't think quite that much.
To be fair, my discussions may not cover a whole sequence, I have the opportunity to pick out what is needed in a particular instance.