I could add: Objective punishments and rewards need objective justification.
From my perspective, treating rationality as always instrumental, and never a terminal value is playing around with it's traditional meaning. (And indiscriminately teaching instrumental rationality is like indiscriminately handing out weapons. The traditional idea, going back to st least Plato, is that teaching someone to be rational improves them...changes their values)
I am aware that humans hav a non zero level of life threatening behaviour. If we wanted it to be lower, we could make it lower, at the expense of various costs. We don't which seems to mean we are happy with the current cost benefit ratio. Arguing, as you have, that the risk of AI self harm can't be reduced to zero doesn't mean we can't hit an actuarial optimum.
It is not clear to me why you think safety training would limit intelligence.
Regarding the anvil problem: you have argued with great thoroughness that one can't perfectly prevent an AIXI from dropping an anvil on its head. However, I can't see the necessity. We would need to get the probability of a dangerously unfriendly SAI as close to zero as possible, because it poses an existential threat. However, a suicidally foolish AIXI is only a waste of money.
Humans have a negative reinforcement channel relating to bodily harm called pain. It isn't perfect, but it's good enough to train most humans to avoid doing suicidal stupid things...
An entity that has contradictory beliefs will be a poor instrumental rationalist. It looks like you would need to engineer a distinction between instrumental beliefs and terminal beliefs. While we're on the subject, you might need a firewall to stop an .AI acting on intrinsically motivating ideas, if they exist. In any case, orthogonality is an architecture choice, not an ineluctable fact about minds.
The OT has multiple forms, as Armstrong notes. An OT that says you could make arbitrary combinations of preference and power if you really wanted to, can't p...
something previously deemed "impossible"
It's clearly possible for some values of "gatekeeper", since some people fall for 419 scams. The test is a bit meaningless without information about the gatekeepers
The problem is that I don't see much evidence that Mr. Loosemore is correct. I can quite easily conceive of a superhuman intelligence that was built with the specification of "human pleasure = brain dopamine levels", not least of all because there are people who'd want to be wireheads and there's a massive amount of physiological research showing human pleasure to be caused by dopamine levels.
I don't think Loosemore was addressing deliberately unfriendly AI, and for that matter EY hasn't been either. Both are addressing intentionally friendly...
Trying to think this out in terms of levels of smartness alone is very unlikely to be helpful.
Then solve semantics in a seed.
My answer: who knows? We've given it a deliberately vague goal statement (even more vague than the last one), we've given it lots of admittedly contradictory literature, and we've given it plenty of time to self-modify before giving it the goal of self-modifying to be Friendly.
Humans generally manage with those constraints. You seem to be doing something that is kind of the opposite of anthropomorphising -- treatiing an entity that is stipulated as having at least human intelligence as if were as literal and rigid as a non-AI computer.
Semantcs isn't optional. Nothing could qualify as an AGI,let alone a super one, unless it could hack natural language. So Loosemore architectures don't make anything harder, since semantics has to be solved anyway.
"code in the high-level sentence, and let the AI figure it out."
So it's impossible to directly or indirectly code in the compex thing called semantics, but possible to directly or indirectly code in the compex thing called morality? What? What is your point? You keep talking as if I am suggesting there is someting that can be had for free, without coding. I never even remotely said that.
...If the AI is too dumb to understand 'make us happy', then why should we expect it to be smart enough to u
Goertzel appears to be a respected figuer in the field. Could you point the interested reader to your critique of his work?
Now, why exactly should we expect the superintelligence that grows out of the seed to value what we really mean by 'pleasure', when all we programmed it to do was X, our probably-failed attempt at summarizing our values?
Maybe we didn't do it ithat way. Maybe we did it Loosemore's way, where you code in the high-level sentence, and let the AI figure it out. Maybe that would avoid the problem. Maybe Loosemore has solved FAi much more straightforwardly than EY.
Maybe we told it to. Maybe we gave it the low-level expansion of "happy" that we or
He's achieved about what Ayn Rand achieved, and almost everyone thinks she wasa crank.
And to be fair, you'd have to give ten or a hundred votes to people with PhD's in political science.
It is almost always bad Bayes, or any other kind of reasoning, to make judgements about individuals based on group characeristics, since there is almost always information about them as individuals available which is almost always more reliable.
As evidence for this, he used the fact that right here in America, test scores have gone up over the past several decades. This clearly isn't caused by some genetic change, so the most likely explanation is cultural change.
Is that actually more likely than environmental change?
ith permission from both their spouses, which counts for a lot in my view. If you want to turn that into a "problem", you have to specify that the spouses were unhappy—and then it's still not a matter for outsiders.
I dare say many a guru or cult leader has similar "permission". It often isn't taken to ecuse their actions, because people recognise that such permission can be browbeaten ot of people by someone who seems to them to be an authority figure.
Rand herself didn't understand emergence (she casting a biologist as the embodiment of scientific corruption, because there is too much complexity in his area of study for any one human brain to be familiar with), and also didn't understand much about cybernetics, etc.
That's hardly the start of it. She opposed relativity and QM, and fence-sat on Evolution.
ETA:
I don't think "1957" is mcuh of an excuse either, particularly about evolution. For another thing, she never wavered till her death in the 80s. It makes no sense to focus on Bayes, unle...
Suggestion: teach rationality as an open spirit of enquiry, not as a secular rleigion that will turn you into a clone of Richard Dawkins.
There are already precise terms for most of the concepts LW discusses.
State three examples?
You want to teach philosophy as rationality?
Philosophy includes epistemology, which is kind of important to epistemic ratioanlity.
Philosophy is a toolbox as well as a set of doctrines.
Or maybe there's a lot of utility in not coming accross geeky and selfish, so they are already being instementally rational.
Einstein backed local realism and the ensemble interpretation, both of which have been "thrown out".
It may be correct, but not for the reasons usually given: the non existence of ontological randomness is in now way entailed buy the existence of epistemic inderminism.
As a former boss of mine used to say: "Bloody five o-clocker"
. At first, complementary effects dominate, and human wages rise with computer productivity. But eventually substitution can dominate, making wages fall as fast as computer prices now do."
But presumably, productivity would rise as well, increasing the real value of wages of a certain face value.
It's plausible we'll never see a city with a high-speed all-robotic all-electric car fleet because the government, after lobbying from various industries, will require human attendants on every car - for safety reasons, of course!
I believe I have alredy pointed out that automatic trains already exist. Putting a human superintendent onto a train with nothing to do except watch it drive itslef would be quite ineffectice, because the job is so boring they are unlikely to concetrate. I believe exisitng driverless trains are monitored by CCTV, which is more effective since the monotirs actually ahve something to do in flicking between channels, and could be appied to driverless cars.
...The US educational system is either getting worse at training people to handle new jobs, or getting so much more expensive that people can't afford retraining, for various other reasons. (Plus, we are really stunningly stupid about matching educational supply to labor demand. How completely ridiculous is it to ask high school students to decide what they want to do with the rest of their lives and give them nearly no support in doing so? Support like, say, spending a day apiece watching twenty different jobs and then another week at their top three cho
the question of how many worlds can be answered pretty well by ~2 to the power of the average number of decoherence events since the beginning.
Deocherence evernts aren't well defined .. they are always FAPP. That;s the source of the problem.
he Copenhagen interpretation suggests there's a priveliged branch in some way, which is the one we actually perceive. Why should there be? This priveliged branch idea is adding something that we don't need to add.
MWI adds a privileged basis that is also unnecessary.
Many worlds is pretty much that view.
MWI adds a universal quantum state that is not , and cannot be, observed.
A very quick but sufficient refutation is that the same math taken as a description of an objectively existing causal process gives us MWI, hence there is no reason to complicate our epistemology beyond this
Or MWI could be said to be complicating the ontology unnecessarily. To be sure, rQM answers epistemologically some questions that MWI answers ontologically, but that isn't obviously a Bad Thing. A realistitc interpretation of the WF is a postive metaphyscial assumption, not some neutral default. A realistic quantum state of the universe is a further assumption that buys problems other interpretations don't have.
The arguments don't apply to interpretations that don't require a real WF or real collapse, and EY has struggled with them,.
...The basic wrong assumption being made is that quantum superposition by default equals multiplicity - that because the wavefunction in the double-slit experiment has two branches, one for each slit, there must be two of something there - and that a single-world interpretation has to add an extra postulate to this picture, such as a collapse process which removes one branch. But superposition-as-multiplicity really is just another hypothesis. When you use ordinary probabilities, you are not rationally obligated to believe that every outcome exists somewhere
on a large scale
Which is to say, MWI is what you get if you assume there is a universal state without an observer to observe the state of fix the basis. As it happens, it is possible to reject Universal State AND real collapse.
But we don't know that qualia aren;t anything and we don't know that about free will either.
Yep. A morality that leads to the conclusion that we should all work like slaves for a future paradise, the slightest lapse incurring a cost equivalent to untold numbers of dead babies, and the enormity of the task meaning that we shall never experience it ourselves, is prima facie a broken morality.
It is counterintuitive that you should slave for people you don't know, perhaps because you can't be sure you are serving their needs effectively. Even if that objection is removed by bringing in an omniscient oracle,there still seems to be a problem because the prospect of one generation slaving to create paradise for another isn't fair. the simple version of utilitiarianism being addressed here only sums individual utilities, and us blind to things that can only be defined at the group level like justice and equaliy.
Of course it is unworkable for politicians to stick rigidly to their manifestos. It is also unworkable for them to discard their manifestos on day one.
On the other hand...
http://en.m.wikipedia.org/wiki/Is_logic_empirical%3F