LESSWRONG
LW

All of XerxesPraelor's Comments + Replies

Debunking Fallacies in the Theory of AI Motivation
XerxesPraelor30

Also, little you've written about CLAI or Swarm Connectionist AI corresponds well to what I've seen of real-world cognitive science, theoretical neuroscience, or machine learning research, so I can't see how either of those blatantly straw-man designs are going to turn into AGI. Please go read some actual scientific material rather than assuming that The Metamorphosis of Prime Intellect is up-to-date with the current literature ;-).

The content of your post was pretty good from my limited perspective, but this tone is not warranted.

Reply
5[anonymous]
Perhaps not, but I don't understand why "AI" practitioners insist on being almost as bad as philosophers for butting in and trying to explain to reality that it needs to get into their models and stay there, rather than trying to understand existing phenomena as a prelude to a general theory of cognition.
Debunking Fallacies in the Theory of AI Motivation
XerxesPraelor50

namely the cases where the AI is trying really hard to be friendly, but doing it in a way that we did not intend.

If the AI knows what friendly is or what mean means, than your conclusion is trivially true. The problem is programming those in - that's what FAI is all about.

Reply
Debunking Fallacies in the Theory of AI Motivation
XerxesPraelor20

Can someone who down voted explain what I got wrong? (note: the capitalization was edited in at the time of this post.)

(and why the reply got so up voted, when a paragraph would have sufficed (or saying "my argument needs multiple paragraphs to be shown, so a paragraph isn't enough"))

It's kind of discouraging when I try to contribute for the first time in a while, and get talked down to and completely dismissed like an idiot without even a rebuttal.

Reply
Debunking Fallacies in the Theory of AI Motivation
XerxesPraelor20

You could at least point to the particular paragraphs which address my points - that shouldn't be too hard.

Reply
4[anonymous]
Sometimes it seems that a commenter did not slow down enough to read the whole paper, or read it carefully enough, and I find myself forced to rewrite the entire paper in a comment. The basic story is that your hypothetical internal monologue from the AGI, above, did not seem to take account of ANY of the argument in the paper. The goal of the paper was not to look inside the AGI's thoughts, but to discuss its motivation engine. The paper had many constructs and arguments (scattered all over the place) that would invalidate the internal monologue that you wrote down, so it seemed you had not read the paper.
Debunking Fallacies in the Theory of AI Motivation
XerxesPraelor10

So, this is supposed to be what goes through the mind of the AGI. First it thinks “Human happiness is seeing lots of smiling faces, so I must rebuild the entire universe to put a smiley shape into every molecule.” But before it can go ahead with this plan, the checking code kicks in: “Wait! I am supposed to check with the programmers first to see if this is what they meant by human happiness.” The programmers, of course, give a negative response, and the AGI thinks “Oh dear, they didn’t like that idea. I guess I had better not do it then."

But now Yud

... (read more)
Reply
2XerxesPraelor
Can someone who down voted explain what I got wrong? (note: the capitalization was edited in at the time of this post.) (and why the reply got so up voted, when a paragraph would have sufficed (or saying "my argument needs multiple paragraphs to be shown, so a paragraph isn't enough")) It's kind of discouraging when I try to contribute for the first time in a while, and get talked down to and completely dismissed like an idiot without even a rebuttal.
1[anonymous]
You completely ignored what the paper itself had to say about the situation. [Hint: the paper already answered your speculation.] Accordingly I will have to ignore your comment. Sorry.
Uncategories and empty categories
XerxesPraelor10

Try this experiment on a religious friend: Tell him you think you might believe in God. Then ask him to list the qualities that define God.

Before reading on, I thought "Creator of everything, understands everything, is in perfect harmony with morality, has revealed himself to the Jews and as Jesus, is triune."

People seldom start religions by saying they're God. They say they're God's messenger, or maybe God's son. But not God. Then God would be this guy you saw stub his toe, and he'd end up like that guy in "The Man Who Would Be King.&q

... (read more)
Reply
2gjm
Of the qualities you list, the last two -- the specifically Christian ones -- mean that if you really take this as defining "God" then you have to say that, e.g., Muslims do not believe in God (because they are very insistent that he is not triune) and nor do any monotheists who don't come from the "Abrahamic" tradition. Of course you are at liberty to define "God" that way, but it doesn't look to me like a great idea. If we drop those (considering them as defining "Christianity" rather than "God", or something), and also (in accordance with Phil's stipulation, though I'm not sure it's a fair one) throw out "is in perfect harmony with morality" on the grounds that Phil doesn't like the term "perfect", we're left with "Creator of everything, understands everything". I think "Fred" satisfies the first condition but not the second. So I think Phil's thought experiment is a bit broken. Maybe we can salvage it. Suppose now that our universe was indeed created by a graduate student in some other universe; his mental capacities far exceed ours and he does in fact understand everything that goes on in our universe. And let's suppose our moral values were somehow implanted in us by this graduate student in accordance with his own. Does the grad student now fit your first three criteria? If so, would you call him God? (My guess: he doesn't, because if there are universes other than ours then creating just our universe doesn't for you count as the right sort of thing to be God. That seems reasonable, though I remark that e.g. the creation story in Genesis 1 seems to show God already having some raw material to work with before he gets started creating; you probably don't want to say that whoever wrote Genesis 1 didn't believe in God.) What the onlookers are complaining at there isn't that Jesus is claiming to be God. It's that he's claiming to have come down from heaven. It's far from clear that Jesus actually claimed to be God -- there are a couple of places in the gospe
Rationality Quotes September 2013
XerxesPraelor410

There is one very valid test by which we may separate genuine, if perverse and unbalanced, originality and revolt from mere impudent innovation and bluff. The man who really thinks he has an idea will always try to explain that idea. The charlatan who has no idea will always confine himself to explaining that it is much too subtle to be explained. The first idea may be really outree or specialist; it may be really difficult to express to ordinary people. But because the man is trying to express it, it is most probable that there is something in it, after

... (read more)
Reply
8ChristianKl
I don't think that's the case. There are plenty of shy intellectuals who don't push their ideas on other people. Darwin sat more than a decade on his big idea. There are ideas that are about qualia. It doesn't make much sense to try to explain a blind person what red looks like and the same goes for other ideas that rest of observed qualia instead of resting on theory. If I believe in a certain idea because I experienced a certain qualia and I have no way of giving you the experience of the same qualia, I can't explain you the idea. In some instances I might still try to explain the blind what red looks like but there are also instance where I see it as futile. One way of teaching certain lessons in buddhism is to give a student a koan that illustrates the lesson and let him meditate over the koan for hours. I don't see anything dishonest about teaching certain ideas that way. If someone thinks about a topic in terms of black and white it just takes time to teach him to see various shades of grey.