ata comments on David Chalmers' "The Singularity: A Philosophical Analysis" - Less Wrong

33 Post author: lukeprog 29 January 2011 02:52AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Article Navigation

Comments (202)

Comments (202)

Sort By: Best

Show:

You are viewing a single comment's thread. Show more comments above.

Comment author: ata 29 January 2011 03:53:27AM * 9 points [-]

If you're a seed AI and you're somewhat confused about what your creator meant when they said "valueOfCreatingPaperclips=infinity", so you do what you think they were trying to get you to do, which was to create economic value by making paperclips...

But that sounds like we're programming the AI in English. I can't see an AI with a motivational system well-defined enough to work at all getting confused in that way; would "Do what my creator intended me to do, if I can't figure out what else to do" even show up as a motivational drive if it is not explicitly coded in?

Comment author: PhilGoetz 31 January 2011 12:22:34AM 0 points [-]

He's speaking to you in English, because you speak English. Also, the website wouldn't let him post the bytecode.

You made a sweeping statement about all possible AI architectures. What are your reasons for it?

Comment author: jacob_cannell comment score below threshold [+] (19 children)

Comment author: ata 01 February 2011 05:23:03AM 3 points [-]

There's reason to suspect that any human-level AI must be programmed in human languages.

In fact, that's almost tautological by virtue of the Turing Test.

What?

Do you mean humanlike AIs? An AI capable of passing the Turing Test would of course need to understand human language well enough to act convincingly human (or at least do a really good imitation), but that's not necessarily a human-level AI (convincing people that you're human is a separate task from actually being human, probably a much easier one), and human-level AIs in general needn't necessarily understand human language any better than any other sort of language by default.

Anyway, an AI being "programmed in human languages" seems to be going by the "programming = instructions being given to a human servant" metaphor, and if you want that to work, you clearly first need to write the servant in something other than human language. And copying human psychology well enough that the AI actually understands human language as well as a human does, rather than being able to imitate understanding well enough to carry on a text-based conversation, is no easy task, and is probably a lot harder than manually coding a simple goal system like paperclip maximization in a lower-level language. But that could still be an AGI.

Comment author: jacob_cannell 01 February 2011 05:44:59AM * -2 points [-]

Human level AI - an AGI design capable of matching the full intellectual capabilities of the best human scientists/engineers.

To get to H level in a practical timeframe, a human AI will have to learn human knowledge, it will have to experience an equivalent to a standard 20-25 year education.

Learning human knowledge in practice requires learning human language as an early initial precursor step.

The software of a human mind - the memeset or belief network, is essentially a complex human language program.

For an AI to achieve human-level, it will have to actually understand human language as well as a human does, and this requires a bunch of algorithmic complexity from the human brain at the hardware level and it implies the capability to parse and run human language programs.

So you only need to program the infant brain in a programming language - the rest can be programmed in human language.

is probably a lot harder than manually coding a simple goal system like paperclip maximization in a lower-level language. But that could still be an AGI

If it doesn't have the capacity to understand human level language then it's not an AGI - as that is the defining characteristic of the concept (by my/Turing's definition).

And thus by extension, the defining characteristic of a human-mind is human language capability.

EDIT: Why are you downvoting? Don't agree and don't want to comment?

Comment author: wedrifid 01 February 2011 06:15:43AM 1 point [-]

If it doesn't have the capacity to understand human level language then it's not an AGI - as that is the defining characteristic of the concept (by my/Turing's definition).

Turing never intended his test to be adopted as "the defining characteristic of the concept [of AGI]" in anything like this fashion. Human 'level' language is also somewhat misleading in as much as it implies it is reaching a level of communication power rather than adapting specifically to the kind of communications humans happen to have evolved - especially the quirks and weaknesses.

Comment author: jacob_cannell 01 February 2011 06:48:16AM 0 points [-]

Turing never intended his test to be adopted as "the defining characteristic of the concept [of AGI]" in anything like this fashion.

I disagree somewhat. It's difficult to know exactly what "he intended", but the opening of his paper which introduces the concept, starts with "Can machines think?", and describes a reasonable language based test: an intelligent machine is one that can convince us of it's intelligence in plain human language.

Human 'level' language is also somewhat misleading in as much as it implies it is reaching a level of communication power rather than adapting specifically to the kind of communications humans happen to have evolved - especially the quirks and weaknesses

I meant natural language, the understanding of which certainly does require a certain minimum level of cognitive capabilities.

Comment author: shokwave 01 February 2011 07:24:58AM 2 points [-]

We have a much greater understanding of what the "think" in "Can machines think?" means now. We have better tests than seeing if they can fake human language.

Comment author: jacob_cannell 01 February 2011 07:31:14AM 0 points [-]

The test isn't about faking human language, it's about using language to probe another mind. Whales and elephants have brains built out of similar quantities of the same cortical circuits but without a common language stepping into their minds is very difficult.

What's a better test for AI than the turing test?

Comment author: wedrifid 01 February 2011 07:46:14AM 3 points [-]

What's a better test for AI than the turing test?

Give it a series of fairly difficult and broad ranging tasks, none of which it has been created with existing specialised knowledge to handle.

Comment author: jacob_cannell 01 February 2011 07:57:22AM 0 points [-]

Yes - the AIQ idea.

But how do you describe the task and how does the AI learn about it? There's a massive gulf between AI's which can have the task/game described in human language and those that can not. Whale brains and elephants fall in the latter category. An AI which can realistically self-improve to human levels needs to be in the former category, like a human child.

You could define intelligence with an AIQ concept so abstract that it captures only learning from scratch without absorbing human knowledge, but that would be a different concept - it wouldn't represent practical capacity to intellectually self-improve in our world.

continue this thread »

Comment author: anon895 01 February 2011 07:55:34AM 0 points [-]

Possibly relevant: AIXI-style IQ tests.

Comment author: wedrifid 01 February 2011 06:19:44AM * 2 points [-]

Programming an AI in C++? Doesn't compute.

Of the concepts named for Alan Turing it is "Turing Completeness" that is far more interesting and important than the chatbot test. If you think on the concept of a Turing complete computation system you will perhaps realise why the rest of us would consider your claim extremely silly. Well, one of the reasons anyway.

Comment author: jacob_cannell 02 February 2011 07:38:11AM 0 points [-]

That last statement you quoted was silly. Not funny at all, apparently.

Comment author: Will_Newsome comment score below threshold [+] (4 children)

ata comments on David Chalmers' "The Singularity: A Philosophical Analysis" - Less Wrong

Article Navigation

Comments (202)

Nearest Meetups

Virtual Study Room

Recent Comments

Recent Posts

Recent Rationality Quotes

Recent Wiki Edits

Top Contributors, 30 Days

Recent Karma Awards

Comments (202)

Top Contributors, 30 Days

You'll need to login or register to do that

(Don't worry, it only takes a few seconds)

Create

Login