Qiaochu_Yuan comments on Evaluating the feasibility of SI's plan - Less Wrong

25 Post author: JoshuaFox 10 January 2013 08:17AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Article Navigation

Comments (186)
Tags: agi friendliness singularity institute si strategy ai safety

Comments (186)

You are viewing a single comment's thread. Show more comments above.

Comment author: Qiaochu_Yuan 11 January 2013 08:35:44PM *  0 points [-]

The AI program could have many different types of output, some of which are controlled by the AI, and some of which are not.

Can you elaborate on what you mean by "control" here? I am not sure we mean the same thing by it because:

This is not "AI output", because the AI cannot control it (except by actually changing its own beliefs), but it is "AI program output", because the program that outputs the answer is the same program as the one that performs all the cognition.

If the AI can control its memory (for example, if it can arbitrarily delete things from its memory) then it can control its beliefs.

Comment author: [deleted] 12 January 2013 12:41:35AM 0 points [-]

Yeah, I guess I'm imagining the AI as being very much restricted in what it can do to itself. Arbitrarily deleting stuff from its memory probably wouldn't be possible.

Powered by Reddit Powered by Reddit