dlthomas comments on Thoughts on the Singularity Institute (SI) - Less Wrong

256 Post author: HoldenKarnofsky 11 May 2012 04:31AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (1270)

You are viewing a single comment's thread. Show more comments above.

Comment author: Will_Newsome 11 May 2012 08:05:43PM *  11 points [-]

It's mostly a question for philosophy of mind, I think specifically a question about intentionality. I think the closest you'll get to a mathematical framework is control theory; controllers are a weird edge case between tools and very simple agents. Control theory is mathematically related to Bayesian optimization, which I think Eliezer believes is fundamental to intelligence: thus identifying cases where a controller is a tool or an agent would be directly relevant. But I don't see how the mathematics, or any mathematics really, could help you. It's possible that someone has mathematized arguments about intentionality by using information theory or some such, you could Google that. Even so I think that at this point the ideas are imprecise enough such that plain ol' philosophy is what we have to work with. Unfortunately AFAIK very few people on LW are familiar with the relevant parts of philosophy of mind.

Comment author: shminux 11 May 2012 08:14:07PM *  8 points [-]

It is an EY's announced intention to work toward an AI that is provably friendly. "Provably" means that said AI is defined in some mathematical framework first. I don't see how one can make much progress in that area before rigorously defining intentionality.

I guess I am getting ahead of myself here. What would a relevant mathematical framework entail, to begin with?

Comment author: dlthomas 11 May 2012 08:29:20PM 10 points [-]

I guess I am jumping the shark here.

I don't think that idiom means what you think it means.

Comment author: shminux 11 May 2012 08:35:20PM 3 points [-]

Thank you, fixed.

Comment author: quintopia 17 May 2012 06:23:07AM 3 points [-]

You were probably fishing for "jumping the gun".

Comment author: shminux 18 May 2012 12:55:36AM 3 points [-]

Yeah, should have been shooting instead of fishing.

Comment author: Bugmaster 18 May 2012 12:59:00AM 2 points [-]

It could be said that you shot yourself in the foot by jumping the shark while fishing for a gun.