Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

diegocaleiro comments on Nonperson Predicates - Less Wrong

29 Post author: Eliezer_Yudkowsky 27 December 2008 01:47AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (175)

Sort By: Old

You are viewing a single comment's thread. Show more comments above.

Comment author: diegocaleiro 23 November 2010 01:50:57AM 3 points [-]

An obvious yet brilliant point, which should be on the main post (and in your book), not in the replies (Inferential distance to Robin Hanson is supposed to be minimal, yet...)

It is interesting that people working in AI don't want to tackle this problem. When I was Diego 2004, equivalent age of Eliezer 1998, I decided that The Most Important problem was how to avoid catastrophic events from happening either because a part of a program was conscious and suffering, or because everyone uploaded to an unconscious machine. So I dedicated the last 6 years to this impossible problem.

But unlike other problems that interested me "What should I do?" "What is the universe all about anyway" "How the mind works" "How can a brain be intelligent", this one has not become less and less impossible over time.

In fact, when one reads Chalmers' formulations of the hard problem, he can keep you trapped for a long time. It is very hard to understand where he made mistakes (which seem to be on purpose).

So you can stick to Dennett, and some form of monism, but that will not dissolve the problem of how to detect unconscious AI and differentiate it.