Vaniver comments on Avoiding Your Belief's Real Weak Points - Less Wrong

49 Post author: Eliezer_Yudkowsky 05 October 2007 01:59AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (203)

Sort By: Old

You are viewing a single comment's thread. Show more comments above.

Comment author: Vaniver 20 April 2012 02:27:35PM 1 point [-]

Is it that an unbiased, unfailing intelligence replaces humans as the primary organizers and arbiters of power in our society, or is it that humanity itself is digitized?

Pretty much the first, but with a perspective worth mentioning. Expressing human values in terms that humans can understand is pretty easy, but still difficult enough to keep philosophy departments writing paper after paper and preachers writing sermon after sermon. Expressing human values in terms that computers can understand- well, that's tough. Really tough. And if you get it wrong, and the computers become the primary organizers and arbiters of power- well, now we've lost the future.