= 27d567bc2d48d9f40e9ccc20ea370b15)
nshepperd comments on Why didn't people (apparently?) understand the metaethics sequence? - Less Wrong Discussion
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (229)
What I meant is that "prime heaps" are not about pebblesorters. There are exactly zero pebblesorters in the definitions of "prime", "pebble" and "heap".
If I told you to sort pebbles into prime heaps, the first thing you'd do is calculate some prime numbers. If I told you to do whatever pebblesorters care about, the first thing you'd do is find one and interrogate it to find out what they valued.
If I gave you a source code of a Friendly AI, all you'd have to do would be to run the code.
If I told you to do whatever human CEV is, you'd have to find and interrogate some humans.
The difference is that by analysing the code of the Friendly AI you could probably learn some facts about humans, while by learning about prime numbers you don't learn about the pebblesorters. But that's a consequence of humans caring about humans, and pebblesorters not caring about pebblesorters. Our values are more complex than prime numbers and include caring about ourselves... which is probably likely to happen to a species created by evolution.