= 27d567bc2d48d9f40e9ccc20ea370b15)
datadataeverywhere comments on The Design Space of Minds-In-General - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (82)
A hundred bytes in what language? I get the intuition, but it really seems to me like paper clips are really complex. There are lots of important qualities of paperclips that make them clippy that seem to me like they'd be very hard to get an AI to understand. You say you're giving it a shape, but that shape is not at all easily defined. And its molecular structure might be important, and its size, and its density, and its ability to hold sheets of paper together... Shape and molecular component aren't fundamental attributes of the universe that an AI would have a native language for. This is why we can't just keep an oracle AI in a box -- it turns out that our intuitive idea of what a box is is really hard to explain to a de novo AI. Paperclips are similar. And if the AI is smart enough to understand human concepts that well, then you should also be able to just type up CEV and give it that instead... CEV is easier to describe than a paperclip in that case, since CEV is already written up. (Edit: I mean a description of CEV is written up, not CEV. We're still working on the latter.)
Sample a million paperclips from different manufacturers. Automatically cluster to find the 10,000 that are most similar to each other. Anything is a paperclip if it is more similar to one of those along every analytical dimension than another in the set.
Very easy, requires no human values. The AI is free to come up with strange dimensions along which to analyze paperclips, but it now has a clearly defined concept of paperclip, some (minimal) flexibility in designing them, and no need for human values.
Counter:
Sample a million people from different continents. Automatically cluster to find the 10,000 that are most similar to each other. Anything is a person if it is more similar to one of those along every analytical dimension than another in the set.
This is already tripping majoritarianism alarm bells.
I would meditate on this for a while when trying to define a paperclip.
Sure.
For that matter, one could play all kinds of Hofstadterian games along these lines... is a staple a paperclip? After all, it's a thin piece of shaped metal designed and used to hold several sheets of paper together. Is a one-pound figurine of a paperclip a paperclip? Does it matter if you use it as a paperweight, to hold several pieces of paper together? Is a directory on a computer file system a virtual paperclip? Would it be more of one if we'd used the paperclip metaphor rather than the folder metaphor for it? And on and on and on.
In any case, I agree that the intuition that paperclips are an easy set to define depends heavily on the idea that not very many differences among candidates for inclusion matter very much, and that it should be obvious which differences those are. And all of that depends on human values.
Put a different way: picking 10,000 manufactured paperclips and fitting a category definition to those might exclude any number of things that, if asked, we would judge to be paperclips... but we don't really care, so a category arrived at this way is good enough. Adopting the same approach to humans would similarly exclude things we would judge to be human... and we care a lot about that, at least sometimes.
TheOtherDave got it right; I'm wasn't trying to give a complete definition of what is and isn't a paperclip, I was just offering forth an easy to define (without human values) subset that we would still call paperclips.
It has plenty of false negatives, but I don't really see that as a loss. Likewise, your personhood algorithm doesn't bother me as long as we don't use it to establish non-personhood.
Ah. This distinction escaped me; I tend to use definitions in a formal logical style (P or ~P, no other options).