= 27d567bc2d48d9f40e9ccc20ea370b15)
sark comments on Is Google Paperclipping the Web? The Perils of Optimization by Proxy in Social Systems - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (104)
Nice write-up. A couple of comments:
First, what is the real difference between "optimisation by proxy" and the general notion of drawing inferences from evidence? The PageRank algorithm infers the usefulness of a page from its link structure, but any search engine whatsoever will always draw this inference from some evidence. There is no such thing, even in principle for a superintelligence, as just looking directly at the underlying "quality" characteristic.
Second, I don't think that Google is a good FAI analogy because any negative Google effects are due to its design deficiencies (plus computational imitations, etc etc), which is fully within our control to change, whereas a fundamental FAI consideration is our inability to outwit or modify it. An unfriendly AI works intelligently towards undesirable goals, where as Google, if it can be said to have any goals, works unintelligently towards them.
I think when we talk about optimization by proxy we usually compare something that tracks the desired quality much more precisely compared to another variable which we use as a computational shortcut. That is meaningful enough a distinction.
This reminds me of a discussion in some forum where somebody said that it was "impossible to compress a DVD losslessly because it is already lossy". Of course, what was actually meant was lossless compression of that lossy data, not of the original source.