Blueberry comments on The Social Coprocessor Model - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (570)
Yes, I understand the signal must be hard to fake. But if the concern is merely about optimizing signal quality, wouldn't it be an even stronger mechanism to noticeably couple your payoff profile to a credible mechanism?
Just as a sketch, find some "punisher" that noticeably imposes disutility (like repurposing the signal faker's means toward paperclip production, since that's such such a terrible outcome, apparently) on you whenever you deviate from your purported decision theory. It's rather trivial to have a publicly-viewable database of who is coupled to the punisher (and by what decision theory), and to make it verifiable that any being with which you are interacting matches a specific database entry.
This has the effect of elevating your signal quality to that of the punisher's. Then, it's just a problem of finding a reliable punisher.
Why not just do that, for example?
We do. That's one of the functions of reputation and gossip among humans, and also the purpose of having a legal system. But it doesn't work perfectly: we have yet to find a reliable punisher, and if we did find one it would probably need to constantly monitor everyone and invade their privacy.
Yet another reason why people invented religion...