= 27d567bc2d48d9f40e9ccc20ea370b15)
Epictetus comments on Futarchy and Unfriendly AI - Less Wrong Discussion
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (27)
Feedback controls. Futarchy is transparent,carried out in real time, and gives plenty of room to adjust values and change strategies if the present ones prove defective. On the other hand, a superintelligent AI would basically run as a black box. The operators would set the values, then the AI would use some method to optimize and then spit out the optimal strategy (and presumably implement it). There's no room for human feedback between setting the values and implementing the optimal strategy.
This relates to my previous post on confounding in Prediction Markets. In my analysis, if we allow human feedback between setting the values and implementing the strategy, you break the causal interpretation of the prediction market and therefore lose the ability to use it for optimization. This is obviously a trade-off between other considerations that may be more important, but you will run into big problems if the market participants expect there is a significant probability that humans will override the market
Here and elsewhere I've advocated* that, rather than using Hanson's idea of target-values that are objectively verifiable like GDP, futarchy would do better to add human feedback in the stage of the process where it gets decided whether the goals were met or not. Whoever proposed the goal would decide after the prediction deadline expired, and thus could respond to any improper optimizing by refusing to declare the goal "met" even if it technically was met.
[ * You can definitely do better than the ideas on that blog post, of course.]