= 27d567bc2d48d9f40e9ccc20ea370b15)
Eliezer_Yudkowsky comments on CEV: coherence versus extrapolation - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (17)
What if Archipelagos, your off-the-cuff solution, turn out to be a bad idea? You're now baking in the notion of Archipelagos beyond any hope of revocation even if there's some piece of knowledge that would make you flee in horror from it. Thinking about this for another year isn't going to make me significantly less nervous about it.
(paraphrased quote)
That's the generic concern. The only way to circumvent the concern seems to be to have all the relevant pieces of knowledge, including about what we really want and how we'd react to getting it. But that's not knowledge we're able to have. We'll be left with critical engineering choices underconstrained by our knowledge. Well worth being nervous about, but also worth suggesting possible improvements to the engineering choice, I suppose. :(