= 27d567bc2d48d9f40e9ccc20ea370b15)
gwern comments on Open Thread: November 2009 - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (539)
If it's that concerned, it can just blast off into space, couldn't it? Might slow down development, but the hypothetical mud footprints out to be fine... No harm done by computronium in the sun.
The question is should we program it to be that concerned? The human predicate is necessary for CEV if I remember correctly, you would want to extrapolate the volition of everyone currently informationally smeared across the planet as well as the more concentrated humans. I can't find the citation at the moment, I'll hunt tomorrow.
I think the (non)person predicate is necessary for CEV only to avoid stomping on persons while running it. It may not be essential to try to make the initial dynamic as expansive as possible, since a less-expansive one can always output "learn enough to do a broader CEV, and do so superseding this".
Hmm, I think you are right.
We still need to have some estimate of what it will do though so we can predict its speed somewhat.