= 27d567bc2d48d9f40e9ccc20ea370b15)
diegocaleiro comments on Superintelligence 21: Value learning - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (33)
If all civilizations HailMary to value-code they would all find out the others did the same and because the game doesn't end there, in round two they would decide to use a different approach. Possibly, like undifferentiated blastula cells use an environmental asymmetric element (gravity) to decide to start differentiating, AGI's could use local information to decide whether they should HailMary again on the second hypothetical round or if they should be the ones deciding for themselves (say information about where you are located in your Hubble volume, or how much available energy there still is in your light cone or something).
Isn't it the civilization not the AGI that will need to decide what to do?
That depends on whether the AGI is told (and accepts) to HailMary once, or to HailMary to completion, or something in between. It also depends on which decision theory the AGI uses to decide I believe. There seem to be, for a large ensemble of decisions, a one-round version of the many-round decision ("No Regrets" Arntzenius2007, "TDT" Yudkowksy2010, "UDT" WeiDai 20xx).