= 27d567bc2d48d9f40e9ccc20ea370b15)
paulfchristiano comments on Siren worlds and the perils of over-optimised search - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (411)
It's not really clear why you would have the searching process be more powerful than the evaluating process, if using such a "search" as part of a hypothetical process in the definition of "good."
Note that in my original proposal (that I believe motivated this post) the only brute force searches were used to find formal descriptions of physics and human brains, as a kind of idealized induction, not to search for "good" worlds.
Because the first supposes a powerful AI, while the second supposes an excellent evaluation process (essentially a value alignment problem solved).
Your post motivated this in part, but it's a more general issue with optimisation processes and searches.
Neither the search nor the evaluation presupposes an AI when a hypothetical process is used as the definition of "good."