= 27d567bc2d48d9f40e9ccc20ea370b15)
Slider comments on Presidents, asteroids, natural categories, and reduced impact - Less Wrong Discussion
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (16)
If ¬X happened, the result would be missaiming. But since X happens (almost certainly), it aims correctly.
The programmer expects to ¬X and it must program the bot with things that are X agnostic so it is planning to not aim. Then because the programmed bot can't be X sensitive it will make essentially as if ¬X.
If the mission is to do the grue thing given that t<1 and the executor of the strategy can not have references to t the simple strategy is for the bot to do the green thing. Now if we put the programmed bot in a environment where t>1 the grue thing to do would be to press the blue button but the bot presses the green button. Such a solution is not grue-friendly or blue-friendly.
I'm not sure what you're saying. The AI is programmed to be reduced impact, conditional on ¬X. If ¬X happens, then outputting the correct y coordinates is reduced impact, which it will thus do (as it is separately motivated to do that).
So, given ¬X, the AI is motivated to: a) output the correct y coordinate (or cause its subagent to do so), b) have a reduced impact overall.
The whole construction is an attempt to generalise a) and b) to X, even though they are in tension/contradiction with each other in X (because outputing the correct y coordinate will have a high impact).
These bits are contradictory. One tells of a story where two low-impact options are tie-breaked by an aiming instinct to aim anyway. The other tells that "sit tight" instinct will overwhelm the aiming instinct.
If you want to control what happens in X, drives that are conditioned on ¬X are irrelevant. In my understanding the attempt is to generalise the reduced impact drive by not having it conditioned on X. Then what it does in ¬X can not be based on the fact that ¬X. But it can't deduce that aiming is low impact even in ¬X because it must assume that the x-aiming robot could be on and that would make it a high impact decision. It must use the same decision process in both X and ¬X and the X decision process can't be based on what it would do if it where allowed to assume that ¬X (that is you are not allowed to know whether the grue object is currently green or blue and you can't decide what you would do if it were green based on what you would do if it was blue).
Indeed. I have corrected the top post. Thanks!