= 27d567bc2d48d9f40e9ccc20ea370b15)
Wei_Dai comments on Tiling Agents for Self-Modifying AI (OPFAI #2) - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (260)
(Quick note: Wei's quining violates the naturalistic principle, not the Vingean principle. Wei's actions were still inside quantifiers but had separate forms for self-modification and action. So did Benja's original proposal in the Quirrell game, which Wei modified - I was surprised and impressed when Benja's polymorphism approach carried over to a naturalistic system.)
Was it UDT1.1 (as a solution to this problem) that violates the Vingean principle?
Also, I'm wondering if Benja's polymorphism approach solves the "can't decide whether or not to commit suicide" problem that I described here. Your paper doesn't seem to address this problem since the criteria of action you use all talk about "NULL or GOAL" and since suicide leads to NULL, an AI using your criterion of action has trouble deciding whether or not to commit suicide for an even more immediate reason. Do you have any ideas how your framework might be changed to allow this problem to be addressed?
As I remarked in that thread, there are many possible designs that violate the Vingean principle, AFAICT UDT 1.1 is one of them.
Suicide being permitted by the NULL option is a different issue from suicide being mandated by self-distrust. Benja's TK gets rid of distrust of offspring. Work on reflective/naturalistic trust is ongoing.