9eB1 comments on Superintelligence 13: Capability control methods - Less Wrong

7 Post author: KatjaGrace 09 December 2014 02:00AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (44)

You are viewing a single comment's thread. Show more comments above.

Comment author: 9eB1 09 December 2014 06:04:41AM *  0 points [-]

This is a surprisingly brilliant idea, which should definitely have a name. For humans, part of the benefit of this is that it appeals to risk aversion, so people wouldn't want to completely write off one of the scenarios. It also makes it so complex to analyze, that many people would simply fall back to "doing the right thing" naturally. I could definitely foresee benefits by, for example, randomizing whether members of a team are going to be judged based on individual performance or team performance.

I'm not totally sure it would work as well for AIs, which would naturally be trying to optimize in the gaps much more than a human would, and would potentially be less risk averse than a human.

Comment author: diegocaleiro 09 December 2014 07:24:01AM *  1 point [-]

Let's name it:

Hidden Agency Solution

Caleiro Agency Conjecture

Principal agent blind spot

Cal-agency

Stochastic principal agency solution.

(wow, this naming thing is hard and awfully awkward, whether I'm optimizing for mnemonics or for fame - is there any other thing to optimize for here?)

Comment author: Halfwitz 10 December 2014 01:10:25AM 1 point [-]

Fuzzy metrics?

Comment author: diegocaleiro 11 December 2014 12:10:29AM 1 point [-]

Doesn't refer to Principal Agency.