ChristianKl comments on What should a friendly AI do, in this situation? - Less Wrong

8 Post author: Douglas_Reay 08 August 2014 10:19AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (66)

You are viewing a single comment's thread. Show more comments above.

Comment author: [deleted] 08 August 2014 01:21:01PM 11 points [-]

Let me offer another possibility for discussion.

Neither of the two original powerpoints should be presented, because both rely on an assumption that should not have been present. Albert, as an FAI under construction, should have been preprogrammed to automatically submit any kind of high impact utility calculations to human programmers without it being an overridable choice on Albert's part.

So while they were at the coffee machine, one of the programmers should have gotten a text message indicating something along the lines of 'Warning: Albert is having a high impact utility dilemma considering manipulating you to avert an increased chance of an apocalypse.'

My general understanding of being an FAI under construction is that you're mostly trusted in normal circumstances but aren't fully trusted to handle odd high impact edge cases (Just like this one)

At that point, the human programmers, after consulting the details, are already aware that Albert finds this critically important and worth deceiving them about (If Albert had that option) because the oversight committee isn't fast enough. Albert would need to make a new powerpoint presentation taking into account that he had just automatically broadcasted that.

Please let me know about thoughts on this possibility. It seems reasonable to discuss, considering that Albert, as part of the set up, is stated to not want to deceive his programmers. He can even ensure that this impossible (or at least much more difficult) by helping the programmers in setting up a similar system to the above.

Comment author: ChristianKl 08 August 2014 07:07:31PM 0 points [-]

I'm not sure if identifying high impact utility calculations is that easy. A lot of Albert's decisions might be high utility.

Comment author: [deleted] 08 August 2014 08:44:05PM -1 points [-]

I was going by the initial description from Douglas_Reay:

Albert is a relatively new AI, who under the close guidance of his programmers is being permitted to slowly improve his own cognitive capability.

That does not sound like an entity that should be handling a lot of high impact utility calculations. If an entity was described as that and was constantly announcing it was making high impact utility decisions, that either sounds like a bug or people are giving it things it isn't meant to deal with yet.