You're looking at Less Wrong's discussion board. This includes all posts, including those that haven't been promoted to the front page yet. For more information, see About Less Wrong.

AmagicalFishy comments on xkcd on the AI box experiment - Less Wrong Discussion

15 Post author: FiftyTwo 21 November 2014 08:26AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (229)

You are viewing a single comment's thread. Show more comments above.

Comment author: AmagicalFishy 24 November 2014 08:23:58PM *  1 point [-]

I mean moreso: Consider a FAI so advanced that it decides to reward all beings who did not contribute to creating Roko's Basilisk with eternal bliss, regardless of whether or not they knew of the potential existence of Roko's Basilisk.

Why is Roko's Basilisk any more or any less of a threat than the infinite other hypothetically possible scenarios that have infinite other (good and bad) outcomes? What's so special about this one in particular that makes it non-negligible? Or to make anyone concerned about it in the slightest? (That is the part I'm missing. =\ )

Comment author: ChristianKl 25 November 2014 04:19:01PM 0 points [-]

Why is Roko's Basilisk any more or any less of a threat than the infinite other hypothetically possible scenarios that have infinite other (good and bad) outcomes?

The idea is that an FAI build on timeless decision theory might automatically behave that way. There's also Eliezer's conjecture that any working FAI has to be build on timeless decision theory.

Comment author: RowanE 25 November 2014 01:37:56PM 0 points [-]

Well, in the original formulation, Roko's Basilisk is an FAI that decided the good from bringing an FAI into the world a few days earlier (saving ~150,000 lives per day eralier it gets here) outweighs the bad from making the threats, so there's no reason it shouldn't want you to aid FAI projects that promise not to make a Basilisk, just as long as you do something instead of sitting around, so there's no inconsistency and now there's more than one being trying to acausally motivate you into working yourself to the bone for something that most people think is crazy.

More generally, we have more than zero information about future AI, because they will be built by humans if they are built at all. Additionally, we know even more if we rule out certain categories, such as the archetypal "paperclip maximiser". There's room for a lot of speculation and uncertainty, but far from enough room to assume complete agnosticism and that for every AI that wants one thing from us there's an equal and opposite AI that wants the opposite.

Comment author: ChristianKl 25 November 2014 04:19:00PM 1 point [-]

FAI projects that promise not to make a Basilisk

A priori it's not clear that a project can hold such a promise.