DanArmak comments on Are you a Democrat singletonian, or a Republican singletonian? - Less Wrong

-12 Post author: PhilGoetz 01 October 2009 09:35PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (37)

You are viewing a single comment's thread. Show more comments above.

Comment author: DanArmak 02 October 2009 04:39:09PM *  1 point [-]

Yes. But when it's being built by someone else in a not very transparent setting, what probability do you assign to its being generally-human-Friendly instead of particular-human-Friendly towards its creators? And how do you make sure its creators don't commit fatal mistakes?

Suppose a group (foreign government, startup, etc) is building a singleton. They say it's for the benefit of all mankind and other generally nice things. They won't let you inspect it up close and don't reveal their theories & tech, for fear of competition. Even if you offer to help, they say the value of your help is too small to risk your subverting their goals. You have the choice of:

  1. Stopping them by force - use your biggest bombs, no questions asked.
  2. Conquering them, trying to recover their technology, and using it to try to build your own singleton.
  3. As above, but releasing all information and trying to make your singleton-building process 100% transparent and open to anyone who wants to participate.
  4. Ignoring them.
  5. Defending them against any attackers.

Edit: 6. Ignoring them but starting your own singleton project and trying to finish first. Variations: (6a) secret project, (6b) public but not transparent project, just like theirs, and (6c) transparent project inviting others to assist.

Anyone choosing 6 would have to present a very good reason for not choosing 2 or 3 instead, since competition is an obvious problem, winner takes all.

What do you do? I argue that most people/governments/armies will choose 1 or 2.

Comment author: wedrifid 02 October 2009 04:47:56PM 0 points [-]

Yes. But

... We are diverting a bit from 'any', 'any other' and 'any kind' here.

And yes, the only credible alternative to 1 and 2 would be 4 in those cases where the agent predicts their own singleton to foom first.

Comment author: DanArmak 02 October 2009 04:55:39PM 1 point [-]

We are diverting a bit from 'any', 'any other' and 'any kind' here.

Well, I think any group would tend to consider any other group 'not transparent and reliable enough', given the extremely high stakes involved. A development effort that most people felt was open enough would be so open that there would effectively be no separate groups.

So, in any situation where the description of "one powerful group building a singleton and another group watching them" applies, I stand by my prediction.

Comment author: wedrifid 02 October 2009 05:22:27PM 1 point [-]

How about if the figurehead made the effort to signal that he is too silly for a sophisticated body to consider a threat? After all, I'd sooner put plastic bottles of water on my roof than admit that some anime obsessed nerd was a bigger threat than me.