I found my launch code in my spam folder after the deadline passed. Also I was preoccupied with other things and did not have much time to follow the discussions, or to think about what to do if I had seen the launch code in time. I wonder how many people were in one of these situations (and therefore how much to discount the 125 number by).
One thing I will say is that I’d ideally like to be able to reach an equilibrium where 100s of users every year don’t fire the launch codes, to build up a real tradition of not taking unilateralist action—sitting around and not pressing buttons.
I'm worried about this building a false sense of security, based around a narrative that social dynamics around such buttons are largely similar/transferable so if we can refrain from pressing one kind of button we can also refrain from pressing another kind of button (which in reality may actually be very different both qualitatively and quantitatively).
Even ignoring the above problem, I'm confused why it's valuable to build up a "real tradition" among LW users, given that the wider unilateralist curse problem that our world faces can't possibly be solved by LW users having such a tradition.
Even ignoring the above problem, I'm confused why it's valuable to build up a "real tradition" among LW users, given that the wider unilateralist curse problem that our world faces can't possibly be solved by LW users having such a tradition.
A few points.
First, I don't think it's clear that in the Rationalist / EA community, there is enough reinforcement of this, and I routinely see issues with people "going rogue" and unilaterally engaging in activities that others have warned them would be dangerous, net negative, etc.
Second, it's valuable even as an exemplar; we should be able to say that there is such a community, and that they are capable of exercising at least this minimal level of restraint.
Third, I think it's clear that in the next decade the number of people in the Rationalist-sphere that are in actual positions of (relatively significant) power will continue to grow, and we have already seen some such people emerge in government and in the world of NGOs. For AI, in particular, there are many people who have significant influence in making decisions that could significantly affect Humanity's future. Their active (i.e. passive) participation in this seems likely to at least give them a better understanding of what is needed when they are faced with these choices.
I think these are good points. (I thought of points 2 and 3 myself after writing my comment, but refrained from posting them due to not wanting to accidentally straw-man supporters of the button game/tradition, in case they had stronger arguments in mind.) I'm curious about examples of 1, if you have any in mind that you can share.
Re: examples of point #1, I don't think that shaming in this forum is productive - it's polarizing and stigmatizing rather than helpful. But I do know of several individuals and a couple organizations which are guilty of this, each repeatedly.
I do think that people should be more willing to personally / privately respond if someone does something, and I have done so in several specific cases where someone decided on a unilateralist approach that I thought was damaging.
Yesterday, I held a small Petrov Day ceremony at my house for roommates and a couple friends.
We placed a laptop at the end of the table, displaying the LessWrong website, set to refresh every 30 seconds.
The ruleset we chose for the ceremony was "if it turned out the LessWrong site went down, we would immediately end the ceremony and go home silently."
Also, I entrusted everyone at the gathering with my launch codes.
I am pleased to report that none of them destroyed the frontpage. Hurray!
to build up a real tradition of not taking unilateralist action - sitting around and not pressing buttons.
I don't really understand the analogy to the Unilateralist's Curse here.
The UC is about a situation where an action can legitimately be considered to have both benefits and drawbacks, and someone may then do damage by unilaterally taking action when their estimate of the benefits is incorrectly larger than their estimate of the drawbacks (and people who estimate differently can only avoid taking action).
That doesn't seem very analogous to this situation, where pressing the button only produces harm, and anyone who does press the button will be publicly shamed. Yes there may be some private benefits, such as a spite-motivated person getting some enjoyment. But as I understand it, the UC is supposed to be about a disagreement concerning public benefits vs. harm, so a private benefit just makes someone a troll, not a person taking unilateralist action.
So it's hard for me to see how we would be practicing the act of not taking unilateralist action, by having a ritual where there's no real tradeoff. You can't practice making the right choice in a dilemma, if there's actually no choice.
If we really did want to practice that, then there would need to be a corresponding benefit set up for pressing the button. Maybe something like: when someone presses the button, they get a pre-selected list of ten effective charities from which they choose one. Upon entering the correct launch code, the site goes down and the LW team makes a $10,000 donation to the selected charity, which they have precommitted not to donate otherwise. The person who submitted the codes remains anonymous.
I think you are overestimating the degree to which there was "no real tradeoff". I bet with you that many people felt tempted to enter the launch codes, for reasons that Said mentioned, but also as a resource to be sold (like Jeff Kauffman considered).
Like, it's obvious that if you didn't do the thing with the codes, and just had a button available to every LessWrong user, someone would have pressed it "just for the lulz", which isn't a very good reason to press the button, but as you scale the number of people who are trying to balance the pros and cons, someone will think that the lulz are more important than not pressing the button (and I think "for the lulz" can be pretty validly described as someone doing something for the public good).
I was quite uncertain whether anyone would press the button over the course of the day, and don't think that uncertainty was unjustified. And I would be surprised if you were confident at 90%+ that no one would press the button over the course of the day.
I do think that given how it worked out this time, adding a more concrete ritualized incentive seems maybe good to me, like giving $500 to a charity chosen by whoever presses the button.
I also don't think the constraint of "what matters are only considerations about the public good" is a good constraints for a tradition like this, because in reality obviously many people will not only think about the public good, which actually significantly worsens the problems that the unilateralist curse is talking about, and it seems bad to not incorporate those into this tradition (and the UC is the best abstraction we currently have for talking about this reference class).
FWIW, I thought the ritual this year was fine and I'm not sure adding a cash prize to the ritual itself will be communicating the right lesson. It then starts to feel like a ritual about 'do we care more about symbolism than about saving lives?', rather than a ritual about coordination.
I've had pretty much the same objection to making Petrov day about UC and overall haven't felt entirely satisfied with the counters. (Same with Petrov kind of being a unilateralist, though I moderately buy Habryka's view that he was unilateralist with respect to local command structure but non-unilateralist with respect to humanity.) I generally have felt averse to what feels like dilution of the meaning of UC to something much closer to simple defection.
However, there is a way in which there is kind of disagreement about public benefits in the recent LW button scenario if you can sell your ability to nuke the site to someone who is a troll or motivated by spite, and then use the funds for something you think is good that outweighs the harm. At last Jeff K considered this but was dissuaded by others thinking it was a bad idea.
I like your donation/anonymity suggestion. The public shaming seems like a strong disincentive that disanalogizes the scenario from real world scenarios without this element (or where action would at least draw obvious praise from many).
The other direction is having an "opposing" site that our button takes down and vice versa plus chance of false alarm to more accurately resemble the history (though that is still more what I'd call a standard game theory/cooperation/defection thing than UC).
What an incredible experience.
Felt like I got to understand myself a bit better, got exposed to a variety of arguments I never would have anticipated, forced to clarify my own thoughts and implications, did some math, did some sanity-check math on "what's the value of destroying some of Ben Pace's faith in humanity" (higher than any reasonable dollar amount alone, incidentally — and that's just one variable)... and yeah, this was really cool and legit innovative.
We should make sure the word about this gets out more.
We need more people on LessWrong, and more stuff like this.
People thinking this is just a chat board should think a little bigger. There's some real visionary thinking going on here, and an exceptionally smart and thoughtful community. I'm really grateful I got to see and participate in this. Thanks for all the great work — and for trusting me. Seriously. Y'all are aces.
You know what, I think LessWrong has collectively been worth more than $1,672 to me — especially after the re-launch. Heck, maybe even Petrov Day alone was. Incredibly insightful and potentially important.
I'd do this privately, but Eliezer wrote that story about how the pro-social people are too quiet and don't announce it. So yeah, I'm in for $1,672. Obviously, I wouldn't have done this if some knucklehead had nuked the site.
Now for the key question —
What kind of numbers do we need to put together to get another Ben Pace quality dev on the team? (And don't tell us it's priceless, people were willing to sell out your faith in humanity for less than the price of a Macbook Air! ;)
And yeah, mechanics for donating to LW specifically? Can follow up on email but I imagine it'd be good to have in this thread.
Edit: Before anyone suggests I donate to some highly-ranked charity, after I'd had some success in business I was in the nonprofit world for years and always 100% volunteer, have spent an immense amount of hours both understanding the space and getting things done, and was reasonably effective though not legendarily so or anything. By my quick back of the envelope math, I imagine any given large country's State Department would have paid $50,000 to $100,000 to have Petrov Day happen successfully in such a public way. Large corporations — I've worked with a few — maybe double that range. It was a really important thing and while "budget for hiring developers on a site that facilitates discussion of rationality" has far more nebulous and hard-to-pin-down value than some very worthy projects, it's first a threshold-break thing where a little more might produce much more results, and I think this site can be really important. If I might suggest something, though, perhaps an 80/20 eng-driven growth plan for the site that prioritizes preserving quality and norms would also make sense? We should have 10x the people here. It's very doable. I'm really busy but happy to help if I can. I think a lot of us would be happy to help make it happen if y'all would make it a little easier to know how. Something special is happening here.
Edit2: Okay, my donation is now conditional on banning whoever downvoted this ;) - just kidding. But man, what a strange mix of really great people and total idiots here huh? "I liked this a lot and I'd like to give money." WTF who does this guy think he is. Oh, me? Just someone trying to support the really fucking cool thing that's happening and asking for the logistics of doing so to be posted in case anyone else thinks it's been really cool and great for their life.
Mechanics for donating to LessWrong:
LessWrong is sponsored by CFAR, so if you just make a donation to CFAR and earmark it for the LessWrong project it will show up in our bank accounts. For anything larger than a few hundred dollars it's probably best to send me an email at oliver.habryka@gmail.com and then figure out the details from there.
For smaller donations just using the CFAR PayPal, with a note saying it's for the LessWrong project, and also sending me a small email to make sure it ends up in the right accounts is likely the best thing to do.
Please don't let a downvote or two discourage you. I appreciate your participation here, including these comments :)
I wonder how high we can make the number of trusted users go. If we break a thousand we'll have done something special.
Taking the site down for 24 hours seems far too tame. I use the site weekly, I would rarely even notice it being down for a day.
Thanks for doing this! It was a cool unique experience, and good for community-bonding all us LWers.
On general principles, given the Lizardman's constant of 4-5%, one would expect at least several people to nuke the site. Strange that it didn't happen.
Lizardman's Constant is an observation seen in polls of unfiltered groups of people, but the people who were given the launch codes were selected for trustworthiness.
I clicked the button :( But, I clicked it because the first thing I saw on the page was a big shiny button and hadn't been on the site in a few day and didn't know what it would do. I would not have entered codes if I'd had them. Maybe give a more noticeable warning next to the button next time? Maybe something like "Warning, pressing this button will..." in red text.
Fun experiment
Hurrah! Success! I didn't know what to expect, and am pleasantly surprised to find the Frontpage is still intact. My thanks to everyone who took part, to everyone who commented on yesterday's post, and to everyone who didn't unilaterally blow up the site.
Launch Attempts Results
I said I would share usernames and codes of all attempts to launch the codes. Others on the team told me this seemed like a bad idea in many ways, and on reflection I agree - I think many people were not aware they were signing up for being publicly named and shamed, and I think it's good that people aren't surprised by their actions becoming public. Though if someone had successfully nuked the site I would have named them.
Nonetheless, I’ll share a bunch of info. First of all, the button was in a pretty central place, and it turns out you can hit it accidentally. Ray built the button so that you could only hit it once - it was forever after pressed.
LW user ciphergoth (Paul Crowley) shared his launch codes on Facebook (indeed I had sent him real launch codes), and two users copied and entered them. However, he had actually shared fake codes. "The Last Harbinger" entered them.
A second user entered them, who had positive karma, and was not someone to whom I had sent real codes. However, they failed to properly copy it, missing the final character. To them, I can only say what I had prepared to say to anyone who mis-entered what they believed were correct launch codes. "First, you thought you were a failure to the community. But then, you learned, you were a failure to yourself."
Oli and Ray decided that anyone submitting launch codes deserved a janky user-experience. I hope all of the users enjoyed finding out that when you try to nuke the site, regardless of whether you enter correct or incorrect launch codes, the launch pad just disappears and nothing else happens. (Once you refresh, the page is of course nuked.)
Last night during my house's Petrov Day ceremony, which ran from about 8:10-9:10, I nervously glanced over at the LW frontpage on the open laptop as it refreshed every 60 seconds. Some small part of me was worried about Quirinus_Quirrell following through on his threat to nuke the site at 9pm. I honestly did not expect that someone could create a character hard enough that it would leap out of the book and hold us all hostage in a blackmail attempt. Damn you Eliezer Yudkowsky!
Looking Ahead
I thought the discussion was excellent. I mostly avoided participating to let others decide for themselves, but I might go back and add more comments now it's done. As Said Achmiz pointed out, it'll be better next year to have more time in advance for people to discuss the ethics of the situation and think, and that will be even more informative and valuable. Though I still learned a lot this year, and I think overall it turned out as well as I could've hoped.
I'll think more about how to do it next year. One thing I will say is that I'd ideally like to be able to reach an equilibrium where 100s of users every year don't fire the launch codes, to build up a real tradition of not taking unilateralist action - sitting around and not pressing buttons. Several users have suggested to me fun, gamified ways of changing the event (e.g. versions where users are encouraged to trick other users into thinking you can trust them but then nuke the site), but overall in ways that I think decreased the stakes and common knowledge effects, which is why I don't feel too excited about them.