I deliberately refrained from mentioning this in public yesterday out of respect for the spirit of the game, but I was disappointed that the SHA-256 hashes of the launch codes were publicly visible in the source code (such that someone could try to crack them offline) rather than being stored in the database (such that all guesses have to go online through the server).
Okay, yes, I understand that this wasn't a "serious" vulnerability because the launch codes were sufficiently high-entropy (actually random, not chosen by a human) that no one is going to be able to crack the hash. (I didn't make the list this year, but my launch code from last year was 16 alphanumeric characters, which is bits of entropy, which I think is being an expectation of 7 billion years at 100 billion hashes per second? Oh, except knock off a couple orders of magnitude, because there were a hundred codes.)
Still, in principle, as a matter of security mindset, putting the hashes in the database lets you depend on one fewer assumption (about how much computing power the adversary has). It seems like the sort of thing to do as part of the spirit of the game, given that this ki...
I made this decision a while ago, mostly because it was something like an hour faster to get things going this way. I did the fermi, did some research into SHA-256 to make sure it was sufficiently pre-image resistant, and decided to go ahead. I stand by this decision, and think your comment is a straightforward misapplication of security mindset.
I think going through the motions of making sure the hashes weren't publicly available would have been just virtue signaling, and the real security risks almost certainly live somewhere else in our stack (some out-of-date-library that has a security vulnerability, some OS-level issue, some configuration issue in our VPN, a social engineering attack on someone in our team or a past collaborator). There is no point in pursuing a security mindset if you are virtually certain that the thing you would be investing resources into would not be your weakest attack point. I know that LessWrong is not robust against a sophisticated dedicated attacker. I think it would be dumb of me to look at an insulated part of our stack and harden that to withstand a sophisticated attacker, when there are many other attack vectors that are much more fragile.
I sort of disagree. Not necessarily that it was the wrong choice to invest your security resources elsewhere--I think your threat model is approximately correct--but I disagree that it's wrong to invest in that part of your stack.
My argument here is that following best practices is a good principle, and that you can and should make exceptions sometimes, but Zack is right to point it out as a vulnerability. Security best practices exist to help you reduce attack surface without having to be aware of every attack vector. You might look at this instance and rightly think "okay but SHA-256 is very hard to crack with keys this long, and there is more low hanging fruit". But sometimes you're going to make a wrong assumption when evaluating things like this, and best practices help protect you from limitations of your ability to model this. Maybe your SHA-256 implementation has a known vulnerability that you didn't check, maybe your secret generation wasn't actually random, etc. I don't think any of these apply in this case, but I think sometimes you're likely to be mistaken about your assumptions. The question becomes a more general one about when it makes sense to follow best practices ...
I agree that this is a correct application of security mindset; exposures like these can compound with, for example, someone's automatic search of the 100 most common ways to screw up secure random number generation such as by using the current time as a seed. Deep security is about reducing the amount of thinking you have to do and your exposure to wrong models and stuff you didn't think of.
Furthermore, It is also not inconceivable to me that an adversary might be able to use the hash itself without cracking it. For example, the sha256 hash of some information is commonly used to prove that someone has that information without revealing it, so an adversary, using the hash, could credibly lie that he already possesses a launch code and in a possible counterfactual world where no one found about the client side leaking the hash except this adversary, use this lie to acquire an actual code with some social engineering.
Like:
"Attention Lesswrong! With trickery I have acquired a launch code capable of destroying your site. As proof here is the sha256 hash of it: <hash>.
This is not a trick, I will leave plenty of time for you to check with your EA buddies that the hash is valid before you need to meet my demands.
I demand a launch code capable of destroying the EA forum sent to me until <time> or I will nuke this site and to this I precommitted. I won't reveal what I plan to do with the launch code you will send to me, but by basic game theory your interest is in sending it to me as your site's destruction is not certain that way.
I can't prove it to you, but irl I precommitted to nuking the site if my demands are not met and also that I won't send any more messages to prevent useless debating.
I hope you will make the correct choice!"
going through the motions of making sure the hashes weren't publicly available would have been just virtue signaling
Yes. That's what I meant by "the sort of thing to do as part of the spirit of the game": in an actual nuclear (or AI) application, you'd want to pick the straightforwardly best design, not the design which was "something like an hour faster to get things going this way", right?
So as part of the wargame ritual, maybe you should expect people to leave annoying nitpicky comments in the genre of "Your hashes are visible", even if you don't think there's any real risk?
Does that seem weird? For more context on why I'm thinking this way, I thought last year's phishing attack provided us with a very valuable and educational "red team" service that it'd be fun to see continued in some form. ("Coordinate to not destroy the world" is an appropriate premise for an existential-risk-reduction community ritual, but so is intelligent adversaries making that difficult.) I'm not personally vicious enough to try to get the site nuked, but putting on a white hat and thinking about how it could be done feels on-theme.
...your comment is a straightforward misapplication of security mindse
I had one of the EA Forum's launch codes, but I decided to permanently delete it as an arms-reduction measure. I no longer have access to my launch code, though I admit that I cannot convincingly demonstrate this.
Attention LessWrong - I do not have any sort of power as I do not have a code. I also do not know anybody who has the code.
I would like to say, though, that I had a very good apple pie last night.
That’s about it. Have a great Petrov day :)
I am opposed to the implementation of this exercise, I believe its basic concept seriously undercuts the moral lesson we should take from Petrov Day.
The best way to not blow ourselves up is to not make nuclear weapons. On a day dedicated to not blowing ourselves up, LW has decided to manufacture a bunch of completely unneeded nuclear weapons, hand them out to many people, and then hope really hard that no one uses them. This is like a recovering addict carrying drugs on his person in order to make a point about resisting temptation: he is at best bragging and at worst courting disaster so boldly that one should wonder if he really wants to avoid self-destruction. This makes a good allegory for the senseless near-apocalypse of the Cold War, but deliberately creating a senseless risk does not seem like an appropriate way of celebrating the time we narrowly avoided triggering a senseless risk.
My perspective is that the ritual has more than one dimension: I claim that this is low-risk training for future events, rather than only a celebration of a past event. Many senseless risks remain (including nuclear weapons), and we have no control whatsoever over whether they persist. Petrov is a foundation story because rationalist-as-we-use-it means making good decisions; even when we did not make the risks; even when the situation fundamentally stupid.
If we never even attempt to simulate these situations, then I believe we're not giving the problem its due.
I think to the extent that the Petrov Day game is training anything, it's training the opposite of what we should want. In the game, all the social pressure is unanimously and strongly opposed to pressing the button (sometimes to the extent of ostracizing people and threatening their careers). But in real life, if everyone were unanimously opposed to pressing the button, the button would never have been constructed in the first place. The real Petrov was not rewarded for his actions but demoted and sidelined. In the real situation that's supposedly being trained for, the social pressure will be ambivalent at best, but more likely telling you that you should press the button, and only your own moral compass and fear of death would be telling you not to.
Today I learned that loss aversion is very weird.
I had a code for this last year. I thought this game was cool, and I faithfully refrained from using them. Then today I saw this post, checked my email, and when I didn't find any codes I noticed that I felt sad. I'm not deeply sad, to be clear, but I definitely am a little disappointed that I don't have a string of numbers which have exactly one use which I have next to zero intention of ever using, that only two hundred people get, and which would be useless after a day. On reflection, this disappointment is obviously silly, and in particular the way this initially parsed as a thing being taken away from me is very silly.
So yeah, that's how my day is going so far. To those who got codes, thank you for not temporarily destroying a small part of the internet!
Attention LessWrong - I am a chosen user of EA Forum and I have the codes needed to destroy LessWrong. I hereby make a no first use pledge and I will not enter my codes for any reason, even if asked to do so. I also hereby pledge to second strike - if the EA Forum is taken down, I will retaliate.
Regarding your second strike pledge: it would of course be wildly disingenuous to remember Petrov's action, which was not jumping to retaliation, by doing the opposite and jumping to retaliation.
I believe you know this, and would guess that if in fact one of the sites went down, you'd do nothing but instead later post about your moral choice of not retaliating.
(I'd also guess, if you choose to respond to this comment, it'd be to reiterate the pledge to retaliate, as you've done elsewhere. This does make sense--threats must be unequivocal to be believed, even if follow through is illogical.)
Furthermore, the way for someone to test your intention to follow through or not... is to push the red button. By posting this here and on the EA forum, you may have actually increased the motivation to push the button, so that the pushee can see what you do in response.
For what it's worth, this game and the past reactions to losing it have burnt the last of my willingness to identify as a LW rationalist. Calling a website going down for a bit "destruction of real value" is technically true, but connotationally just so over the top. A website going down is just not that big a deal. I'm sorry, but it's not. Go outside or something. It will make you feel good, I promise.
Then getting upset at other people when they don't a take strange ritual as seriously as you do? As you've decided to, seemingly arbitrarily? When you've deliberately given them the means to upset you? It's tantamount to emotional blackmail. It's just obnoxious and strange behaviour.
As a trust building exercise, this reduces my confidence in the average lesswronger's ability to have perspective about how important things are, and to be responsible for their own emotional wellbeing.
I understand why this was downvoted and I think it is harsh, but I also think it might be good if people take the sentiment seriously rather than bury+ignore it.
If I received a code, I would do nothing, because it's clear by now that pressing the button would seriously upset some people. (And the consequences seem potentially more significant this year than last.) And I think the parent commenter undervalues the efforts the pro-taking-it-seriously people made to keep their emotions in check and explain why they take the ritual seriously and would like others to do so too.
But I share the instinctive reaction that the whole thing is a bit overblown and pompous, and even on reflection I think it's at least reasonable to hold that it was obnoxious to throw unconsenting people into a situation that looked like a game, where the stakes appeared (and IMO were) very low, only to reveal after the fact that playing the game -- by taking an action explicitly enabled by the people who run and probably care most about the site -- had apparently caused non-trivial distress to others and significant reputational harm to the player.
You make good points. I, for one, strong-downvoted OP because “emotional blackmail” seems not at all accurate, and the criticism itself was shaded “go outside, nerd”, when I would have been more interested in OP’s actual arguments.
Emotional blackmail would be if Ruby emailed me and said “TurnTrout, unless you participate in this ritual, I will be upset at you.” In this situation, if I do nothing, nothing happens to me, whereas Ruby may feel differently about me if I choose to participate in the game by entering launch codes.
It’s like if I built a sand castle, put some light explosives inside, and handed 100 people detonators. If someone blows it up, I could be mad at them. Sure, that might be foreseeable, and probably “my fault” in a sense.
but it seems unnatural to describe this kind of situation as “tantamount to emotional blackmail.”
I don't want to have a ton of meta discussion on the day of the experiment, but I am pretty interested in ideas from people on how to reduce the bad parts of the social ritual. I think the benefits of doing a thing like this are pretty high, and I am pretty excited about the benefits of the trust exercise, but also don't want to needlessly distress people. So if people have any ideas on wording or additional text we could add to the announcements or emails, I think that would be a productive use of time.
The obvious thing is to ask people to consent before entering the game? It's weird to get an email, out of the blue, with launch codes, telling you that you are now part of this game. While an email that spells out some of the explicit norms, and asks people to opt-in, seems great.
A light-touch intervention could just be giving people a link to click to get the launch codes, that shows some text spelling out norms like this, and ask people to only click the link if they actually want to participate.
EDIT: To be clear, I am participating in this, and would have opted-in - I just think it's a really bad norm to not ask for consent first, when we're putting people in a situation with real risks and social consequences, and with wildly differing perceptions of the depth of meaning in this event.
I was one of 270 last year and am one of 100 this year, I did not understand the context last year. Empirically, neither did Chris last year. Multiple people on the EA Forum have commented about not understanding the context
I don't know if this would defeat part of the purpose, but what about making it opt-in over a long time period, e.g. giving people all year to put themselves on the list of people who might be chosen to receive codes?
Other than that, I think it's mostly a question of (to the extent possible without undermining what you're trying to do) making it pretty clear to the recipients that people take this seriously and would genuinely like them to refrain from using the codes. As far as I can tell, that has already improved from last year. (It seems like there might have been some tonal ambiguity last year, with phrasing intended to be heightened but mostly serious coming across to some readers as playful and mostly joking.)
You're right, I haven't been active in a long time. I'm mostly a lurker on this site. That's always been partly the case, but as I mentioned, it was the last of my willingness to identify as a LWer that was burnt, not the entire thing. I was already hanging by a thread.
My last comment was a while ago, but my first comment is from 8 years ago. I've been a Lesswronger for a long time. HPMOR and the sequences were pretty profound influences on my development. I bought the sequence compendiums. I still go to local LW meetups regularly, because I have a lot of friends there.
So, you can dismiss me as some random who has just come here to hate if you want to, I guess, but I don't think that makes much sense. Definitely the fact that I was a bit obnoxious with my criticism probably makes it tempting to. You can tell I'm here in bad faith from all the downvotes, right?
I think the audience seeing this comment is heavily self selected to care about the Petrov day celebration and think it's good and important. These present core LWers risk severely underestimating how off-putting this stuff is. How many people would be interested in participating in this community, constructively, if the vibes were a little less weird. These people, unlike me, mostly don't care enough to rock up and criticize.
The reason I was rude was because I am frustrated at feeling like I have to abandon my identification as an LW rat, because I just don't want to be associated with it anymore. I got so much value from less wrong, and it feels so unnecessary.
Thank you for clarifying. I think your stance is a reasonable one, and (although I maintain that your initial comment was a poor vehicle for conveying them) I am largely sympathetic to your frustrations. Knowing that your initial comment came from a place of frustration also helps to recontextualize it, which in turn helps to look past some of the rougher wording.
Having said that: while I can't claim to speak for the mods or the admins of LW, or what they want to accomplish with the site and larger community surrounding it, I think that I personally would like to offer some further pushback. In particular, I think that there is a tension between what you term "making the vibes a little less weird", and something that I might term "being able to visibly, publicly care about things most people haven't thought about".
There is an argument, perhaps, to be had about whether the Petrov Day game is something "worth" caring about, even for a group of people with a history of caring about strange things. As I wrote in a separate comment response, I don't necessarily have a strong opinion about this. I am less involved in the "rationalist community" than many members of this site; I have not ...
I want to be clear that it's not having rituals and taking them seriously that I object to. It's sending the keys to people who may or may not care about that ritual, and then castigating them for not playing by rules that you've assigned them. They didn't ask for this.
In my opinion Chris Leong showed incredible patience in writing a thoughtful post in the face of people being upset at him for doing the wrong thing in a game he didn't ask to be involved in. If I'd been in his position I would have told the people who were upset at me that this was their own problem and they could quite frankly fuck off.
Nobody has any right to involve other people in a game like that without consulting them, given the emotional investment in this that people seem to have.
I want to be clear that it's not having rituals and taking them seriously that I object to. It's sending the keys to people who may or may not care about that ritual, and then castigating them for not playing by rules that you've assigned them. They didn't ask for this. [...]
Nobody has any right to involve other people in a game like that without consulting them, given the emotional investment in this that people seem to have.
Cool. So, on the object level, there is a discussion to be had about this... but I want to point out the extent to which, if this was your concern, your initial comment entirely failed to convey it. Not to put too fine a point on it, but there is a stark difference between what you wrote here, and and what you wrote here:
Calling a website going down for a bit "destruction of real value" is technically true, but connotationally just so over the top. A website going down is just not that big a deal. I'm sorry, but it's not. Go outside or something. It will make you feel good, I promise. [...]
...As a trust building exercise, this reduces my confidence in the average lesswronger's ability to have perspective about how important things are, and to be respon
On to the object level:
I want to be clear that it's not having rituals and taking them seriously that I object to. It's sending the keys to people who may or may not care about that ritual, and then castigating them for not playing by rules that you've assigned them. They didn't ask for this. [...]
Nobody has any right to involve other people in a game like that without consulting them, given the emotional investment in this that people seem to have.
(with the disclaimer that—again—I am not strongly invested in the Petrov Day game as practiced, nor do I have a strong opinion on whether the mods are doing it right)
I think it is an entirely reasonable thing to do, if you are attempting to establish a high-trust community, to assume a certain level of "buy-in" among core members of said community. I think one of the things that having a high-trust community gives you, is precisely the ability to coordinate actions and activities in ways more subtle and less legible than "opt-in only" (and to be clear, I view this as a positive externality; I would like more communities to have this ability!). I think, to the extent that a community is not yet at the level where [it is common know...
This is just such a bizarre tack to take. You can go down the "toughen up" route if you want to, but it's then not looking good for the people who have strong emotional reactions to people not playing along with their little game. I'm really not sure what point you're trying to make here. It seems like this is a fully general argument for treating people however the hell you want. After all, it's not worse than the vagaries of life, right? Is this really the argument you're going with, that if something is a good simulation of life, we should just unilaterally inflict it on people?
The Petrov Day event is a trivial to nonexistent burden to place on those who received the launch code. They were told the background and the launch code and told what it would do if they used it. They were not even asked to do or not do anything in particular. Similar events have been run in the past, and those selected are likely to have been around long enough to have seen at least the last such event.
The obvious way to not participate is to ignore the whole matter.
I don't think there is any violation of consent here.
I agree that life is like that. However, the game still violates consent, the same way as if I assaulted you on the street because I think it's good preparation for being assaulted "for real".
To me, this game falls in the same category as gift giving, surprise parties, pranks, rude/aggressive jokes etc. There needs to be a meta-level agreement that this kind of thing is ok, even though "being a surprise" is an essential part of the thing itself.
my willingness to identify as a LWer that was burnt [...] HPMOR and the sequences were pretty profound influences on my development [...] frustrated at feeling like I have to abandon my identifaction as an LW rat
I've struggled a lot with this, too. The thing I keep trying to remember is that identification with the social group "shouldn't matter": you can still cherish the knowledge you gained from the group's core texts, without having to be loyal to the group as a collective (as contrasted to your loyalty to individual friends).
I don't think I've been very successful; multiple years after my alienation from "the community", I'm still hanging around leaving bitter comments about how terrible they/we are. It's very dysfunctional! Why can't I just write it off as a loss and move on?
I guess the main difficulty is that, for humans, knowledge actually isn't easily separable from a community of other people with shared vocabulary. I can't just ignore what the central rat social hierarchy is doing, because the central hierarchy nodes exert a lot of control over everyone in the world who I can really talk to.
I briefly saw a "Missile Incoming" message with a 60:00 timer (that wasn't updating) on the buttons on the front pages of both LW and the EA Forum, at around 12pm EST, on mobile. Both messages were gone when I refreshed. Was this a bug or were they testing the functionality, testing us or preparing to test us?
Typically the mods put out a Petrov Day Retrospective the day after; however some of this year's organizers are now away at a conference, so we're going to wait a few days until they're so they can have input before we publish.
Sorry for the delay. Stay tuned.
(And thanks to everyone who participated and/or weighed in on the discussion.)
An attempted non-mystical justification for Petrov day sensitivity, for those who think it's ridiculous:
If the LW home page were 'nuked', my day would be slightly worse: there would be interesting posts and comments I wouldn't as easily find out about (e.g. this one by Katja about literal and metaphorical fire alarms). So it makes sense for me to feel a bit bummed if it gets taken down. In addition, if someone else takes the page down, I should feel more bummed: not only did this slightly bad thing happen, but I just learned that someone will make my life ...
As a counter-point, my day was made significantly better by the front page being nuked in 2020 - it was exciting, novel, hilarious (by my lights - clearly not to some people), made some excellent points about phishing and security, and gave me opportunities to dissect why people oriented to this event differently from me. I expect my experience would have been less good last year had the phishing attempt not happened, and we all simply coordinated. More generally, when a website does something unusual and novel like this, I feel like the value of novelty and interestingness can outweigh the costs of a single day of disrupted use?
I'd further argue that the people highly invested in this seem much more invested in the abstract ideas of trust, community, shared ritual and cohesion, more so than the object level of the frontpage being down (besides, people can always use greaterwrong.com )
And you can also play on hard mode: "During said ceremony, unveil a large red button. If anybody presses the button, the ceremony is over. Go home. Do not speak."
This has been a common practice at Petrov Day celebrations in Oxford, Boston, Berkeley, New York, and in other rationalist communities. It is often done with pairs of celebrations, each whose red button (when pressed) brings an end to the partner celebration.
First time I heard this idea and I really like it. I would like to organize something like it next locally for my friends, hopefully I remember.
Grammar: I think there's a few words missing in:
there is real value on the line here and this is a real trust-building exercise that [is not?] undertaken lightly by either LessWrong or the EA Forum.
I have pushed the button.
I do not have launch codes.
I did not attempt to launch nukes.
I pushed the button to see if there was any exploitable way to brute-force nuke codes or otherwise determine a valid code.
I got bored and gave up after seeing the launch code textbox had no ID.
I don't know what you should take away from this. Beware chaos, I guess, and those who think it would be funny to do damage or try to find flaws in a system. To many, taking down a website for a day would be funny. Don't even show buttons to people unless you trust them with the responsibility and know them unreasonably well.
Social deduction games
I saw the button, moused over it and got the "Are you sure?" popup, then carefully refrained from pushing the button. (I do not have launch codes.) As Batman says after destroying a supposed magical idol that probably does not actually summon demons, some things you don't take chances with.
It means that you don't need to have the other person's login credentials to launch the nukes (I don't want to encourage password theft, and also think that the case of someone sharing just their codes is more interesting than someone sharing full access to their account). It also creates common-knowledge of what is happening on the site, in a pretty clear and obvious way.
Why only 100 members? Shouldn't we want to be able to trust our entire community?
Why not give codes to everyone or everyone who'd registered before X date with X karma to avoid trolls?
More trustworthy people is good but it doesn't follow that acting as if more people are trustworthy is good.
I expect the button can be pressed without lunching nukes (though I didn't check, won't check, and encourage others not to check - even if admins confirm this to be the case) because launch codes are required. My question is, do you track how many times the button gets clicked without someone entering a code?
I'm glad one apparently can't take down the site merely by pressing the Big Red Button. I didn't press it, but I have tons of muscle memory on sites like reddit of opening a gazillion links in new tabs, so I wouldn't have been surprised if I'd clicked it before even noticing what it was about. Which is a far cry from intentionally pressing the button, of course, but still.
Pictured Right: Enter launch codes to destroy LessWrong
Petrov Day
Today we celebrate not destroying the world. We do so today because 38 years ago, Stanislav Petrov made a decision that averted tremendous calamity. It's possible that an all-out nuclear exchange between the US and USSR would not have actually destroyed the world, but there are few things with an equal chance of doing so.
For more information see 1983 Soviet nuclear false alarm incident
Petrov is not alone in having made decisions that averted destruction–presidents, generals, commanders of nuclear submarines, and similar also made brave and fortunate calls–but Petrov's story is salient, so today we celebrate him and all those who chose equally well.
As the world progresses, likely many more people will face decisions like Petrov's. Let's celebrate in advance that they'll make good decisions! And if we expect to face decisions ourselves, let us resolve to decide wisely!
Mutually Assured Destruction (??)
The Petrov Day tradition is to celebrate Petrov's decisions and also to practice not destroying things, even when it's tempting.
In both 2019 and 2020, LessWrong placed a large red button on the frontpage and distributed "launch codes" to a few hundred "trustworthy" people. A launch would bring down the frontpage for the duration of Petrov Day, denying hundreds to thousands of people access to LessWrong. In 2019, all was fine. In 2020...let's just say some bad decisions were made.
Yet having a button on your own page that brings down your own site doesn't make much sense! Why would you have nukes pointed at yourself? It's also not very analogous to the cold war nuclear scenario between major world powers.
For those reasons, in 2021, LessWrong is teaming up with the Effective Altruism Forum to play a little game of mutual destruction. Two buttons, two sets of codes, and two sets of hopefully trustworthy users.
If LessWrong has trusted launch code recipients poorly, the EA Forum will go down, and vice versa. One of the sites going down means hundreds to thousands of people being denied to important resources: the destruction of significant real value. What's more it will damage trust between the two sites ("I guess your most trusted users couldn't be trusted to not take down our site") and also for each site itself ("I guess the admins couldn't find a hundred people who could be trusted".)
For exact rules of the game, see the final section below.
Last year it emerged that there was ambiguity about how serious the Petrov Day exercise was. I'll be clear as I can via text: there is real value on the line here and this is a real trust-building exercise that is not undertaken lightly by either LessWrong or the EA Forum. Both sites have chosen recipients who we believe we can trust to not destroy each others' communal resources.
How Do I Celebrate?
If you were one of the two hundred people to receive launch codes for LessWrong or the EA Forum, celebrate by doing nothing!
Other ways of celebrating:
Rules of the Exercise
The following email was sent last night to 100 users from LessWrong and 100 from the EA Forum:
To all, I wish you a safe and stable Petrov Day.
Here is the mirror of this post on the EA Forum. You may wish to view it for the discussion there.