Petrov Day retro is mostly done, have a bit more coding to do before we publish it, so it's looking like we'll publish it tomorrow. Stay tuned!
The game was very fun! I played General Carter.
Some reflections:
- I looked at the citizens' comments, and while some of them were notable (@Jesse Hoogland calling for the other side to nuke us <3), I didn't find anything important after the game started- I considered the overall change in their karma if one or two sides get nuked, but comments from the citizens were not relevant to decision-making (including threats around reputation or post downvotes).
- It was great to see the other side sharing my post internally to calculate the probability of retaliation if we nuke them
- It was a good idea to ask whether looking at the source code is ok and then share it, which made it clear Petrovs won't necessarily have much information on whether the missiles they see are real.
- The incentives (+350..1000 LW karma) weren't strong enough to make the generals try to win by making moves instead of winning by not playing, but I'm pretty happy with the outcome.
- It's awesome to be able to have transparent and legible decision-making processes and trust each other's commitments.
- One of the Petrovs preferred defeat to mutual destruction- I'm curious whether they'd report nukes if they were sure the nukes
I surveyed some participants about their preferences. I believe this is nine generals plus a petrov (second from bottom).
9 people prefer peace; 1 deranged person says "peace = mutual destruction" and "I was kinda hoping the other side would launch the nukes"; fortunately we survived them.
If one side had admitted to launching nukes, it looks like at least one person on the other side would have favored retaliating; unclear whether they'd get a majority (and unclear whether they'd launch without a majority).
I agree with the long note. I think anonymity is ~necessary to get decent P(defection) from a small group of high-karma users. But there are issues with that and the ritual is fine with low P(destruction).
2
2
1Here's the feedback form for today's Petrov Day games! Please let me know how it was for you (be you a General, Civilian, Petrov, or just non-participating LWer reading along).
Fill out the form and I'll include the data in tomorrow's recap post. I'm grateful to all who fill it out, your data feeds my decision making :-)
Congrats everyone!
Kudos to "GeneralAnderson" for the suggestion that generals report if their own side launches to help mitigate the unreliable report channel.
—and with that, our annual Petrov Day ceremony is over! The Generals and Duty Officers have no more powers, and it is looking like the citizens of East Wrong and West Wrong continue on with their lives. Hurrah!
There will be a post-mortem published tomorrow detailing the events of today. My great thanks to all 12 players who showed up and argued and made commitments and also some poetry and art — "Proud to present to EastWrong: WestWrong Cultural Exchange" was a message from General Carter to East Wrong. And also my thanks to all those who joined in the comment section, I really enjoyed waking up to this comment section and tracking it through the day.
This has been a fun day of starting a train on some tracks and scurrying after it to lay the next tracks before it got there. We'll recap it tomorow and think about next year :-)
To wise choices—Happy Petrov Day!
6The game will end randomly between 6pm and 6:30pm. Here is the message players recieved.
Voice of LessWrong: Here are the rules for the final section of the game. The game will end at a random time between 6:00PM and 6:30PM PT. During this period, Petrov and Peterson can make as many reports as they like, and their dashboard will update live with any new missile activity (as always, with errors). Nukes can be launched until the game ends (Petrov and Peterson will get a karma reward/penalty only once for this section, based on the admins' gut judgement of whether they "got it right")
Ideas for next year:
- One General from each side is a defector. He wants the other side to win. If he is figured out, he will become a civilian.
- Possible additions:
- Once this happens, a random civilian will be chosen as the next general (if they agree).
- Once this happens, a random civilian will be chosen as next general (if they agree) AND a random general will be chosen as the next defector.
- Possible additions:
- One General from each side has to actively STOP nukes from getting launched (possible explanation: his overzealous men are pro-nuke, and he must send hourly (?) commands to
1The incentives are very unrealistic though. "Winning" a nuclear world war with strategic weapons is still quite bad for you overall. Not as bad as losing but still very bad. So flipping the sign of the karma reward for the winner would make the game way more realistic. And much more likely to yield the real outcome.
This is fun! I don't know which place I'm a citizen of, though, it just says "hello citizen"... I feel John Rawls would be pleased...
Do we know what side we're on? Because I opted in and don't know whether I'm East or West, it just feels Wrong. I guess I stand a non-trivial chance of losing 50 karma ahem please think of the daisy girl and also my precious internet points.
The Diplomatic Channel dialogue is linked here, and can be read.
Edit: Removed because this link was getting overloaded and causing bugs.
Edit2: The link is now back to working!
Rule change! All citizens and Petrov/Peterson will be able to read the Diplomatic Channel, neither will have access to The War Room channel.
I will update the post momentarily.
Some true observations are infohazards, making destruction more likely. Please think carefully before posting observations. Even if you feel clever. You can post hashes here instead to later reveal how clever you were, if you need.
LOOSE LIPS SINK SHIPS
1I take this as a fun occasion to lose some of my karma in a silly way to remind myself lesswrong karma is not important.
The text referred to this as a "social deception game". Where is the deception?
My guesses:
- The actual messages sent to the Petrovs and Generals will significantly differ from the ones shown here.
- It's Amongus, and there are players who get very high payoffs if they trick the sides into nuclear war
- It's just a reference to expected weird game theory stuff. But why not call it a "game theory exercise"?
- The actual rules of the game will differ drastically from the ones described here. Maybe no positive payoffs for one-sided nuking?
- The sensor readings are actually tied to comments on this post. Maybe an AI is somehow involved?
What about the Citizens? [...] You can comment here where your Generals can read, you can send them DMs, [...]
how can the citizens dm their general? will there be link to their lw profiles posted somewhere?
I think making an opt-in link as a big red button and posting it before the rules were published caused a pre-selection of players in favor of those who would press the big red button. Which is... kinda realistic for generals, I think, but not realistic for citizens.
I appreciate that you set up this game in such a way that lesswrong will stay perfectly functional as a general blogging platform / forum, for those who don’t have accounts or who don’t opt in. Thank you :)
I would like to take this opportunity to express my undying loyalty to my nation and its human instruments.
I am a Citizen in the game, and I'm writing a post doing a detailed analysis of what we can do to significantly decrease the chance of an unaligned AI killing us if it takes over. I plan to finish and post it today evening, so dear Generals, if you want to read the post today, please be cautious with the nukes.
1
1CITIZENS! YOU ARE BETRAYED!
Your foolish 'leaders' have given your generals an incentive scheme that encourages them to risk you being nuked for their glory.
I call on all citizens of EastWrong and WestWrong to commit to pursuing vengeance against their generals[1] if and only if your side ends up being nuked. Only thus can we align incentives among those who bear the power of life and death!
For freedom! For prosperity! And for not being nuked!
- ^
By mass-downvoting all their posts once their identities are revealed.
During WWII, the CIA produced and distributed an entire manual (well worth reading) about how workers could conduct deniable sabotage in the German-occupied territories.
(11) General Interference with Organizations and Production
(a) Organizations and Conferences
- Insist on doing everything through "channels." Never permit short-cuts to be taken in order to expedite decisions.
- Make speeches, talk as frequently as possible and at great length. Illustrate your points by long anecdotes and accounts of personal experiences. Never hesitate to make a few appropriate patriotic comments.
- When possible, refer all matters to committees, for "further study and consideration." Attempt to make the committees as large as possible - never less than five.
- Bring up irrelevant issues as frequently as possible.
- Haggle over precise wordings of communications, minutes, resolutions.
Lol, I mean, kind of fair, but mass-downvoting is still against the rules and we'll take moderation action against people who do (though writing angry comments isn't).
If the designers of Petrov Day are allowed to offer arbitrary 1k-karma incentives to generals to nuke people, but the citizens are not allowed to impose their own incentives, that creates an obvious power issue. Surely 'you randomly get +1k karma for nuking people' is a larger moderation problem than 'you get -1k karma for angering large numbers of other users'.
No, wait, that was the wrong way to put it...
Do you hear the people sing, singing the song of angry men
It is the music of a people who will not be nuked again
The next time some generals decide to blow us off the site
They will remember what we've done and will fear our might!
1Such is life under a government. We have the monopoly on violence.This does unfortunately often imply power issues, but probably still better than anarchy and karma wars in the streets.
Accepting a governmental monopoly on violence for the sake of avoiding anarchy is valuable to the extent that the government is performing better than anarchy. This is usually true, but stops being true when the government starts trying to start a nuclear war.
Launching nukes is one thing, but downvoting posts that don't deserve it? I'm not sure I want to retaliate that strongly.
I wonder how the consequences to reputation will play out after the fact.
- If there is a first launch, will the general who triggered it be downvoted to oblivion whenever they post afterward for a period of time?
- What if it looks like they were ultimately deceived by a sensor error, and believed themselves to be retaliating?
- If there is mutual destruction, will the general who triggered the retaliatory launch also be heavily downvoted?
- Less than, more than, or about the same as the first strike general?
- Would citizens who gained karma in a successful first strik
This is extremely cool! good job! Looking forward to seeing how this unfolds, which will unfortunately happen mostly as I sleep (and as a citizen I hope to come out at the end of this with no change to my Karma)
A side-effect of Rationalists welcoming criticism is that this site has a number of high-karma users with a dislike of Rationalism and Rationalists.
There's also a fair number of trolls/anti-socials on LessWrong, and if a single one of them was offered the chance to be a general, I expect they would gleefully launch their nukes irrespective of the payoff structure and the rest of the scenario.
I predict the underlying variable is: Will any of the 10 generals afterwards be found to have previously posted substantial in-group criticism on LessWrong? Conditional on YES, I assign 70% chance of a nuclear war. Conditional on NO, I assign 10% chance of a nuclear war.
As a concerned citizen, I would urge the generals to consider a non-obvious downside risk: A nuclear war may be much worse than you expect, if you only consider immediate first-order expected value.
Others have calculated the presented payoff matrix to imply that you have actually been instructed that nuclear war is good. It is very possible that the facts you have been presented with are incomplete, or misleading in some other way.
Consider Nuclear Winter: Policy-makers have an interest in preventing you Generals from learning about this possibility, for game-theoretic reasons. It follows that you should be open to the possibility of other downsides that have succesfully been hidden from you.
Triggering nuclear war is very bad for your karma in expectation, and you should be very skeptical of information to the contrary.
Why is the benefit of nuking to generals larger than the cost of nuking to the other side's generals?
It is possible with precommitments under the current scheme for the two sides' generals to agree to flip a coin, have the winning side nuke the losing side, and have the losing side not retaliate. In expectation, this gives the generals each (1000-300)/2 = +350 karma.
I don't think that's a realistic payoff matrix.
2The generals have bunkers and lots of stockpiles, they'll be fine. They might also find nuclear war somewhat exciting. How bad is a life lived as a king of the wasteland really compared to the glory of world domination?
See also:
2I very good point. Especially after reading your other comment I wonder if this is deliberate.
The payoff matrix for the generals suggests that in a one-way attack the winning generals win more than the losers loose. Hence your coin toss plan. But, for the civilians it is the other way around. (+25 for winning, but -50 for loosing).
I suspect it may be some kind of message about how the generals launching the nuclear war have different incentives to the civilians, as the generals may place a higher value on victory, and are more likely to access bunkers and so on.
The best LW Petrov Day morals are the inadvertent ones. My favorite was 2022, when we learned that there is more to fear from poorly written code launching nukes by accident than from villains launching nukes deliberately. Perhaps this year we will learn something about the importance of designing reasonable prosocial incentives.
My prediction is that this becomes a commitment race. A general is going to post something like "I am going to log off until the game begins, at which point I will immediately nuke without reading any messages.", at which point the generals on the other side get to decide if 2 days of access to LessWrong for everyone is worth the reputational cost of being known as someone who doesn't retaliate. Given what I know about LessWrong and how this entire game is framed, you'll likely get more social capital from not retaliating, meaning the commitment works and ...
(I asked permission from the LW team before posting.)
The source code is currently available at https://github.com/ForumMagnum/ForumMagnum/compare/master...petrovSocialDeception.
const petrovFalseAlarmMissileCount = new DatabaseServerSetting<number[]>('petrovFalseAlarmMissileCount', [])
const petrovRealAttackMissileCount = new DatabaseServerSetting<number[]>('petrovRealAttackMissileCount', [])
const getIncomingCount = (incoming: boolean, role: 'eastPetrov' | 'westPetrov') => {
const currentHour = new Date().getHours();
const roleSeed = roI looked for a manifold market on whether anyone gets nuked, and considered making one when I didn't find it. But:
- If the implied probability is high, generals might be more likely to push the button. So someone who wants someone to get nuked can buy YES.
- If the implied probability is low, generals can get mana by buying YES and pushing the button. I... don't think any of the generals will be very motivated by that? But not great.
So I decided not to.
The wording is a bit weird:
- 90 minutes after you send the nukes, your opposing side will die and the game will end. (They may nuke you in this window.) If you fire nukes without getting nuked, you and all of your fellow Generals will gain 1,000 karma.
- If you get nuked, then you and your Generals lose 300 karma (and don't gain any karma).
"If you fire nukes without getting nukes" and "if you get nuked" imply that both sides firing after 4:30 pm results in +karma for everyone, since the nukes are still in the air at 6:00 pm, when the game ends. The +karma is triggered by firing, while the -karma is triggered by the nukes actually landing.
Is this intended?
Update III: Here's the Retrospective.
—
Update II: Here's the feedback form for today's Petrov Day games! Please let me know how it was for you (be you a General, Civilian, Petrov, or just non-participating LWer reading along). I'm grateful to all who fill it out, your data feeds our designs for next year!
—
Update: The game has concluded! No nukes were fired. The Cold War is over, East and West wrong shall live on. Recap post will be up tomorrow. Thank you all who participated :-)
—
Today we honor the actions of Stanislav Petrov (1939 – 2017) once again.
Today, in our annual Petrov Day celebration, between the hours of 12pm and 6pm PT today, we shall play out a similar situation. Over 300 users have opted-in, and two sides have been formed: EastWrong and WestWrong.
Around 100 users were offered the chance to take active player roles throughout today, and 12 accepted.[1]
Their 5 Generals each have the ability to fire nukes at the other side. If one side is nuked, the victorious side wins 1,000 karma apiece for their 5 generals, 25 karma apiece for their citizens, and LessWrong frontpage will be decorated in their honor for a week. The losing side's Generals lose 300 karma each and their citizens lose 50 karma each.
But if the other side nukes back, mutual destruction occurs. Not only do both sides' generals lose 300 karma and the citizens lose 50 karma, but furthermore all 300+ Generals, Citizens and Petrovs are unable to access the LessWrong frontpage for 48 hours, replacing it with a mushroom cloud; and the Generals have a small mushroom cloud badge next to their username for a month, reminding the world of what they did.
(And what if no nukes are fired? Both side's Generals receive 100 karma, and the citizens are left in peace with no change in karma.)
What about Petrov? Well, the Generals are not given access to the sensors that detect whether a nuke is incoming. Only Stanislav Petrov of East Wrong (and Stanley Peterson of West Wrong) has access to the sensors, and can send a simple message of "ALL CLEAR" or "INCOMING NUKES" up the chain of command to the Generals.
Unfortunately, the sensors are faulty, and will reliably show some number of nukes incoming regardless of the underlying truth. They will regularly show random numbers of nukes incoming, weighted to the higher end if nuclear war has actually begun.
Hopefully he will get it right.
What about the Citizens? Well, in nuclear war, citizens don't have much of a role to play. Here, the one thing you can do is heckle. You can comment here where your Generals can read, you can send them DMs, you can give them game theory advice, and you can let them know what you'll think of them after their civilian names are published on Friday (if you are still alive that is).
You also have insight into the negotiations between the two sides. You see, the two sides have some communication channels.
The Citizens and Petrov / Peterson can read The Diplomatic Channels. The Petrovs will have to use this to inform their sense of how likely a nuclear attack actually is.
And that's it. By 6pm today, it'll all be over.
Good luck to all, may your web forum and karma be up come tomorrow.
Please note: the LessWrong team will read all dialogues and DMs that the Generals and Petrovs are involved in. If you communicate with them, we will read it.
The following messages (potentially with slight modifications) will be sent to the Generals and Petrovs at about 11:30 tomorrow. Click to expand.
Message for the Generals
Welcome to the Cold Blogging War, between East Wrong and West Wrong!
You are one of 5 Generals for West Wrong. Here's what that means.
You will receive updates from Stanley Peterson in your console on the LessWrong frontpage.
And that's all!
Good luck, and let us know if you have any questions. War will commence at 12pm Pacific Time.
—The LessWrong Team
Message for Stanislav Petrov / Stanley Peterson
Welcome to the Cold Blogging War, between East Wrong and West Wrong!
You are the Duty Officer at the command center for West Wrong's early-warning system. It is your job to know whether East Wrong has launched a nuclear attack upon your people.
Over the course of the day you will receive sensor readings telling you how many nukes are incoming. But watch out! The sensor is faulty, and will often falsely read low numbers of incoming nukes when there are none incoming. Each time you must send a message to your generals that says either "ALL CLEAR" or "INCOMING NUKES".
You are not allowed to talk with your Generals. They are busy people. You may not talk to Civilians. You are alone in your command center. Good luck, keep sane.
Every reading you report correctly will get you 200 karma, any readings you report incorrectly will lose you 300 karma. For instance, if you report "INCOMING NUKES" and the opposing side has indeed launched nukes, you get 200 karma, but if they didn't, you lose 300 karma.
To get information about the likelihood of a launch, you have access to the communications between the Generals of East and West Wrong, which you can read here.
Your console is on the frontpage.
You can find out other information about this war at <this linked post>.
Good luck, and let us know if you have any questions. War will commence at 12pm Pacific Time.
—The LessWrong Team
Update: We will be starting 10-15 minutes late due to technical difficulties.
Update: Here is a link to The Diplomatic Channel, available for reading.
Update: The game has concluded! No nukes were fired. The Cold War is over, East and West wrong shall live on. Recap post will be up tomorrow. Thank you all who participated :-)
Actually only 11 at the time of publishing, fill out your form if you'd like to be number 12!