Your Substack subtitle is "I won't get to raise a family because of AGI". It should instead be "I don't want to raise a family because of AGI" I think it's >90% likely that if you want and try to, you can raise a family in a relatively normal way (i.e. your wife gives birth to your biological children and you both look after them until they are adults) in your lifetime.  Not wanting to do this because those children will live in a world dissimilar to today's is another matter, but note that your parents also raised you to live in a world very dissimilar from the world they grew up in, but were motivated to do it anyway! So far, over many generations, people have been motivated to build families not by the confidence that their children will live in the same way as they did, but rather by other drives (whether it's a drive towards reproduction, love, curiosity, norm-following, etc.). I also think you're very overconfident about superintelligence appearing in our lifetimes, and X-risk being high, but I don't see why either of those things stop you from having a family. 

Completely agree with your observations, and I say this as someone who (a) grew up with family singing, campfire singing (b) had pretty extensive choral training; (c) nevertheless later participated in and led community singing groups built on the idea that it's totally, absolutely fine to sing "badly"; (d) for many years hosted a successful wassailing party at which most people were not Christian and not familiar with caroling, yet happily and credibly belted out the Christmas carols they had just learned, on the porches of surprised neighbors; (e) is now a folkie very active in the pub singing tradition (f) with an armchair interest in ethnomusicology, oral tradition, and the neurobiosociology of community singing. SO... a few thoughts: You are absolutely right that the most important factor is giving people permission to sing, that everyone has the right to sing, your voice doesn't have to be good, you don't have to be in tune, and in fact it will be fine. If there are professional musicians present it can be important to explain to them what is going on, why they should be happy to hear bad singers sing, and how they can help by singing the melody loudly, and not wincing. You may find helpful resources at https://singout.org/communitysings/, look up Pete Seeger's Tone Deaf Choir (historical), Matt Watroba's Community Sings (current, I think). They were brilliant at getting big crowds of non-singers to sing (and surprise themselves with how good they sound). I'd be happy to exchange notes on repertoire. I am not sure what themes exactly suit the Rationalist Solstice scene, but I know what worked well for my motley wassailing crew and my community singing group. There are characteristics of songs from oral tradition that support/encourage everyone to sing, which are common in older traditional songs, religious/church songs, childrens/camp songs. Yes, they arose from contexts where the participants could not read, but this is irrelevant. You do not want people reading words off of a sheet of paper or their phone. You want them to be present to the room and just sing. You want it to be easy to sing in the pitch dark with a candle in one hand and a glass of grog in the other. So the same rules apply.  Predictability. The tune is repetitive (no modulations, bridges, etc), and the lyrics have a formulaic pattern. A good example: "Where have all the flowers gone?". If someone has never heard the song, they have to stop singing and listen to hear the one word that is new in each verse; but then they can predict how the entire next verse will go, and can sing along to the whole verse as well as the chorus. And that song is not silly, and not a bad candidate.  Repetition. Songs with a chorus, as noted, you can sing the chorus once at the outset, and then people can sing it every time it repeats. Pro tip: sing every verse, and repeat the chorus after every verse. Stage folk performers will skip verses and only sometimes sing the chorus to avoid boring the audience; but it's not boring when there is no audience and everyone is singing.  But choruses aren't the only form of this. Some songs have call and response where you repeat each line (or there's a formula for the response to the called line). Some have a refrain in which the last line or two of each verse is sung again. Look for songs with these features. Familiarity. Obvious one. If a lot of people recognize a song it helps, even if they just hum the tune. A good example of this might be Silent Night. In  traditional music, including church hymnals and pub ballads, tunes are heavily re-used. The same tunes are re-used for many different sets of lyrics, so you can leverage that everyone already knows the tune. Parodies (writing new songs to well known tunes) work well for this reason. Physicality. Clapping, stomping, snapping, whatever, lets people participate even if they don't know the words or tune -- and surprisingly lowers inhibitions for singing. For totally novel songs I think the best you can do is have an optional pre-run for people who want to learn them, use the same ones year after year, and make it ok to just listen and enjoy the ones you don't know.  People pick up songs up with remarkable ease. They will accidentally find they are singing it next time.  In this context, story songs are the easiest for people to remember; our brains are wired for stories. For example 'Good King Wenceslas' was the favorite most belted out at my Wassail, even though there are no choruses or refrains.  Since you are taking a long term view on this: wanna co-org a workshop on "singing for people who can't sing" at LessOnline next year? 

I would term □x→x "hope for x" rather than "reliability", because it's about willingness to enact x in response to belief in x, but if x is no good, you shouldn't do that. Indeed, for bad x, having the property of □x→x is harmful fatalism, following along with destiny rather than choosing it. In those cases, you might want to □x→¬x or something, though that only prevents x from being believed, that you won't need to face □x in actuality, it doesn't prevent the actual x. So □x→x reflects a value judgement about x reflected in agent's policy, something downstream of endorsement of x, a law of how the content of the world behaves according to an embedded agent's will. Payor's Lemma then talks about belief in hope □(□x→x), that is hope itself is exogenous and needs to be judged (endorsed or not). Which is reasonable for games, since what the coalition might hope for is not anyone's individual choice, the details of this hope couldn't have been hardcoded in any agent a priori and need to be negotiated during a decision that forms the coalition. A functional coalition should be willing to act on its own hope (which is again something we need to check for a new coalition, that might've already been the case for a singular agent), that is we need to check that □(□x→x) is sufficient to motivate the coalition to actually x. This is again a value judgement about whether this coalition's tentative aspirations, being a vehicle for hope that x, are actually endorsed by it. Thus I'd term □(□x→x) "coordination" rather than "trust", the fact that this particular coalition would tentatively intend to coordinate on a hope for x. Hope □x→x is a value judgement about x, and in this case it's the coalition's hope, rather any one agent's hope, and the coalition is a temporary nascent agency thing that doesn't necessarily know what it wants yet. The coalition asks: "If we find ourselves hoping for x together, will we act on it?" So we start with coordination about hope, seeing if this particular hope wants to settle as the coalition's actual values, and judging if it should by enacting x if at least coordination on this particular hope is reached, which should happen only if x is a good thing. (One intuition pump with some limitations outside the provability formalism is treating □x as "probably x", perhaps according to what some prediction market tells you. If "probably x" is enough to prompt you to enact x, that's some kind of endorsement, and it's a push towards increasing the equilibrium-on-reflection value of probability of x, pushing "probably x" closer to reality. But if x is terrible, then enacting it in response to its high probability is following along with self-fulfilling doom, rather doing what you can to push the equilibrium away from it.) Löb's Theorem then says that if we merely endorse a belief by enacting the believed outcome, this is sufficient for the outcome to actually happen, a priori and without that belief yet being in evidence. And Payor's Lemma says that if we merely endorse a coalition's coordinated hope by enacting the hoped-for outcome, this is sufficient for the outcome to actually happen, a priori and without the coordination around that hope yet being in evidence. The use of Löb's Theorem or Payor's Lemma is that the condition (belief in x, or coordination around hope for x) should help in making the endorsement, that is it should be easier to decide to x if you already believe that x, or if you already believe that your coalition is hoping for x. For coordination, this is important because every agent can only unilaterally enact its own part in the joint policy, so it does need some kind of premise about the coalition's nature (in this case, about the coalition's tentative hope for what it aims to achieve) in order to endorse playing its part in the coalition's joint policy. It's easier to decide to sign an assurance contract than to unconditionally donate to a project, and the role of Payor's Lemma is to say that if everyone does sign the assurance contract, then the project will in fact get funded sufficiently.