I would term □x→x "hope for x" rather than "reliability", because it's about willingness to enact x in response to belief in x, but if x is no good, you shouldn't do that. Indeed, for bad x, having the property of □x→x is harmful fatalism, following along with destiny rather than choosing it. In those cases, you might want to □x→¬x or something, though that only prevents x from being believed, that you won't need to face □x in actuality, it doesn't prevent the actual x. So □x→x reflects a value judgement about x reflected in agent's policy, something downstream of endorsement of x, a law of how the content of the world behaves according to an embedded agent's will. Payor's Lemma then talks about belief in hope □(□x→x), that is hope itself is exogenous and needs to be judged (endorsed or not). Which is reasonable for games, since what the coalition might hope for is not anyone's individual choice, the details of this hope couldn't have been hardcoded in any agent a priori and need to be negotiated during a decision that forms the coalition. A functional coalition should be willing to act on its own hope (which is again something we need to check for a new coalition, that might've already been the case for a singular agent), that is we need to check that □(□x→x) is sufficient to motivate the coalition to actually x. This is again a value judgement about whether this coalition's tentative aspirations, being a vehicle for hope that x, are actually endorsed by it. Thus I'd term □(□x→x) "coordination" rather than "trust", the fact that this particular coalition would tentatively intend to coordinate on a hope for x. Hope □x→x is a value judgement about x, and in this case it's the coalition's hope, rather any one agent's hope, and the coalition is a temporary nascent agency thing that doesn't necessarily know what it wants yet. The coalition asks: "If we find ourselves hoping for x together, will we act on it?" So we start with coordination about hope, seeing if this particular hope wants to settle as the coalition's actual values, and judging if it should by enacting x if at least coordination on this particular hope is reached, which should happen only if x is a good thing. (One intuition pump with some limitations outside the provability formalism is treating □x as "probably x", perhaps according to what some prediction market tells you. If "probably x" is enough to prompt you to enact x, that's some kind of endorsement, and it's a push towards increasing the equilibrium-on-reflection value of probability of x, pushing "probably x" closer to reality. But if x is terrible, then enacting it in response to its high probability is following along with self-fulfilling doom, rather doing what you can to push the equilibrium away from it.) Löb's Theorem then says that if we merely endorse a belief by enacting the believed outcome, this is sufficient for the outcome to actually happen, a priori and without that belief yet being in evidence. And Payor's Lemma says that if we merely endorse a coalition's coordinated hope by enacting the hoped-for outcome, this is sufficient for the outcome to actually happen, a priori and without the coordination around that hope yet being in evidence. The use of Löb's Theorem or Payor's Lemma is that the condition (belief in x, or coordination around hope for x) should help in making the endorsement, that is it should be easier to decide to x if you already believe that x, or if you already believe that your coalition is hoping for x. For coordination, this is important because every agent can only unilaterally enact its own part in the joint policy, so it does need some kind of premise about the coalition's nature (in this case, about the coalition's tentative hope for what it aims to achieve) in order to endorse playing its part in the coalition's joint policy. It's easier to decide to sign an assurance contract than to unconditionally donate to a project, and the role of Payor's Lemma is to say that if everyone does sign the assurance contract, then the project will in fact get funded sufficiently.

I think this part of Heroic Responsibility isn't too surprising/novel to people. Obviously the business owner has responsibility for the business. The part that's novel is more like: If I'm some guy working in legal, and I notice this hot potato going around, and it's explicitly not my job to deal with it, I might nonetheless say "ugh, the CEO is too busy to deal with this today and it's not anyone else's job. I will deal with it." Then you go to each department head, even if you're not even a department head you're a lowly intern (say), and say "guys, I think we need to decide who's going to deal with this." And if their ego won't let them take advice from an intern, you might also take it as your responsibility to figure out how to navigate their ego – maybe by making them feel like it was their own idea, or by threatening to escalate to the CEO if they don't get to it themselves, or by appealing to their sense of duty. A great example of this, staying with them realm of "random Bureaucracy", I got from @Elizabeth: E. D. Morel was a random bureaucrat at a shipping company in 1891. He noticed that his company was shipping guns and manacles into the Congo, and shipping rubber and other resources back out to Britain. It was not Morel's job to notice that this was a bit weird. It was not Morel's job to notice that that weirdness was a clue, and look into those clues. And then find out that what was happening was, weapons were being sent to the Congo to forcibly steal resources at gunpoint. It was not his job to make it his mission to raise awareness of the Congo abuses and stop them. But he did. ... P.S. A failure mode of rationalists is to try to take Heroic responsibility for everything, esp. in a sort of angsty way that is counterproductive and exhausting. It's also a failure mode to act as if only you can possibly take Heroic responsibility, rather than trying to model the ecosystem around you and the other actors (some of whom might be Live Players who are also taking Heroic Responsibility, some of whom might be sort of local actors following normal incentives but are still, like, part of the solution) There is nuance to when and how to do Heroic Responsibility well.

Hm, I am unsure how much to believe this, even though my intuitions go the same way as yours. As a correlational datapoint, I tracked my success from cold approach and the time I've spent meditating (including a 2-month period of usually ~2 hours of meditation/day), and don't see any measurable improvement in my success rate from cold approach: (Note that the linked analysis also includes a linear regression of slope -6.35e-08, but with p=0.936, so could be random.) In cases where meditation does stuff to your vibe-reading of other people, I would guess that I'd approach women who are more open to being approached. I haven't dug deeper into my fairly rich data on this, and the data doesn't include much post-retreat approaches, but I still find the data I currently have instructive. I wish more people tracked and analyzed this kind of data, but I seem alone in this so far. I do feel some annoyance at everyone (the, ah, "cool people"?) in this area making big claims (and sometimes money off of those claims) without even trying to track any data and analyze it, leaving it basically to me to scramble together some DataFrames and effect sizes next to my dayjob.[1] > So start meditating for an hour a day for 3 months using the mind illuminated as an experiment (getting some of the cool skills mentioned in Kaj Sotala's sequence?) and see what happens? Do you have any concrete measurable predictions for what would happen in that case? ---------------------------------------- 1. I often wonder if empiricism is just incredibly unintuitive for humans in general, and experimentation and measurement even more so. Outside the laboratory very few people do it, and see e.g. Aristotle's claims about the number of women's teeth or his theory of ballistics, which went un(con)tested for almost 2000 years? What is going on here? Is empiricism really that hard? Is it about what people bother to look at? Is making shit up just so much easier so that everyone keeps in that mode, which is a stable equilibrium? ↩︎