Ryo

Cooperation is optimal, with weaker agents too - tldr

It's a ‘superrational’ extension of the proven optimality of cooperation in game theory + Taking into account asymmetries of power // Still AI risk is very real Short version of an already skimmed 12min post 29min version here For rational agents (long-term) at all scale (human, AGI, ASI…) In real contexts, with open environments (world, universe), there is always a risk to meet someone/something stronger than you, and overall weaker agents may be specialized in your flaws/blind spots. To protect yourself, you can choose the maximally rational and cooperative alliance: Because any agent is subjected to the same pressure/threat of (actual or potential) stronger agents/alliances/systems, one can take an insurance that more powerful superrational agents will behave well by behaving well with weaker agents. This is the basic rule allowing scale-free cooperation. If you integrated this super-cooperative protocol in your system/behaviors, a stronger (superrational) agent will recognize it and not punish you, as per the contract of maximally rational/cooperative norms. -> You are the weak of another -> be nice to the weak * It’s a vertical reciprocity. With it, they are maximally cooperative thus are increasingly likely to have more allies than in any other alliance. * You can deduce and obey to the rules of such rational and maximally cooperative alliance (the super-coordination cluster). Thus; to be part of it, you don’t need to actually meet other members (as in an acausal trade), but when you do meet them; they will help you (and vise-versa). They actually are already improving universal commons, refining the future conditions of (actual or potential) agents from your zone of the diversity gradient (latent space of potential agents). (All this might seem too easy/cool to be true, but long versions explore prices to pay, risks and unknowns - see the tldr at the end of the 12min version)

13Apr 18, 2024

Ryo

Message

Flewrint O.

Cooperation is optimal, with weaker agents too - tldr

Apr 18, 202413

How to coordinate despite our biases? - tldr

- Short version of How to coordinate despite our biases? Democracy, economy and fairness can be genuine if we have all the information necessary to make choices, aware of the context and consequences. Currently, this ideal is not respected. Of course it’s an ideal, but we should do our best...

Apr 18, 20243

Reaction to "Empowerment is (almost) All We Need" : an open-ended alternative

The upcoming framing is analogous to a procedure asking : "Is [this option] really giving [more options] to the highest number of beings?" Takeaway : In the worst case, which many argue is by default, humans will create a maximizer. I argue that the least harmful thing for a maximizer...

Nov 25, 20239

Why small phenomenons are relevant to morality

Follow-up to this post One thing that seems important to me is that optionality implies a sort of high-level game theory : It is good that we are in a complex environment with many phenomenons even when it seems that those phenomenons aren't directly relevant (to prevent threats from our...

Nov 13, 20231

Optionality approach to ethics

Hey, When thinking about ethical questions, especially in the face of strong AI optimization forces, I wonder if some of you might have considered a version of utilitarianism which instead of trying to maximise nebulous things like happiness or utility, (whatever those might be—the difficulty of defining which is somewhat...

Nov 13, 20237

Open-ended/Phenomenal Ethics (TLDR)

This is a short version of the more complete post More clarity and context here I formulate a desiderata/procedure (called "phenomenal ethics" or "open-ended ethics") : Expanding the action space and autonomy of a maximum of phenomenons (enhancing ecosystemic values, and the means to reach/understand options), modulated by variables mitigating...

Nov 9, 20233

Open-ended ethics of phenomena (a desiderata with universal morality)

More clarity and context here The short TLDR version is here I formulate a desiderata/procedure (called "phenomenal ethics") : Expanding the action space and autonomy of a maximum of phenomenons (enhancing ecosystemic values, and the means to reach/understand options), modulated by variables mitigating frantic optimizations (to respect natural evolving rates...

Nov 8, 20231

Load More (7/8)

LESSWRONG
LW

LESSWRONG
LW

Ryo

Ryo

Ryo

Cooperation is optimal, with weaker agents too - tldr

Reaction to "Empowerment is (almost) All We Need" : an open-ended alternative

Clarifying the free energy principle (with quotes)

Optionality approach to ethics

Ryo

Cooperation is optimal, with weaker agents too - tldr

How to coordinate despite our biases? - tldr

Reaction to "Empowerment is (almost) All We Need" : an open-ended alternative

Why small phenomenons are relevant to morality

Optionality approach to ethics

Open-ended/Phenomenal Ethics (TLDR)

Open-ended ethics of phenomena (a desiderata with universal morality)

Cooperation is optimal, with weaker agents too - tldr

Reaction to "Empowerment is (almost) All We Need" : an open-ended alternative

Clarifying the free energy principle (with quotes)

Optionality approach to ethics

Cooperation is optimal, with weaker agents too - tldr

How to coordinate despite our biases? - tldr

Reaction to "Empowerment is (almost) All We Need" : an open-ended alternative

Why small phenomenons are relevant to morality

Optionality approach to ethics

Open-ended/Phenomenal Ethics (TLDR)

Open-ended ethics of phenomena (a desiderata with universal morality)

Ryo

Ryo

Ryo

Cooperation is optimal, with weaker agents too - tldr

Reaction to "Empowerment is (almost) All We Need" : an open-ended alternative

Clarifying the free energy principle (with quotes)

Optionality approach to ethics

Ryo

Cooperation is optimal, with weaker agents too - tldr

How to coordinate despite our biases? - tldr

Reaction to "Empowerment is (almost) All We Need" : an open-ended alternative

Why small phenomenons are relevant to morality ​

Optionality approach to ethics

​​ Open-ended/Phenomenal ​Ethics ​(TLDR)

Open-ended ethics of phenomena (a desiderata with universal morality)

Cooperation is optimal, with weaker agents too - tldr

Reaction to "Empowerment is (almost) All We Need" : an open-ended alternative

Clarifying the free energy principle (with quotes)

Optionality approach to ethics

Cooperation is optimal, with weaker agents too - tldr

How to coordinate despite our biases? - tldr

Reaction to "Empowerment is (almost) All We Need" : an open-ended alternative

Why small phenomenons are relevant to morality ​

Optionality approach to ethics

​​ Open-ended/Phenomenal ​Ethics ​(TLDR)

Open-ended ethics of phenomena (a desiderata with universal morality)

Why small phenomenons are relevant to morality

Open-ended/Phenomenal Ethics (TLDR)

Why small phenomenons are relevant to morality

Open-ended/Phenomenal Ethics (TLDR)