The Growth of My Pessimism: Transhumanism, Immortalism, Effective Altruism.
- Why I Grew Skeptical of Transhumanism
- Why I Grew Skeptical of Immortalism
- Why I Grew Skeptical of Effective Altruism
- Only Game in Town
Wonderland’s rabbit said it best: The hurrier I go, the behinder I get.
We approach 2016, and the more I see light, the more I see brilliance popping everywhere, the Effective Altruism movement growing, TEDs and Elons spreading the word, the more we switch our heroes in the right direction, the behinder I get. But why? - you say.
Clarity, precision, I am tempted to reply. I have left the intellectual suburbs of Brazil, straight into the strongest hub of production of things that matter, The Bay Area, via Oxford’s FHI office, I now split my time between UC Berkeley, and the CFAR/MIRI office. In the process, I have navigated an ocean of information, read hundreds of books, papers, saw thousands of classes, became proficient in a handful of languages and a handful of intellectual disciplines. I’ve visited the Olympus and I met our living demigods in person as well.
Against the overwhelming forces of an extremely upbeat personality surfing a hyper base-level happiness, these three forces: approaching the center, learning voraciously, and meeting the so-called heroes, have brought me to the current state of pessimism.
I was a transhumanist, an immortalist, and an effective altruist.
Why I Grew Skeptical of Transhumanism
The transhumanist in me is skeptical of technological development fast enough for improving the human condition to be worth it now, he sees most technologies as fancy toys that don’t get us there. Our technologies can’t and won’t for a while lead our minds to peaks anywhere near the peaks we found by simply introducing weirdly shaped molecules into our brains. The strangeness of Salvia, the beauty of LSD, the love of MDMA are orders and orders of magnitude beyond what we know how to change from an engineering perspective. We can induce a rainbow, but we don’t even have the concept of force yet. Our knowledge about the brain, given our goals about the brain, is at the level of knowledge of physics of someone who found out that spraying water on a sunny day causes the rainbow. It’s not even physics yet.
Believe me, I have read thousands of pages of papers in the most advanced topics in cognitive neuroscience, my advisor spent his entire career, from Harvard to Tenure, doing neuroscience, and was the first person to implant neurons that actually healed a brain to the point of recovering functionality by using non-human neurons. As Marvin Minsky, who invented the multi-agent computational theory of mind, told me: I don’t recommend entering a field where every four years all knowledge is obsolete, they just don’t know it yet.
Why I Grew Skeptical of Immortalism
The immortalist in me is skeptical because he understands the complexity of biology from conversations with the centimillionaires and with the chief scientists of anti-aging research facilities worldwide, he met the bio-startup founders and gets that the structure of incentives does not look good for bio-startups anyway, so although he was once very excited about the prospect of defeating the mechanisms of ageing, back when less than 300 thousand dollars were directly invested in it, he is now, with billions pledged against ageing, confident that the problem is substantially harder to surmount than the number of man-hours left to be invested in the problem, at least during my lifetime, or before the Intelligence Explosion.
Believe me, I was the first cryonicist among the 200 million people striding my country, won a prize for anti-ageing research at the bright young age of 17, and hang out on a regular basis with all the people in this world who want to beat death that still share in our privilege of living, just in case some new insight comes that changes the tides, but none has come in the last ten years, as our friend Aubrey will be keen to tell you in detail.
Why I Grew Skeptical of Effective Altruism
The Effective Altruist is skeptical too, although less so, I’m still founding an EA research institute, keeping a loving eye on the one I left behind, living with EAs, working at EA offices and mostly broadcasting ideas and researching with EAs. Here are some problems with EA which make me skeptical after being shook around by the three forces:
-
The Status Games: Signalling, countersignalling, going one more meta-level up, outsmarting your opponent, seeing others as opponents, my cause is the only true cause, zero-sum mating scarcity, pretending that poly eliminates mating scarcity, founders X joiners, researchers X executives, us institutions versus them institutions, cheap individuals versus expensive institutional salaries, it's gore all the way up and down.
-
Reasoning by Analogy: Few EAs are able to and doing their due intellectual diligence. I don’t blame them, the space of Crucial Considerations is not only very large, but extremely uncomfortable to look at, who wants to know our species has not even found the stepping stones to make sure that what matters is preserved and guaranteed at the end of the day? It is a hefty ordeal. Nevertheless, it is problematic that fewer than 20 EAs (one in 300?) are actually reasoning from first principles, thinking all things through from the very beginning. Most of us are looking away from at least some philosophical assumption or technological prediction. Most of us are cooks and not yet chefs. Some of us have not even waken up yet.
-
Babies with a Detonator: Most EAs still carry their transitional objects around, clinging desperately to an idea or a person they think more guaranteed to be true, be it hardcore patternism about philosophy of mind, global aggregative utilitarianism, veganism, or the expectation of immortality.
-
The Size of the Problem: No matter if you are fighting suffering, Nature, Chronos (death), Azathoth (evolutionary forces) or Moloch (deranged emergent structures of incentives), the size of the problem is just tremendous. One completely ordinary reason to not want to face the problem, or to be in denial, is the problem’s enormity.
-
The Complexity of The Solution: Let me spell this out, the nature of the solution is not simple in the least. It’s possible that we luck out and it turns out the Orthogonality Thesis and the Doomsday Argument and Mind Crime are just philosophical curiosities that have no practical bearing in our earthly engineering efforts, that the AGI or Emulation will by default fall into an attractor basin which implements some form of MaxiPok with details that it only grasps after CEV or the Crypto, and we will be Ok. It is possible, and it is more likely than that our efforts will end up being the decisive factor. We need to focus our actions in the branches where they matter though.
-
The Nature of the Solution: So let’s sit down side by side and stare at the void together for a bit. The nature of the solution is getting a group of apes who just invented the internet from everywhere around the world, and get them to coordinate an effort that fills in the entire box of Crucial Considerations yet unknown - this is the goal of Convergence Analysis, by the way - find every single last one of them to the point where the box is filled, then, once we have all the Crucial Considerations available, develop, faster than anyone else trying, a translation scheme that translates our values to a machine or emulation, in a physically sound and technically robust way (that’s if we don’t find a Crucial Consideration otherwise which, say, steers our course towards Mars). Then we need to develop the engineering prerequisites to implement a thinking being smarter than all our scientists together who can reflect philosophically better than the last two thousand years of effort while becoming the most powerful entity in the universe’s history, that will fall into the right attractor basin within mindspace. That’s if Superintelligences are even possible technically. Add to that we or it have to guess correctly all the philosophical problems that are A)Relevant B)Unsolvable within physics (if any) or by computers, all of this has to happen while the most powerful corporations, States, armies and individuals attempt to seize control of the smart systems themselves. without being curtailed by the hindrance counter incentive of not destroying the world either because they don’t realize it, or because the first mover advantage seems worth the risk, or because they are about to die anyway so there’s not much to lose.
-
How Large an Uncertainty: Our uncertainties loom large. We have some technical but not much philosophical understanding of suffering, and our technical understanding is insufficient to confidently assign moral status to other entities, specially if they diverge in more dimensions than brain size and architecture. We’ve barely scratched the surface of technical understanding on happiness increase, and philosophical understanding is also in its first steps.
-
Macrostrategy is Hard: A Chess Grandmaster usually takes many years to acquire sufficient strategic skill to command the title. It takes a deep and profound understanding of unfolding structures to grasp how to beam a message or a change into the future. We are attempting to beam a complete value lock-in in the right basin, which is proportionally harder.
-
Probabilistic Reasoning = Reasoning by Analogy: We need a community that at once understands probability theory, doesn’t play reference class tennis, and doesn’t lose motivation by considering the base rates of other people trying to do something, because the other people were cooks, not chefs, and also because sometimes you actually need to try a one in ten thousand chance. But people are too proud of their command of Bayes to let go of the easy chance of showing off their ability to find mathematically sound reasons not to try.
-
Excessive Trust in Institutions: Very often people go through a simplifying set of assumptions that collapses a brilliant idea into an awful donation, when they reason:
I have concluded that cause X is the most relevant
Institution A is an EA organization fighting for cause X
Therefore I donate to institution A to fight for cause X.
To begin with, this is very expensive compared to donating to any of the three P’s: projects, people or prizes. Furthermore, the crucial points to fund institutions are when they are about to die, just starting, or building a type of momentum that has a narrow window of opportunity where the derivative gains are particularly large or you have private information about their current value. To agree with you about a cause being important is far from sufficient to assess the expected value of your donation. -
Delusional Optimism: Everyone who like past-me moves in with delusional optimism will always have a blind spot in the feature of reality about which they are in denial. It is not a problem to have some individuals with a blind spot, as long as the rate doesn’t surpass some group sanity threshold, yet, on an individual level, it is often the case that those who can gaze into the void a little longer than the rest end up being the ones who accomplish things. Staring into the void makes people show up.
-
Convergence of opinions may strengthen separation within EA: Thus far, the longer someone is an EA for, the more likely they are to transition to an opinion in the subsequent boxes in this flowchart from whichever box they are at at the time. There are still people in all the opinion boxes, but the trend has been to move in that flow. Institutions however have a harder time escaping being locked into a specific opinion. As FHI moves deeper into AI, and GWWC into poverty, 80k into career selection etc… they become more congealed. People’s opinions are still changing, and some of the money follows, but institutions are crystallizing into some opinions, and in the future they might prevent transition between opinion clusters and free mobility of individuals, like national frontiers already do. Once institutions, which in theory are commanded by people who agree with institutional values, notice that their rate of loss towards the EA movement is higher than their rate of gain, they will have incentives to prevent the flow of talent, ideas and resources that has so far been a hallmark of Effective Altruism and why many of us find it impressive, it’s being an intensional movement. Any part that congeals or becomes extensional will drift off behind, and this may create unsurmountable separation between groups that want to claim ‘EA’ for themselves.
Only Game in Town
The reasons above have transformed a pathological optimist into a wary skeptical about our future, and the value of our plans to get there. And yet, I don’t see other option than to continue the battle. I wake up in the morning and consider my alternatives: Hedonism, well, that is fun for a while, and I could try a quantitative approach to guarantee maximal happiness over the course of the 300 000 hours I have left. But all things considered, anyone reading this is already too close to the epicenter of something that can become extremely important and change the world to have the affordance to wander off indeterminately. I look at my high base-happiness and don’t feel justified in maximizing it up to the point of no marginal return, there clearly is value elsewhere than here (points inwards), clearly the self of which I am made has strong altruistic urges anyway, so at least above a threshold of happiness, has reason to purchase the extremely good deals in expected value happiness of others that seem to be on the market. Other alternatives? Existentialism? Well, yes, we always have a fundamental choice and I feel the thrownness into this world as much as any Kierkegaard does. Power? When we read Nietzsche it gives that fantasy impression that power is really interesting and worth fighting for, but at the end of the day we still live in a universe where the wealthy are often reduced to having to spend their power in pathetic signalling games and zero sum disputes or coercing minds to act against their will. Nihilism and Moral Fictionalism, like Existentialism all collapse into having a choice, and if I have a choice my choice is always going to be the choice to, most of the time, care, try and do.
Ideally, I am still a transhumanist and an immortalist. But in practice, I have abandoned those noble ideals, and pragmatically only continue to be an EA.
It is the only game in town.
Sidekick Matchmaking - How to tackle the problem?
Some of us enjoy being sidekicks.
Some of us would like to meet sidekicks in potential, see how the interaction goes, and have sidekicks.
Last time I tried posting about sidekick matchmaking here, it turned out to be very valuable for me, but not for many people (I think only two pairs of sidekick were created as a result). Now, once again I'd like to find someone who enjoys that role to help me out with many projects.
I'm looking for suggestions on how to get people together to do that. For the time being, if someone needs a sidekick or wants to be one, post about it in the comment section. I'd love to see a permanent solution for this information spreading problem.
My experience with Sidekicks
I'm not sure what Anna and Nick thought of their sidekicks, but my experience was undeniably positive. Having a sidekick was motivating, saved me great time, and, most importantly, felt like I got a surge of muscle strength specifically in the types of tasks I'm particularly inept at.
By contrast, my experience with people hired to help was mixed (virtual assistants) or negative (personal assistant).
Use the comment section to either offer or request sidekicks, explaining a little more about you and what you'd like this partnership to mean
Effectively Less Altruistically Wrong Codex
My post on the fact that incentive structures are eating the central place to be for rationalists has generated 140 comments which I have generated no clear action in the horizon.
I post here again to incentivize that it also generates some attempts to shake the ground a bit. Arguing and discussing are fun, and beware of things that are fun to argue.
Is anyone actually doing anything to mitigate the problem? To solve it? To have a stable end state in the long run where online discussions still preserve what needs being preserved?
Intelligent commentary is valuable, pools are interesting. Yet, at the end of the day, it is the people who show up to do something who will determine the course of everything.
If you care about this problem, act on it. I care enough to write these two posts.
Lesswrong, Effective Altruism Forum and Slate Star Codex: Harm Reduction
Cross Posted at the EA Forum
At Event Horizon (a Rationalist/Effective Altruist house in Berkeley) my roommates yesterday were worried about Slate Star Codex. Their worries also apply to the Effective Altruism Forum, so I'll extend them.
The Problem:
Lesswrong was for many years the gravitational center for young rationalists worldwide, and it permits posting by new users, so good new ideas had a strong incentive to emerge.
With the rise of Slate Star Codex, the incentive for new users to post content on Lesswrong went down. Posting at Slate Star Codex is not open, so potentially great bloggers are not incentivized to come up with their ideas, but only to comment on the ones there.
The Effective Altruism forum doesn't have that particular problem. It is however more constrained in terms of what can be posted there. It is after all supposed to be about Effective Altruism.
We thus have three different strong attractors for the large community of people who enjoy reading blog posts online and are nearby in idea space.
Possible Solutions:
(EDIT: By possible solutions I merely mean to say "these are some bad solutions I came up with in 5 minutes, and the reason I'm posting them here is because if I post bad solutions, other people will be incentivized to post better solutions)
If Slate Star Codex became an open blog like Lesswrong, more people would consider transitioning from passive lurkers to actual posters.
If the Effective Altruism Forum got as many readers as Lesswrong, there could be two gravity centers at the same time.
If the moderation and self selection of Main was changed into something that attracts those who have been on LW for a long time, and discussion was changed to something like Newcomers discussion, LW could go back to being the main space, with a two tier system (maybe one modulated by karma as well).
The Past:
In the past there was Overcoming Bias, and Lesswrong in part became a stronger attractor because it was more open. Eventually lesswrongers migrated from Main to Discussion, and from there to Slate Star Codex, 80k blog, Effective Altruism forum, back to Overcoming Bias, and Wait But Why.
It is possible that Lesswrong had simply exerted it's capacity.
It is possible that a new higher tier league was needed to keep post quality high.
A Suggestion:
I suggest two things should be preserved:
Interesting content being created by those with more experience and knowledge who have interacted in this memespace for longer (part of why Slate Star Codex is powerful), and
The opportunity (and total absence of trivial inconveniences) for new people to try creating their own new posts.
If these two properties are kept, there is a lot of value to be gained by everyone.
The Status Quo:
I feel like we are living in a very suboptimal blogosphere. On LW, Discussion is more read than Main, which means what is being promoted to Main is not attractive to the people who are actually reading Lesswrong. The top tier quality for actually read posting is dominated by one individual (a great one, but still), disincentivizing high quality posts by other high quality people. The EA Forum has high quality posts that go unread because it isn't the center of attention.
Confession Thread: Mistakes as an aspiring rationalist
We looked at the cloudy night sky and thought it would be interesting to share the ways in which, in the past, we made mistakes we would have been able to overcome, if only we had been stronger as rationalists. The experience felt valuable and humbling. So why not do some more of it on Lesswrong?
An antithesis to the Bragging Thread, this is a thread to share where we made mistakes. Where we knew we could, but didn't. Where we felt we were wrong, but carried on anyway.
As with the recent group bragging thread, anything you've done wrong since the comet killed the dinosaurs is fair game, and if it happens to be a systematic mistake that over long periods of time systematically curtailed your potential, that others can try to learn avoiding, better.
This thread is an attempt to see if there are exceptions to the cached thought that life experience cannot be learned but has to be lived. Let's test this belief together!
Compilation of currently existing project ideas to significantly impact the world
One of the problems the LW, EA, CFAR X-risk community has been faced with recently discussed on Slate Star Codex is the absorption of people interested in researching, volunteering, helping, participating in the community. A problem worth subdividing into how to get new people into the social community, which is addressed on the link above, and separate problem, absorbing their skills, ability, and willingness to volunteer, to which this post is dedicated:
What should specific person Smith do to help in the project of preventing X-risk, improving the world, saving lives? We assume here Smith will not be a donor - in which case the response would be "donate" - joined the community not long ago and has a skill set X.
Soon this problem will become worse due to influx of more people brought in by the soon to be published books by MacAskill, Yudkowsky and Singer coming out. There will be more people wanting to do something, and able to do some sorts of projects, but who are not being allocated any specific project that matches their skill set and values.
Now is a good time to solve it. I was talking about this problem today with Stephen Frey and we considered it would be a good idea to have a list of specific technical or research projects that can be broken down into smaller chunks for people to pick up and do. A Getting Things Done list for researchers and technology designers. Preferably those would be tractable projects that can be done in fewer than three months. There are some lists of open problems in AI and Superintelligence control, but not for many X-risks or other problems that some of the community frequently considers important.
So I decided to make a compilation of the questions and problems we already have listed here, and then ask someone (Oliver Habryka or a volunteer in the comment section here) to transform the compiled set into a standardized format.
A tentative category list
Area: X-risk, AI, Anti-aging, Cryonics, Rationality, IA, Self-Improvement, Differential Technological Development, Strategy, Civilizational Inadequacy, etc... describes what you have to value/disvalue in order for this project to match your values.
Project: description of which actions need to be taken in 3 month period for this project to be considered complete.
Context: if part of a larger project, which is it, and how will it connect to other parts. Also justification for that project.
Notes: any relevant constraints that may play a role, time-sensitivity, costs, number of people, location, etc...
For example at Luke's list of Superintelligence research questions, the first one:
How strongly does IQ predict rationality, metacognition, and philosophical sophistication, especially in the far right tail of the IQ distribution? Relevant to the interaction of intelligence amplification and FAI chances. See the project guide here.
Would be rendered as
Area: FAI ; Project: Read Rationality and the Reflective Mind, by Keith Stanovich, to become familiar with the model of algorithmic and reflective minds. For this project, investigating metacognition means investigating the reflective mind. Find ways to test Stanovich’s predictions and answer the questions in the previous section. Design the study to give participants tests which high a IQ should help with and tests which a high IQ should not help with. This step will involve searching through Rationality and the Reflective Mind, and then directly contacting Stanovich to ask which tests he has not yet conducted. Context: this is the first of two part sub-study investigating IQ and metacognition, and needs being followed by conducting a new study investigating the correlation. These parts are complimentary with the study of IQ and philosophical success, and are relevant to assess the impact that intelligence augmentation will have in our likelihood of generating Friendly Artificial Intelligence. Notes: needs to be conducted by someone with a researcher affiliation and capacity to conduct a study on human subjects later, six month commitment, some science writing experience.
Edit: Here is a file where to start compiling projects - thanks Stephen!
This is the idea.To gather a comprehensive list of research or technical questions for the areas above, transform them into projects that can be more easily parsed and assigned than their currently scattered counterparts and make the list available to those who want to work on them. This post is the first step in collection, so if there are lists anywhere of projects, or research questions that may be relevant for any of the areas cited above, please post a link to these at the comments - special kudos if you already post it in the format above. Also let me know if you would like to volunteer in this. If you remember any question or specific project but don't see it in any list or on the comments, post it. When we create a standardized list for people to look through it will be separated by area, so people can visualize only projects related to what they value.
Compilation:
Lists of ideas and projects:
Superintelligence Strategic List - Muelhauser
Mechanisms of Aging - Ben Best
Cryonics Strategy Space - Froolow
Ideas and projects:
Go to Mars - Musk
Make it easy for people within the community to move to US, UK.
Preserve Brains
Find moral enhancers that improve global cooperation as well as intra-group cooperation
Open Borders
...
...
Sidekick Matchmaking
Thanks linkhyrule5 for suggesting this.
Post your request for Sidekicks or your desire to be a sidekick in the comment section below.
Send a personal message to your potential match to start communicating instead of replying in the thread, to save space and avoid biases, besides privacy.
[edit] Mathias Zamman suggests some questions:
Questions for both Heroes and Sidekicks (and Dragons, etc.)
- Post a short description of yourself: personality, skills, general goals.
- Where do you live?
- How do you see the contact between the two of you going?
- What you require in your counterpart: This can be a bit vague but it might be too hard to verbalize for some people
Questions for Heroes:
- What is your goal?
- Why are you a Hero?
- Why do you require a Sidekick?
- What specific tasks would a Sidekick perform for you?
- What qualities would you not want in a Sidekick?
Questions for Sidekicks:
- What sort of goals are you looking for?
- Why are you Sidekick material?
- Why do you require a Hero?
- What sort of tasks could you do for a Hero?
- What qualities don't you want in a Hero?
An alarming fact about the anti-aging community
Past and Present
Ten years ago teenager me was hopeful. And stupid.
The world neglected aging as a disease, Aubrey had barely started spreading memes, to the point it was worth it for him to let me work remotely to help with Metuselah foundation. They had not even received that initial 1,000,000 donation from an anonymous donor. The Metuselah prize was running for less than 400,000 if I remember well. Still, I was a believer.
Now we live in the age of Larry Page's Calico, 100,000,000 dollars trying to tackle the problem, besides many other amazing initiatives, from the research paid for by Life Extension Foundation and Bill Faloon, to scholars in top universities like Steve Garan and Kenneth Hayworth fixing things from our models of aging to plastination techniques. Yet, I am much more skeptical now.
Individual risk
I am skeptical because I could not find a single individual who already used a simple technique that could certainly save you many years of healthy life. I could not even find a single individual who looked into it and decided it wasn't worth it, or was too pricy, or something of that sort.
That technique is freezing some of your cells now.
Freezing cells is not a far future hope, this is something that already exists, and has been possible for decades. The reason you would want to freeze them, in case you haven't thought of it, is that they are getting older every day, so the ones you have now are the youngest ones you'll ever be able to use.
Using these cells to create new organs is not something that may help you if medicine and technology continue progressing according to the law of accelerating returns in 10 or 30 years. We already know how to make organs out of your cells. Right now. Some organs live longer, some shorter, but it can be done - for instance to bladders - and is being done.
Hope versus Reason
Now, you'd think if there was an almost non-invasive technique already shown to work in humans that can preserve many years of your life and involves only a few trivial inconveniences - compared to changing diet or exercising for instance- the whole longevist/immortalist crowd would be lining up for it and keeping back up tissue samples all over the place.
Well I've asked them. I've asked some of the adamant researchers, and I've asked the superwealthy; I've asked the cryonicists and supplement gorgers; I've asked those who work on this 8 hour a day every day, and I've asked those who pay others to do so. I asked it mostly for selfish reasons, I saw the TEDs by Juan Enriquez and Anthony Atala and thought: hey look, clearly beneficial expected life length increase, yay! let me call someone who found this out before me - anyone, I'm probably the last one, silly me - and fix this.
I've asked them all, and I have nothing to show for it.
My takeaway lesson is: whatever it is that other people are doing to solve their own impending death, they are far from doing it rationally, and maybe most of the money and psychology involved in this whole business is about buying hope, not about staring into the void and finding out the best ways of dodging it. Maybe people are not in fact going to go all-in if the opportunity comes.
How to fix this?
Let me disclose first that I have no idea how to fix this problem. I don't mean the problem of getting all longevists to freeze their cells, I mean the problem of getting them to take information from the world of science and biomedicine and applying it to themselves. To become users of the technology they are boasters of. To behave rationally in a CFAR or even homo economicus sense.
I was hoping for a grandiose idea in this last paragraph, but it didn't come. I'll go with a quote from this emotional song sung by us during last year's Secular Solstice celebration
Do you realize? that everyone, you know, someday will die...
And instead of sending all your goodbyes
Human Minds are Fragile
We are familiar with the thesis that Value is Fragile. This is why we are researching how to impart values to an AGI.
Embedded Minds are Fragile
Besides values, it may be worth remembering that human minds too are very fragile.
A little magnetic tampering with your amygdalas, and suddenly you are a wannabe serial killer. A small dose of LSD can get you to believe you can fly, or that the world will end in 4 hours. Remove part of your Ventromedial PreFrontal Cortex, and suddenly you are so utilitarian even Joshua Greene would call you a psycho.
It requires very little material change to substantially modify a human being's behavior. Same holds for other animals with embedded brains, crafted by evolution and made of squishy matter modulated by glands and molecular gates.
A Problem for Paul-Boxing and CEV?
One assumption underlying Paul-Boxing and CEV is that:
It is easier to specify and simulate a human-like mind then to impart values to an AGI by means of teaching it values directly via code or human language.
Usually we assume that because, as we know, value is fragile. But so are embedded minds. Very little tampering is required to profoundly transform people's moral intuitions. A large fraction of the inmate population in the US has frontal lobe or amygdala malfunctions.
Finding out the simplest description of a human brain that when simulated continues to act as that human brain would act in the real world may turn out to be as fragile, or even more fragile, than concept learning for AGI's.
Should EA's be Superrational cooperators?
Back in 2012 when visiting Leverage Research, I was amazed by the level of cooperation in daily situations I got from Mark. Mark wasn't just nice, or kind, or generous. Mark seemed to be playing a different game than everyone else.
If someone needed X, and Mark had X, he would provide X to them. This was true for lending, but also for giving away.
If there was a situation in which someone needed to direct attention to a particular topic, Mark would do it.
You get the picture. Faced with prisoner dilemmas, Mark would cooperate. Faced with tragedy of the commons, Mark would cooperate. Faced with non-egalitarian distributions of resources, time or luck (which are convoluted forms of the dictator game), Mark would rearrange resources without any indexical evaluation. The action would be the same, and the consequentialist one, regardless of which side of a dispute was the Mark side.
I never got over that impression. The impression that I could try to be as cooperative as my idealized fiction of Mark was.
In game theoretic terms, Mark was a Cooperational agent.
- Altruistic - MaxOther
- Cooperational - MaxSum
- Individualist - MaxOwn
- Equalitarian - MinDiff
- Competitive - MaxDiff
- Aggressive - MinOther
Under these definitions of kinds of agents used in research on game theoretical scenarios, what we call Effective Altruism would be called Effective Cooperation. The reason why we call it "altruism" is because even the most parochial EA's care about a set containing a minimum of 7 billion minds, where to a first approximation MaxSum ≈ MaxOther.
Locally however the distinction makes sense. In biology Altruism usually refers to a third concept, different from both the "A" in EA, and Alt, it means acting in such a way that Other>Own without reference to maximizing or minimizing, since evolution designs adaptation executors, not maximizers.
A globally Cooperational agent acts as a consequentialist globally. So does an Alt agent.
The question then is,
How should a consequentialist act locally?
The mathematical response is obviously as a Coo. What real people do is a mix of Coo and Ind.
My suggestion is that we use our undesirable yet unavoidable moral tribe distinction instinct, the one that separates Us from Them, and act always as Coos with Effective Altruists and mix Coo and Ind only with non EAs. That is what Mark did.
View more: Next
Subscribe to RSS Feed
= f037147d6e6c911a85753b9abdedda8d)