[Link] Debiasing by rationalizing your own motives

1 Kaj_Sotala 03 September 2017 12:20PM
Comment author: Wei_Dai 28 August 2017 05:44:17PM 2 points [-]

I'm not an expert on bioweapons, but I note that the paper you cite is dated 2005, before the advent of synthetic biology. The recent report from FHI seems to consider bioweapons to be a realistic existential risk.

Comment author: Kaj_Sotala 29 August 2017 10:03:18AM 1 point [-]

Thanks, I hadn't seen that. Interesting (and scary).

Comment author: Wei_Dai 27 August 2017 10:13:51PM 3 points [-]

You could argue that mere destruction would be easier than converting everything to orgasmium, but both seem hard enough to basically require a superintelligence.

We can kill everyone today or in the near future by diverting a large asteroid to crash into Earth, or by engineering a super-plague. Doing either would take significant resources but isn't anywhere near requiring a superintelligence. In comparison, converting everything to orgasmium seems much harder and is far beyond our current technological capabilities.

Comment author: Kaj_Sotala 28 August 2017 11:55:09AM 1 point [-]

On super-plagues, I've understood the consensus position to be that even though you could create one that had really big death tolls, actual human extinction would be very unlikely. E.g.

Asked by Representative Christopher Shays (R-Conn.) whether a pathogen could be engineered that would be virulent enough to “wipe out all of humanity,” Fauci and other top officials at the hearing said such an agent was technically feasible but in practice unlikely.

Centers for Disease Control and Prevention Director Julie Gerberding said a deadly agent could be engineered with relative ease that could spread throughout the world if left unchecked, but that the outbreak would be unlikely to defeat countries’ detection and response systems.

“The technical obstacles are really trivial,” Gerberding said. “What’s difficult is the distribution of agents in ways that would bypass our capacity to recognize and intervene effectively.”

Fauci said creating an agent whose transmissibility could be sustained on such a scale, even as authorities worked to counter it, would be a daunting task.

“Would you end up with a microbe that functionally will … essentially wipe out everyone from the face of the Earth? … It would be very, very difficult to do that,” he said.

Asteroid strikes do sound more plausible, though there too I would expect a lot of people to be aware of the possibility and thus devote considerable measures to ensuring the safety of any space operations capable of actually diverting asteroids.

Comment author: Kaj_Sotala 25 August 2017 06:12:40PM *  4 points [-]

In the second paper, you mention radical negative utilitarians as a force that could be motivated to kill everyone, but similar considerations seem to apply to utilitarianism in general. Hedonistic utilitarians would want to convert the world into orgasmium (killing everyone in the process), varieties of preference utilitarianism might want to rewire everyone's brains so that those brains experience maximum preference satisfaction (thus effectively killing everyone), etc.

You could argue that mere destruction would be easier than converting everything to orgasmium, but both seem hard enough to basically require a superintelligence. And if you can set the goals of a superintelligence, it's not clear that one of the goals would be much harder than the other.

[Link] Multiverse-wide Cooperation via Correlated Decision Making

4 Kaj_Sotala 20 August 2017 12:01PM
Comment author: disconnect 15 August 2017 01:36:15PM 0 points [-]

People who become passionate about meditation tend to say that the hardest part is encountering "dark things in your mind".

What do meditators mean by this?

Comment author: Kaj_Sotala 18 August 2017 06:11:27PM 0 points [-]

There are stages in meditation when painful thoughts and memories might come bubbling up. If you're just sitting still with your mind and have nothing to distract you, you may occasionally end up facing some past trauma, especially if you've previously avoided dealing with it and have e.g. tried to just distract yourself from it whenever it came up.

(This is not necessarily a negative anything in the long run, since facing those negative thoughts can help in getting over them.)

Comment author: entirelyuseless 29 July 2017 02:18:26PM 0 points [-]

I agree it is not a common outcome in practice, although that is largely because people identify as "someone who does things," or at least as "someone who ought to do things," without identifying as something else that would actually drive them to do things. That is a recipe for making yourself miserable. It may be, too, that "someone who ought to do things" is enough of a natural identity, so to speak, that it is very hard for someone not to identify in that way, even if they think they are not doing so.

I suspect that if you're not an expert Buddhist meditator and basically living in a monastery, you'll just fail at this

This is probably right. Fortunately for me, that is not too far off from describing my life.

Comment author: Kaj_Sotala 29 July 2017 06:35:35PM 0 points [-]

largely because people identify as "someone who does things," or at least as "someone who ought to do things,"

(either that, or people identify as "someone who doesn't do things", and find that to be a concept with negative value)

Comment author: entirelyuseless 29 July 2017 01:58:57PM 0 points [-]

So if you do keep your identity small, you might not have a very strong motivation to actually ever do anything.

This is true, but on the other hand, if you actually succeed in keeping it small, as opposed to thinking wishfully about doing that, you will also actually not mind not doing anything.

Comment author: Kaj_Sotala 29 July 2017 02:04:22PM 0 points [-]

I'm skeptical of whether this is a particularly common outcome in practice. I suspect that if you're not an expert Buddhist meditator and basically living in a monastery, you'll just fail at this, and you'd have a much easier time achieving happiness by actually having a strong identity.

Comment author: Kaj_Sotala 29 July 2017 02:01:31PM 7 points [-]

This was really cool. Trust and betrayal, social cohesion, evolutionary game theory, cute music and graphics, and a wonderful message at the end.

<3

Comment author: moridinamael 28 July 2017 01:23:39PM 2 points [-]

That makes sense. I can see how a deep felt certainty that you're already awesome and perfect exactly as you are could have pathological consequences. I'll be careful. =)

Comment author: Kaj_Sotala 29 July 2017 01:32:23PM *  0 points [-]

Based on my reading of the book, I guess the main suggestions that I'd make to anyone interested in playing around with creating new self-concepts would be:

  • Use lots of diverse examples. E.g. if you're creating a self-concept of kindness, look for memories of both large and small acts of kindness. The memories contained within your self-concept serve as kind of templates to match various experiences against; the more diverse the database of templates, the more likely it is that different actions will be correctly identified. (Machine learning folks might say that this avoids overfitting.)
    • You can try to explicitly include memories from different years of life, as well as covering both short and long time periods (e.g. if you hold the door open to someone, that's an act of kindness lasting for a few seconds; if you have a friend who you've been helping with their troubles for the last several years, that can be thought of either as a big set of short acts or a single long-lasting act).
  • See if you can include counterexamples as well, maybe doing a negative-positive reframe to turn them into examples, or including qualifiers with them. Again the template matching thing: if your self-concept for a quality contains instances of occasions when you failed to act according to the quality, then that allows you to recognize future occasions when you're not acting in accordance to the quality. This helps avoid unrealistic overconfidence in the quality.

(And for anyone interested in doing this seriously, I much recommend the book for lots and lots of practical examples and tips.)

View more: Next