Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

A Visualization of Nick Bostrom’s Superintelligence

16 AmandaEHouse 23 July 2014 12:24AM

Through a series of diagrams, this article will walk through key concepts in Nick Bostrom’s Superintelligence. The book is full of heavy content, and though well written, its scope and depth can make it difficult to grasp the concepts and mentally hold them together. The motivation behind making these diagrams is not to repeat an explanation of the content, but rather to present the content in such a way that the connections become clear. Thus, this article is best read and used as a supplement to Superintelligence.


Note: Superintelligence is now available in the UK. The hardcover is coming out in the US on September 3. The Kindle version is already available in the US as well as the UK.

Roadmap: there are two diagrams, both presented with an accompanying description. The two diagrams are combined into one mega-diagram at the end.




Figure 1: Pathways to Superintelligence



Figure 1 displays the five pathways toward superintelligence that Bostrom describes in chapter 2 and returns to in chapter 14 of the text. According to Bostrom, brain-computer interfaces are unlikely to yield superintelligence. Biological cognition, i.e., the enhancement of human intelligence, may yield a weak form of superintelligence on its own. Additionally, improvements to biological cognition could feed back into driving the progress of artificial intelligence or whole brain emulation. The arrows from networks and organizations likewise indicate technologies feeding back into AI and whole brain emulation development.


Artificial intelligence and whole brain emulation are two pathways that can lead to fully realized superintelligence. Note that neuromorphic is listed under artificial intelligence, but an arrow connects from whole brain emulation to neuromorphic. In chapter 14, Bostrom suggests that neuromorphic is a potential outcome of incomplete or improper whole brain emulation. Synthetic AI includes all the approaches to AI that are not neuromorphic; other terms that have been used are algorithmic or de novo AI.

continue reading »

Meetup : Bratislava Meetup XV.

0 Viliam_Bur 22 July 2014 05:25PM

Discussion article for the meetup : Bratislava Meetup XV.

WHEN: 28 July 2014 06:00:00PM (+0200)

WHERE: Bistro The Peach, Mariánska 3, Bratislava

Pozor, zmena adresy! Voľná diskusia.

Discussion article for the meetup : Bratislava Meetup XV.

Meetup : London social meetup - possibly in a park

0 philh 22 July 2014 05:20PM

Discussion article for the meetup : London social meetup - possibly in a park

WHEN: 27 July 2014 02:00:00PM (+0100)

WHERE: Shakespeare's Head, Holborn, WC2B 6BG

The next LW London meetup will be on July 27th. Join us from 2pm to talk about the sorts of things that your other friends will look funny at you for talking about.

If the weather is nice, we'll head to Lincoln's Inn Fields, probably somewhere in the northwest quadrant. If not, we'll be in the usual Shakespeare's Head. If the weather is variable, we might move from one to the other - give me a call or text if you're not sure. My number is 07792009646.

About London LessWrong:

We run this meetup almost every week; these days we tend to get in the region of 5-15 people in attendance. By default, meetups are just unstructured social discussion about whatever strikes our fancy: books we're reading, recent posts on LW/related blogs, logic puzzles, toilet usage statistics....

Sometimes we play The Resistance or other games. We usually finish around 7pm, give or take an hour, but people arrive and leave whenever suits them.

Related discussion happens on both our google group and our facebook group.

Discussion article for the meetup : London social meetup - possibly in a park

Knightian Uncertainty and Ambiguity Aversion: Motivation

9 So8res 21 July 2014 08:32PM

Recently, I found myself in a conversation with someone advocating the use of Knightian uncertainty. I admitted that I've never found the concept compelling. We went back and forth for a little while. His points were crisp and well-supported, my objections were vague. We didn't have enough time to reach consensus, but it became clear that I needed to research his viewpoint and flesh out my objections before being justified in my rejection.

So I did. This is the first in a short series of posts during which I explore what it means for an agent to reason using Knightian uncertainty.

In this first post, I'll present a number of arguments claiming that Bayesian reasoning fails to capture certain desirable behavior. I'll discuss a proposed solution, maximization of minimum expected utility, which is advocated by my friend and others.

In the second post, I'll discuss some more general arguments against Bayesian reasoning as an idealization of human reasoning. What role should "unknown unknowns" play in a bounded Bayesian reasoner? Is "Knightian uncertainty" a useful concept that is not captured by the Bayesian framework?

In the third post, I'll discuss the proposed solution: can rational agents display ambiguity aversion? What does it mean to have a rational agent that does not maximize expected utility, maximizing "minimum expected utility" instead?

In the final post, I'll apply these insights to humans and articulate my objections to ambiguity aversion in general. I'll conclude that while it is possible for agents to be ambiguity-averse, ambiguity aversion in humans is a bias. The maximization of minimum expected utility may be a useful concept for explaining how humans actually act, but probably isn't how you should act.

continue reading »

Weekly LW Meetups

1 FrankAdamek 18 July 2014 04:23PM

Meetup : MIRIxAtlanta: 19 July 2014

1 Adele_L 18 July 2014 04:04AM

Discussion article for the meetup : MIRIxAtlanta: 19 July 2014

WHEN: 19 July 2014 06:00:00PM (-0400)

WHERE: 2388 Lawrenceville Hwy. Unit L, Decatur, GA 30033

We'll be having the first MIRIx in Atlanta - please come if you are interested in math, programming, or AI. One research program at MIRI is to develop better theoretical foundations for making decisions under uncertainty. We'll look at some recent progress in this field, discuss open problems, and explore possible research directions.

Recommended Reading: Summary of problems: http://intelligence.org/research/#decision Robust Cooperation: http://arxiv.org/abs/1401.5577

P.S. There will be snacks!

Discussion article for the meetup : MIRIxAtlanta: 19 July 2014

Meetup : Washington DC: Short Talks

0 RobinZ 18 July 2014 01:02AM

Discussion article for the meetup : Washington DC: Short Talks

WHEN: 20 July 2014 03:00:00PM (-0400)

WHERE: National Portrait Gallery, Washington, DC

We will be meeting in the Kogod Courtyard to take turns presenting short (~10-20 minute) lectures on random topics.

(For newcomers: the courtyard is in the center of the building, and may be reached from either entrance by walking through the doors on the far side of the information desk.)

Discussion article for the meetup : Washington DC: Short Talks

Meetup : Canberra: Intro to Anthropic Reasoning

0 DanielFilan 16 July 2014 01:10PM

Discussion article for the meetup : Canberra: Intro to Anthropic Reasoning

WHEN: 25 July 2014 06:00:00PM (+1000)

WHERE: 108 North Road, Acton, ACT

I Think Therefore I Am; I Am, Therefore The World Is About To Be Destroyed!

Nick will be talking about anthropic reasoning, a collection of theories about how to handle observation selection biases, or in other words, how to reason about a problem when the fact that you exist gives you relevant information about that problem.

He will introduce and discuss anthropic theories such as the Self-Sampling Assumption, philosophical problems such as The Sleeping Beauty Problem and the Doomsday Argument, and a decision theory specifically designed to get around problems that involve anthropic bias. This will be followed by a group discussion and a game involving anthropic reasoning. As always, vegan snacks will be provided.

General meetup info:

If you use Facebook, please join our group: https://www.facebook.com/groups/lwcanberra/

Structured meetups are held on the second Saturday and fourth Friday of each month from 6 pm until late in the CSIT building, room N101.

There will be LWers at the Computer Science Students Association's weekly board games night, held on Wednesdays from 7 pm in the same location.

Discussion article for the meetup : Canberra: Intro to Anthropic Reasoning

Meetup : Melbourne Social Meetup

0 Maelin 16 July 2014 01:09PM

Discussion article for the meetup : Melbourne Social Meetup

WHEN: 18 July 2014 06:30:00PM (+1000)

WHERE: 9/239 Canterbury Rd, Blackburn

The social meetup is on again for July! This time we will be meeting in Blackburn. Come along for a chat and a game or two.

We'll be starting at 6:30pm, but we'll probably go pretty late. Drop by whenever is convenient.

Snack foods will be provided and we'll probably get some pizza or something for dinner.

If you'd like to join in carpooling, please discuss in the Facebook event (see the 'Less Wrong Melbourne' group) and we'll arrange something. Alternatively if you are taking public transport, jump off at Blackburn station (Lilydale/Belgrave line) and call me on 0421231789.

Discussion article for the meetup : Melbourne Social Meetup

Meetup : MIRIx Tel-Aviv

1 eli_sennesh 14 July 2014 01:58PM

Discussion article for the meetup : MIRIx Tel-Aviv

WHEN: 18 July 2014 09:00:00AM (+0300)

WHERE: Tel Aviv Makers Insurgence, Studio 106 (Third Floor), 45 Kibbutz Galuyot Street, Tel-Aviv, Israel

We will be holding a MIRIx workshop on July 18th, 2014, at the Tel-Aviv Makers Insurgence. If you've been reading the invites to regular Tel-Aviv meetups and wondering when we're finally going to teach you to build a Friendly AI that runs on cold fusion... we're sorry, but there is not yet any such thing as cold fusion.

We will, however, be investigating how to take probabilities over sentences in expressive logics (logical probability) for use in Updateless Decision Theory (and its associated Updateless Intelligence Metrics).

Discussion article for the meetup : MIRIx Tel-Aviv

View more: Next