Review about the Rationality Meetup in Vienna on 27.09.2014

Superintelligence Summary by Marko Thiel

People who were there: Andreas, Matthias Brandner, Manuel K., Monika, Marko, Viliam Bur, Alex, Philip, Anna, Philipp, Tino/Sandy, Austin, Milica, Luka, Ivan, Axel, Lea, Lio, Andreas V., Manuel M. (20 people - awesome! And so many were new!)

Superintelligence by Nick Bostrom (presented by Marko Thiel)

First announcement: ethics will not be discussed and rather be ignored today, because otherwise he would never finish

Quote "Let an ultraintelligent machine be defined as a machine that can far surpass all the intellectual activities of any man however clever. Since the design of machines is one of these intellectual activities, an ultraintelligent machine could design even better machines; there would then unquestionably be an “intelligence explosion,” and the intelligence of man would be left far behind. Thus the first ultraintelligent machine is the last invention that man need ever make, provided that the machine is docile enough to tell us how to keep it under control."

definition:

(I sadly didn't take notes here)

paths:

• AI -> strong AI

• whole brain emulation

• biological cognition (improve humans, chemical substances/nootropics, embryo selection -> lots of generations in vitro)

• interface with machine

• networks and organizations

forms:

• speed Superintellicence (like human but faster)

• collective Superintelligence (together smarter)

• quality Superintelligence (totally different to human)

intelligence explosion:

• there will be a crossover after which the machine itself will go on improving the system

• graph to show how similar an idiot and Einstein are compared to a mouse or Superintelligence

rate of intelligence change

AI takeover:

1. precritial

2. intelligence amplification

3. cover + preparation

4. overt implementation

-> arises the question of where the motivation to do so comes from!

the super intelligence will:

• the orthogonality thesis: any intelligence can build a pair with any motivation (Eliezer would probably call it "anti prediction)

Castes:

• oracles

• genies

• sovereign

• tool

predictability:

• design

• convergent instrumental goals (for example: self-preservation, goal-content integrity, cognitive enhancement, technology, resource acquisition -> infrastructure profusion)

other topics: failure modes, control problem, acquire values, choosing who to choose, do what I mean, the strategic picture

Comments I wrote down

by Andreas: "Does SI include self-awareness?" answer: "No"

by Ivan: Has someone tried to breed an human intelligent dog?

by Lio: Is it possible to have something that's good in all the things? -> Viliam: Yes, with a network structure

by Austin: Is there a difference between biological or programmed happiness? -> Manuel: the utility function of a thermostat is to be at 20 degrees, it doesn't WANT to be there

in the evening:

• We talked about daily things members of the group do which they consider rational/daily effective rituals/personal sleeping duration

for the next meetup:

• We will vote on the date in the Facebook group

• we will have three topics and the third one will be to collect personal goals we want to achieve with/in the group/ideas we have for the group