Benya

Results from MIRI's December workshop

Last week (Dec. 14-20), MIRI ran its 6th research workshop on logic, probability, and reflection. Writing up mathematical results takes time, and in the past, it's taken quite a while for results from these workshops to become available even in draft form. Because of this, at the December workshop, we tried something new: taking time during and in the days immediately after the workshop to write up results in quick and somewhat dirty form, while they still feel fresh and exciting. In total, there are seven short writeups. Here's a list, with short descriptions of each. Before you get started on these writeups, you may want to read John Baez's blog post about the workshop, which gives an introduction to the two main themes of the workshop. Theme 1: Scientific induction in mathematics One of the main themes was using Bayesian probability to represent uncertainty about mathematical statements. Like human mathematicians, an AI will be able to outright prove or disprove many mathematical statements, but there will also be many that it will be uncertain about, and the obvious thing to try to get a handle on dealing with such uncertainty is to assign a probability to each such statement. This would mean choosing some sort of prior, and then updating on evidence: For example, if you're not sure whether the twin prime conjecture is true, then each time you discover a new twin prime larger than all that you have seen before, you should ever so slightly increase the probability you assign to the conjecture. But what sort of prior should we choose? That's the problem that people at the December workshop tried to make some progress on. Here is an interesting problem which the best previous proposal, due to Abram Demski, fails on: Suppose that Q(x) is a predicate symbol, and suppose that you find evidence that Q(x) is true of exactly 90% of all numbers between x=1 and x=10^100. Then we would expect that if you plug in some arbitrary number in this range, say n = floor(7^7^7^7

73Jan 15, 2014

Benya

Message

1892

332

17y

The sin of updating when you can change whether you exist

Trigger warning: In a thought experiment in this post, I used a hypothetical torture scenario without thinking, even though it wasn't necessary to make my point. Apologies, and thanks to an anonymous user for pointing this out. I'll try to be more careful in the future. Should you pay up...

Feb 28, 201417

SUDT: A toy decision theory for updateless anthropics

The best approach I know for thinking about anthropic problems is Wei Dai's Updateless Decision Theory (UDT). We aren't yet able to solve all problems that we'd like to—for example, when it comes to game theory, the only games we have any idea how to solve are very symmetric ones—but...

Feb 23, 201427

I like simplicity, but not THAT much

Followup to: L-zombies! (L-zombies?) Reply to: Coscott's Preferences without Existence; Paul Christiano's comment on my l-zombies post In my previous post, I introduced the idea of an "l-zombie", or logical philosophical zombie: A Turing machine that would simulate a conscious human being if it were run, but that is never...

Feb 14, 201438

L-zombies! (L-zombies?)

Reply to: Benja2010's Self-modification is the correct justification for updateless decision theory; Wei Dai's Late great filter is not bad news "P-zombie" is short for "philosophical zombie", but here I'm going to re-interpret it as standing for "physical philosophical zombie", and contrast it to what I call an "l-zombie", for...

Feb 7, 201455

Results from MIRI's December workshop

Jan 15, 201473

Naturalistic trust among AIs: The parable of the thesis advisor's theorem

Eliezer and Marcello's article on tiling agents and the Löbian obstacle discusses several things that you intuitively would expect a rational agent to be able to do that, because of Löb's theorem, are problematic for an agent using logical reasoning. One of these desiderata is naturalistic trust: Imagine that you...

Dec 15, 201337

Meetup : Bristol meetup

Discussion article for the meetup : Bristol meetup WHEN: 20 October 2013 02:00:00PM (+0100) WHERE: Hodgkin House, 3 Meridian Place, Bristol BS8 1JG We'll have another meetup in Bristol this upcoming Sunday, October 20, at the student house where I live. We'll officially start at 2pm to hopefully make it...

Oct 15, 20138

Load More (7/17)

LESSWRONG
LW

LESSWRONG
LW

Benya

Benya

Benya

Results from MIRI's December workshop

A model of UDT with a concrete prior over logical statements

L-zombies! (L-zombies?)

An angle of attack on Open Problem #1

Benya

The sin of updating when you can change whether you exist

SUDT: A toy decision theory for updateless anthropics

I like simplicity, but not THAT much

L-zombies! (L-zombies?)

Results from MIRI's December workshop

Naturalistic trust among AIs: The parable of the thesis advisor's theorem

Meetup : Bristol meetup

Results from MIRI's December workshop

A model of UDT with a concrete prior over logical statements

L-zombies! (L-zombies?)

An angle of attack on Open Problem #1

The sin of updating when you can change whether you exist

SUDT: A toy decision theory for updateless anthropics

I like simplicity, but not THAT much

L-zombies! (L-zombies?)

Results from MIRI's December workshop

Naturalistic trust among AIs: The parable of the thesis advisor's theorem

Meetup : Bristol meetup