LESSWRONG
LW

Comment Permalink
OpenThreadGuy70

I was there.

"but most of our time was spent thinking about probability distributions on consistent theories" seems incorrect to me, unless I'm a lot more confused than I thought I was. What I saw was that we spent an equal amount of time on essentially three topics:

  1. Trying to manufacture a truth-predicate weaker than provability but still able to do work, for getting around Löb's theorem;
  2. Binary logical versions/formulations of Solomonoff Induction (Abram's prior);
  3. What to do if you have an inconsistent set of probabilities over sentences / how to turn inconsistent probabilities into a prior (probability distributions on consistent theories).

(The hosts then talked about #3 for seven more hours after the official end of the mini-workshop. Scott apparently spent n hours after that writing down what he had talked and thought about. And then it was a dead end. Таких математике!)

Despite my lack of math knowledge, I enjoyed attending. I am even told that I did not distract everyone with my stupid questions and uninteresting ideas, so I will go to future ones as well.

The other participants were madmen. They didn't take any breaks at all, only furiously bashed brains against unsolvable problems, demanding solutions from the aether and holding their ideas hostage until some otherworldly god, some invisible champion of mathematics relented to give them what they want. They did not succeed, nor did they fail, for they have not given up. To tell the truth, it was a little bit frightening.

I got all excited about math because of this, which may or may not be a positive effect, and hopefully next time I'll have read some of the relevant papers, and understood more than a modicum.

Reply
Scott Garrabrant20

Both 2 and 3 fall under what I said we spent most of the time doing. Perhaps "majority" would have been a better word.

Reply
See in context
by Scott Garrabrant
2 min read
8

13

Last Saturday, nine people met for the Southern California FAI Workshop. Unsurprisingly, we did not come up with any major results, but I know some people were curious about this experiment, so I a providing a summary anyway. 

First, I would like to say that I consider this first meeting a success. The turnout was higher than I expected. We had 9 participants, and there were 2 other people who did not show up due to scheduling conflicts. We basically stayed on topic the entire 7 hours from 10:00 to 5:00, and then we had dinner at 5:00, generously provided by MIRI. We will be hosting these workshops again. In fact, we have decided to hold them monthly. We will probably follow a schedule of meeting the first Saturday of each month, starting in June. I will make another post announcing the second meetup once this date is finalized. 

We talked about various ideas participants had about FAI, but most of our time was spent thinking about probability distributions on consistent theories. One thing we observed that if you view the space of all probability assignments to logical sentences as living inside the vector space of all functions from sentences to the real numbers, then the collection of coherent probability assignments (those which correspond to probability distributions on consistent theories), is an affine subspace. This is exciting, because we can set up an inner product on this vector space and orthogonally project probability assignments onto the closest point on this subspace (i.e. find a nearby coherent probability assignment to a given probability assignment). Further, while this projection is not computable, there exists a computable procedure which converges to this point. However, I am now convinced that this idea is a dead end, for the following reason: Just because the point you start with has all coordinates between 0 and 1, does not mean that the projection to the subspace containing coherent assignments still has all coordinates between 0 and 1. (Imagine a 3d unit cube, and imagine that a theory is coherent if x+y+z=1. If you project 1,1,0 onto this subspace, you get 2/3,2/3,-1/3, which is not a valid probability assignment) I am now convinced that this idea will not be fruitful.

However, we did get several good things out of the meeting. First, we introduced several new mathy people to the problems associated with FAI. Second, we set up an email list, so that we can bounce ideas we have off of people that we know personally and who are interested in this stuff. Third, and most importantly we have become excited about doing more. I personally spent most the day after the workshop writing up lots of stuff related to what we observed above (This was before I discovered that it did not work), and I know I am not the only one to have this reaction.

Thanks to all of the participants, and please let me know if you would be interested in joining us next time!

Personal Blog

13

Summary of the first SoCal FAI Workshop
New Comment
8 comments, sorted by
top scoring
Click to highlight new comments since:
[-]OpenThreadGuy70

I was there.

"but most of our time was spent thinking about probability distributions on consistent theories" seems incorrect to me, unless I'm a lot more confused than I thought I was. What I saw was that we spent an equal amount of time on essentially three topics:

  1. Trying to manufacture a truth-predicate weaker than provability but still able to do work, for getting around Löb's theorem;
  2. Binary logical versions/formulations of Solomonoff Induction (Abram's prior);
  3. What to do if you have an inconsistent set of probabilities over sentences / how to turn inconsistent probabilities into a prior (probability distributions on consistent theories).

(The hosts then talked about #3 for seven more hours after the official end of the mini-workshop. Scott apparently spent n hours after that writing down what he had talked and thought about. And then it was a dead end. Таких математике!)

Despite my lack of math knowledge, I enjoyed attending. I am even told that I did not distract everyone with my stupid questions and uninteresting ideas, so I will go to future ones as well.

The other participants were madmen. They didn't take any breaks at all, only furiously bashed brains against unsolvable problems, demanding solutions from the aether and holding their ideas hostage until some otherworldly god, some invisible champion of mathematics relented to give them what they want. They did not succeed, nor did they fail, for they have not given up. To tell the truth, it was a little bit frightening.

I got all excited about math because of this, which may or may not be a positive effect, and hopefully next time I'll have read some of the relevant papers, and understood more than a modicum.

Reply
[-]Scott Garrabrant20

Both 2 and 3 fall under what I said we spent most of the time doing. Perhaps "majority" would have been a better word.

Reply
[-]Punoxysm20

However, I am now convinced that this idea is a dead end, for the following reason: Just because the point you start with has all coordinates between 0 and 1, does not mean that the projection to the subspace containing coherent assignments still has all coordinates between 0 and 1. (Imagine a 3d unit cube, and imagine that a theory is coherent if x+y+z=1. If you project 1,1,0 onto this subspace, you get 2/3,2/3,-1/3, which is not a valid probability assignment) I am now convinced that this idea will not be fruitful.

Why is this an issue? Just project onto the simplex (sum x_1... x_n =1, x_1 >=0 ... x_n >= 0) instead of the affine subspace. This is perfectly possible in O(n) time Duchi '08. You can add many more constraints and still make efficient projections.

But I have to admit I'm confused about what use this is. What is the application supposed to be, and why is simply dividing by the sum of probabilities insufficient?

Reply
[-]Scott Garrabrant40

There are 2 issues. First, the actual situation is an infinite dimension vector space, so it is not clear that that result applies. Second, the idea was to take advantage of properties of projection in order to converge to the projection as a series of local moves, and I don't see a way to do that with a simplex.

The goal was to use this projection and apply it to the point that assigns 1/2 to all statements, in order to get a coherent probability distribution on sentences and a simple procedure that converges to it, so that we can look at the properties of this distribution.

It also could have given an answer to the question: How should I correct if I observe that my probability assignment is incoherent.

Reply
[-]AlexMennen00

The basis vectors do not in general correspond to a set of propositions exactly one of which is true, but merely a set of propositions with some set of logical constraints (e.g. if x, y, and z are all logically equivalent, then the assignment is coherent iff x=y=z. The situation where the assignment is coherent iff x+y+z=1 was just an example).

Reply
[-]Scott Garrabrant00

This is correct. My response was thinking he was saying project onto the polytope, (which is not always a simplex), which is a good suggestion, but doesn't work in terms of local moves.

Reply
[-]Punoxysm00

This modified simplex, where some coordinates are constrained to be equal, shouldn't be impossible to project on, but it does seem like you have larger problems. Ah well.

But I figure there should be some approach that works spiritually with whatever you're trying to do (which is not 100% clear to me) just because there are so many ways to project vectors, or sampled versions of those vectors.

What specifically do you hope to gain from your probability distribution over 1/2 probability on all sentences?

Reply
[-]Squark20

Is there a way to join the email list? How about making it a google group?

Reply
Moderation Log
More from Scott Garrabrant
View more
Curated and popular this week
8Comments