xamdam comments on Priors as Mathematical Objects - Less Wrong

24 Post author: Eliezer_Yudkowsky 12 April 2007 03:24AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (19)

Sort By: Old

You are viewing a single comment's thread.

Comment author: xamdam 29 June 2010 01:40:43PM 0 points [-]

Then the task is just to turn this new distribution into a computer program, which turns out not to be difficult.

Can someone please provide a hint how?

Comment author: alex_zag_al 04 December 2013 07:35:39PM *  1 point [-]

Here's some Python code to calculate a prior distribution from a rule for assigning probability to the next observation.

A "rule" is represented as a function that takes as a first argument the next observation (like "R") and as a second argument all previous observations (a string like "RRWR"). I included some example rules at the end.

EDIT: oh man, what happened to my line spacing? my indents? jeez.

EDIT2: here's a dropbox link: https://www.dropbox.com/s/16n01acrauf8h7g/prior_producer.py

from functools import reduce
def prod(sequence):
'''Product equivalent of python's "sum"'''
return reduce(lambda a, b: a*b, sequence)
def sequence_prob(rule, sequence):
'''Probability of a sequence like "RRWR" using the given rule for
computing the probability of the next observation.
To put it another way: computes the joint probability mass function.'''
return prod([rule(sequence[i], sequence[:i]) \
for i in range(len(sequence))])
def number2sequence(number, length):
'''Convert a number like 5 into a sequence like WWRWR.
The sequence corresponds to the binary digit representation of the number: 5 --> 00101 --> WWRWR
This is convenient for listing all sequences of a given length.'''
binary_representation = bin(number)[2:]
seq_end = binary_representation.replace('1', 'R').replace('0', 'W')
if len(seq_end) > length:
raise ValueError('no sequence of length {} with number {}'\
.format(length, number))
# Now add W's to the beginning to make it the right length - # like adding 0's to the beginning of a binary number
return ''.join('W' for i in range(length - len(seq_end))) + seq_end
def prior(rule, n):
'''Generate a joint probability distribution from the given rule over
all sequences of length n. Doesn't feed the rule any background
knowledge, so it's a prior distribution.'''
sequences = [number2sequence(i, n) for i in range(2**n)]
return [(seq, sequence_prob(rule, seq)) for seq in sequences]

And here's some examples of functions that can be used as the "rule" arguments.

def laplaces_rule(next, past):
R = past.count('R')
W = past.count('W')
if R + W != len(past):
raise ValueError('knowledge is not just of red and white balls')
red_prob = (R + 1)/(R + W + 2)
if next == 'R':
return red_prob
elif next == 'W':
return 1 - red_prob
else:
raise ValueError('can only predict whether next will be red or white')
def antilaplaces_rule(next, past):
return 1 - laplaces_rule(next, past)