Heighn

Since its inception in 1960, Newcomb's Problem has continued to generate controversy. Here's the problem as defined by Yudkowsky & Soares: > An agent finds herself standing in front of a transparent box labeled “A” that contains $1,000, and an opaque box labeled “B” that contains either $1,000,000 or $0....

A reaction to omnizoid's latest post in this debate. omnizoid proposed a problem in an earlier post, which I steelmanned as follows: > * there's an agent who can (almost) perfectly predict whether people will cut off their legs once they exist > * this agent only creates people who...

omnizoid has replied to my critique of his "FDT is crazy" position here. This post is my response. > The most important argument against FDT is that, while it’s a fine account of what type of agent you want to be, at least, in many circumstances, it’s a completely terrible...

After omnizoid asked whether people want to debate him on Functional Decision Theory (FDT), he and I chatted briefly and agreed to have a (short) debate. We agreed the first post should be by me: a reaction to omnizoid's original post, where he explains why he believes FDT is "crazy"....

I just skimmed through On the Controllability of Artificial Intelligence, and am wondering if others have read it and what they think about it. It made me quite scared. In particular: is AI Alignment simply unsolvable/not fully solvable?

In the previous post, we applied some calculus to game theoretic problems. Let's now look at the famous Newcomb's problem, and how we can use calculus to find a solution. Newcomb's problem From Functional Decision Theory: A New Theory of Instrumental Rationality: > An agent finds herself standing in front...

Now that we know a bit about derivatives, it's time to use them to find dominant strategies and Nash equilibria. It helps if the reader is familiar with Nash equilibria already. Prisoner's dilemma The payoff matrix of the Prisoner's dilemma can be as follows: We can see that the payoff...