I've written quite a lot here since Less Wrong started up, but I've started to suspect that my writing style is holding me back. Most recently, I wrote two sequences that seemed to garner widespread agreement on content/significance/originality but didn't really seem to excite anyone, which is a pretty clear signal that my style has been hobbling my ideas. So, as I'd promised to do (albeit a few weeks later than I'd expected), I'm trying to improve myself as a writer, and I need your help.

I'm declaring Crocker's Rules on the subject, and I'd like help with both diagnosis and treatment. Let me know, as precisely as you can, what's problematic in my writing, or what you think the root causes might be, or what you think might help me to fix my issues. I'll list what I've thought of so far in a comment below (so that you can make your own suggestions without anchoring issues).

Links to my recent major posts:

Consequentialism Need Not Be Nearsighted

Qualia sequence: Part I, Part II, Part III

And now an odd counterexample: I wrote this post quickly for Discussion, without thinking too much or editing at all, and then it got promoted and was received enthusiastically. That may just be the subject matter, or it may signify that the time I spend editing posts makes them worse...

As promised, I'm posting the answers to the exercises I wrote in the decision theory post.

Exercise 1: Prove that if the population consists of TDT agents and DefectBots, then a TDT agent will cooperate precisely when at least one of the other agents is also TDT. (Difficulty: 1 star.)

With the utility function specified (i.e. the shortsighted one that only cares about immediate children), the TDT agent's decision can be deduced from simple superrationality concerns; that is, if any other TDT agents are present with analogous utility functions, then their decisions will be identical. Thus if a TDT faces off against 2 DefectBots, it chooses between 0 children (for C) and 2 children (for D), and thus it defects. If it faces off against a TDT and a DefectBot, it chooses between 3 children (for C) and 2 children (for D), so it cooperates. And if it faces off against two other TDTs, it chooses between 6 children (for C) and 2 children (for D), so it cooperates. (EDIT: It's actually not that simple after all- see Douglas Knight's comment and the ensuing discussion.)

Exercise 2: Prove that if a very large population starts with equal numbers of TDTs and DefectBots, then the expected population growth in TDTs and DefectBots is practically equal. (If Omega samples with replacement– assuming that the agents don't care about their exact copy's children– then the expected population growth is precisely equal.) (Difficulty: 2 stars.)

For the sake of simplicity, we'll consider only the parenthetical case. (The interested reader can see that if sampled without replacement, the figures will differ by a factor on the order of one divided by the population.) There are four cases to consider when Omega picks a trio: it includes 0, 1, 2 or 3 TDT agents, with probability 1/8, 3/8, 3/8 and 1/8 respectively. The first case results in 6 DefectBots being returned; the second results in 4 DefectBots and 2 TDTs; the third results in 8 DefectBots and 6 TDTs; the last results in 18 TDTs. Weighting and adding the cases, each "side" has expected population growth of 5.25 agents in that round.

Exercise 3:  Prove that if the initial population consists of TDTs and DefectBots, then the ratio of the two will (with probability 1) tend to 1 over time. (Difficulty: 3 stars.)

This is a bit tricky; note that the expected population growth is higher for the more populous side! However, the expected fertility of each agent is higher on the less populous side, and thus its share grows proportionally. (Think of the demographics of a small minority with high fertility- while they won't have as many total children as the rest of the population, their proportion of the population will increase.)

For very large populations, we can model the fraction of DefectBots as a differential equation, and we will show that this differential equation has a single stable attractive equilibrium at 1/2. Let N be the total population at a given moment, and x (in (0,1)) the fraction of that population consisting of DefectBots. Then we let P(x)=6x³+12x²(1-x)+24x(1-x)² and Q(x)=6x²(1-x)+18x(1-x)²+18(1-x)³ denote the expected population growth for DefectBots and TDTs, respectively (these numbers are arrived at in the same way we calculated Exercise 2 in the special case x=1/2), and note that the "difference quotient" between the old and new fractions of the population comes out to ((1-x)P(x)-xQ(x))/(N+P(x)+Q(x)). If we consider this to be x' and study the differential equation (with an extra parameter N for the current population), we see that indeed, x has a stable equilibrium when the expected fertilities are equal (that is, P(x)/x = Q(x)/(1-x)) at x=1/2, and that x steadily increases for x<1/2 and steadily decreases for x>1/2.

I'll admit that this isn't a rigorous proof, but it's the correct heuristic calculation; increased formalism only makes it more difficult to communicate.

Exercise 4: If the initial population consists of CliqueBots, DefectBots and TDTs in any proportion, then the ratio of both others to CliqueBots approaches 0 (with probability 1). (Difficulty: 4 stars.)

We can model this as a differential equation in the two-dimensional region {x+y+z=1: 0<x,y,z<1}, and as in Exercise 3 a stable equilibrium is a point at which the three expected fertilities are equal. At this point, it's worthwhile to note that you can calculate expected fertilities more simply by, for a given agent, counting only its individual fertility given the possible other two partners in the PD. If we let x, y and z denote the proportions of DefectBots, TDTs and CliqueBots, respectively, and let P, Q, and R denote their respective fertilities as functions of x,y and z, then we get

• P(x,y,z)=2x²+4xy+8y²+4xz+4yz+2z²
• Q(x,y,z)=2x²+6xy+6y²+4xz+6yz+2z²
• R(x,y,z)=2x²+4xy+8y²+4xz+4yz+6z²

It is simple to show that if we set x=0, then R>Q for all {(y,z):y+z=1,0<y,z<1}; that is, CliqueBots beat TDTs completely when DefectBots are not present. It is even simpler to show that they beat DefectBots entirely when TDTs are not present. It is a bit more complicated when all three are together: there is a tiny unstable region near (3/4,1/4,0) where Q>R, but the proportions drift out of this region as y gains against x, and they do not return; the stable equilibrium is at (0,0,1) as claimed.

Finally,

Problem: The setup looks perfectly fair for TDT agents. So why do they lose? (Difficulty: 2+3i stars.)

As explained in the consequentialism post, we've handicapped TDT by giving our agents shortsighted utility functions. If they instead care about distant descendants (let's say that Omega is only running the experiment finitely many times, either for a fixed number of tournaments or until the population reaches a certain number), then (unless it's known that the experiment is soon to end) the gains in population made by dominating the demographics will overwhelm the immediate gains of letting a DefectBot or CliqueBot take advantage of the agent. Growing almost 6-fold each time one's selected (or occupying a larger fraction of the fixed final population) will justify the correct decision, essentially via the more complicated calculations we've done above.

Prerequisites: Familiarity with decision theories (in particular, Eliezer's Timeless Decision Theory) and of course the Prisoner's Dilemma.

Summary: I show an apparent paradox in a three-agent variant of the Prisoner's Dilemma: despite full knowledge of each others' source codes, TDT agents allow themselves to be exploited by CDT, and lose completely to another simple decision theory. Please read the post and think for yourself about the Exercises and the Problem below before reading the comments; this is an opportunity to become a stronger expert at and on decision theory!

We all know that in a world of one-shot Prisoner's Dilemmas with read-access to the other player's source code, it's good to be Timeless Decision Theory. A TDT agent in a one-shot Prisoner's Dilemma will correctly defect against an agent that always cooperates (call this CooperateBot) or always defects (call this DefectBot, and note that CDT trivially reduces to this agent), and it will cooperate against another TDT agent (or any other type of agent whose decision depends on TDT's decision in the appropriate way). In fact, if we run an evolutionary contest as Robert Axelrod famously did for the Iterated Prisoner's Dilemma, and again allow players to read the other players' source codes, TDT will annihilate both DefectBot and CooperateBot over the long run, and it beats or ties any other decision theory.¹ But something interesting happens when we take players in threes...

I was reading a post on the economy from the political statistics blog FiveThirtyEight, and the following graph shocked me:

This, according to Nate Silver, is a log-scaled graph of the GDP of the United States since the Civil War, adjusted for inflation. What amazes me is how nearly perfect the linear approximation is (representing exponential growth of approximately 3.5% per year), despite all the technological and geopolitical changes of the past 134 years. (The Great Depression knocks it off pace, but WWII and the postwar recovery set it neatly back on track.) I would have expected a much more meandering rate of growth.

It reminds me of Moore's Law, which would be amazing enough as a predicted exponential lower bound of technological advance, but is staggering as an actual approximation:

I don't want to sound like Kurzweil here, but something demands explanation: is there a good reason why processes like these, with so many changing exogenous variables, seem to keep right on a particular pace of exponential growth, as opposed to wandering between phases with different exponents?

EDIT: As I commented below, not all graphs of exponentially growing quantities exhibit this phenomenon- there still seems to be something rather special about these two graphs.

Previously: Seeing Red: Dissolving Mary's Room and Qualia, A Study of Scarlet: The Conscious Mental Graph

When we left off, we'd introduced a hypothetical organism called Martha whose actions are directed by a mobile graph of simple mental agents. The tip of the iceberg, consisting of the agents that are connected to Martha's language centers, we called the conscious subgraph. Now we're going to place Martha into a situation like Mary's Room: we'll say that a large unconscious agent of hers (like color vision) has never been active, we'll grant her an excellent conscious understanding of that agent, and then we'll see what happens when we activate it for the first time.

But first, there's one more mental agent we need to introduce, one which serves a key purpose in Martha's evolutionary history: a simple agent that identifies learning.

Sequel to: Seeing Red: Dissolving Mary's Room and Qualia

Seriously, you should read first: Dissolving the Question, How an Algorithm Feels From Inside

In the previous post, we introduced the concept of qualia and the thought experiment of Mary's Room, set out to dissolve the question, and decided that we were seeking a simple model of a mind which includes both learning and a conscious/subconscious distinction. Since for now we're just trying to prove a philosophical point, we don't need to worry whether our model corresponds well to the human mind (though it would certainly be convenient if it did); we'll therefore pick an abstract mathematical structure that we can analyze more easily.

Essential Background: Dissolving the Question

How could we fully explain the difference between red and green to a colorblind person?

Well, we could of course draw the analogy between colors of the spectrum and tones of sound; have them learn which objects are typically green and which are typically red (or better yet, give them a video camera with a red filter to look through); explain many of the political, cultural and emotional associations of red and green, and so forth... but it seems that the actual difference between our experience of redness and our experience of greenness is something much harder to convey. If we focus in on that aspect of experience, we end up with the classic philosophical concept of qualia, and the famous thought experiment known as Mary’s Room¹.

Mary is a brilliant neuroscientist who has been colorblind from birth (due to a retina problem; her visual cortex would work normally if it were given the color input). She’s an expert on the electromagnetic spectrum, optics, and the science of color vision. We can postulate, since this is a thought experiment, that she knows and fully understands every physical fact involved in color vision; she knows precisely what happens, on various levels, when the human eye sees red (and the optic nerve transmits particular types of signals, and the visual cortex processes these signals, etc).

One day, Mary gets an operation that fixes her retinas, so that she finally sees in color for the first time. And when she wakes up, she looks at an apple and exclaims, "Oh! So that's what red actually looks like."²

Now, this exclamation poses a challenge to any physical reductionist account of subjective experience. For if the qualia of seeing red could be reduced to a collection of basic facts about the physical world, then Mary would have learned those facts earlier and wouldn't learn anything extra now– but of course it seems that she really does learn something when she sees red for the first time. This is not merely the god-of-the-gaps argument that we haven't yet found a full reductionist explanation of subjective experience, but an intuitive proof that no such explanation would be complete.

The argument in academic philosophy over Mary's Room remains unsettled to this day (though it has an interesting history, including a change of mind on the part of its originator). If we ignore the topic of subjective experience, the arguments for reductionism appear to be quite overwhelming; so why does this objection, in a domain in which our ignorance is so vast³, seem so difficult for reductionists to convincingly reject?

Veterans of this blog will know where I'm going: a question like this needs to be dissolved, not merely answered.

Or: On Truth and Morality in a Non-moral Sense

Not even Nietzsche (except perhaps once he descended into madness) would claim that Nietzsche's philosophy can be understood as a coherent whole, or that the fundamental questions he asked originated with him. And yet his writings have had a far-reaching influence on modern ideas of truth– the type of skepticism or cynicism of his descendants is more pressing and simply more fun than that engendered by Hume or others.

One group of his erstwhile followers focus on what I think is one of his better ideas, which has earned the title of perspectivism. While other philosophers of the time naively went about constructing theories of how the brain absorbed true knowledge through the senses (with an admixture of regrettable errata), Nietzsche drew an analogy from his first academic specialty: philology, or the study and interpretation of ancient texts. From the same text, two different scholars could draw opposite conclusions about the meaning of a word or phrase, because each entered with a larger contextual scheme in which the new text had to fit. In the arena of the mind, he suggested, different interpretations existed and gave different meanings to the same external data. An optical illusion, for instance, could be simultaneously read as a 3-dimensional image and a 2-dimensional illusion.

Had Nietzsche stopped with that analogy, it would have been just one more argument for epistemological relativism; and indeed, plenty of modern people read him with just that interpretive scheme in mind. But there was one key further ingredient: not all interpretations were created equal, and they vied with one another for dominance at every moment. And where they clash, some are stronger than others.

Notice that I didn't say that some were better or truer than others- that would be begging the question at this point.

Consider the case of a smart young woman who's been raised by young-earth creationists and has just stumbled onto some contrary information online.

Now let's consider what might happen once she goes to college.

The key is to understand the different levels of our interpretive framework, rather like distinguishing the different levels of causality in an evolutionary adaptation. The time to recoil from horror that some of the reasons involved are irrational or immoral is after, not before, understanding the nature of the rules.

The original essay, On Truth and Lies in a Non-Moral Sense, merits similar caution; it is not celebrating the limitations of the mind, as it might appear on a first reading. Nor is it self-negating for being an intellectual critique of thought itself. A perspectivist account of the mind doesn't undermine itself, so long as it also contains some inkling that true ideas can have advantages in the arena of the mind.

Rational thought is interpretation according to a scheme that we cannot throw off. (Nietzsche, The Will to Power)

Originally a chaos of ideas. The ideas that were consistent with one another remained, the greater number perished—and are perishing. (Nietzsche, The Will to Power)

Division of the problem into the logical and the moral.

Part I: Assuming we can trust our science

Part II: Can we trust our science?

Part III: What about Morality?