Control Theory Commentary
This is the third and final post in a sequence on control theory. In the first post I introduced the subject of control theory and stepped through some basics. In the second post I outlined Powers's model, as presented in Behavior: The Control of Perception. This post is a collection of comments on the subject that are only somewhat related, and so I'll use section headings to separate them. I'll also explicitly note the absence of a section on the design of control systems, which is where most of the effort used in talking about them in industrial settings goes, and is probably relevant to philosophical discussions surrounding them.
Behavior: The Control of Perception

This is the second of three posts dealing with control theory and Behavior: The Control of Perception by William Powers. The previous post gave an introduction to control theory, in the hopes that a shared language will help communicate the models the book is discussing. This post discusses the model introduced in the book. The next post will provide commentary on the model and what I see as its implications, for both LW and AI.
An Introduction to Control Theory
Behavior: The Control of Perception by William Powers applies control theory to psychology to develop a model of human intelligence that seems relevant to two of LW's primary interests: effective living for humans and value-preserving designs for artificial intelligence. It's been discussed on LW previously here, here, and here, as well as mentioned in Yvain's roundup of 5 years (and a week) of LW. I've found previous discussions unpersuasive for two reasons: first, they typically only have a short introduction to control theory and the mechanics of control systems, making it not quite obvious what specific modeling techniques they have in mind, and second, they often fail to communicate the differences between this model and competing models of intelligence. Even if you're not interested in its application to psychology, control theory is a widely applicable mathematical toolkit whose basics are simple and well worth knowing.
Because of the length of the material, I'll split it into three posts. In this post, I'll first give an introduction to that subject that's hopefully broadly accessible. The next post will explain the model Powers introduces in his book. In the last post, I'll provide commentary on the model and what I see as its implications, for both LW and AI.
Controlling your inner control circuits
On the topic of: Control theory
Yesterday, PJ Eby sent the subscribers of his mailing list a link to an article describing a control theory/mindhacking insight he'd had. With his permission, here's a summary of that article. I found it potentially life-changing. The article seeks to answer the question, "why is it that people often stumble upon great self-help techniques or productivity tips, find that they work great, and then after a short while the techniques either become ineffectual or the people just plain stop using them anyway?", but I found it to have far greater applicability than just that.
Richard Kennaway already mentioned the case of driving a car as an example where the human brain uses control systems, and Eby mentioned another: ask a friend to hold their arm out straight, and tell them that when you push down on their hand, they should lower their arm. And what you’ll generally find is that when you push down on their hand, the arm will spring back up before they lower it... and the harder you push down on the hand, the harder the arm will pop back up! That's because the control system in charge of maintaining the arm's position will try to keep up the old position, until one consciously realizes that the arm has been pushed and changes the setting.
Control circuits aren't used just for guiding physical sequences of actions, they also regulate the workings of our mind. A few hours before typing out a previous version of this post, I was starting to feel restless because I hadn't accomplished any work that morning. This has often happened to me in the past - if, at some point during the day, I haven't yet gotten started on doing anything, I begin to feel anxious and restless. In other words, in my brain there's a control circuit monitoring some estimate of "accomplishments today". If that value isn't high enough, it starts sending an error signal - creating a feeling of anxiety - in an attempt to bring that value into the desired range.
The problem with this is that more often than not, that anxiety doesn't push me into action. Instead I become paralyzed and incapable of getting anything started. Eby proposes that this is because of two things: one, the control circuits are dumb and don't actually realize what they're doing, so they may actually take counter-productive action. Two, there may be several control circuits in the brain which are actually opposed to each other.
Here we come to the part about productivity techniques often not working. We also have higher-level controllers - control circuits influencing other control circuits. Eby's theory is that many of us have circuits that try to prevent us from doing the things we want to do. When they notice that we've found a method to actually accomplish something we've been struggling with for a long time, they start sending an error signal... causing neural reorganization, eventually ending up at a stage where we don't use those productivity techniques anymore and solving the "crisis" of us actually accomplishing things. Moreover, these circuits are to a certain degree predictive, and they can start firing when they pick up on a behavior that only even possibly leads to success - that's when we hear about a great-sounding technique and for some reason never even try it. A higher-level circuit, or a lower-level one set up by the higher-level circuit, actively suppresses the "let's try that out" signals sent by the other circuits.
What is control theory, and why do you need to know about it?
This is long, but it's the shortest length I could cut from the material and have a complete thought.
1. Alien Space Bats have abducted you.
In the spirit of this posting, I shall describe a magical power that some devices have. They have an intention, and certain means available to achieve that intention. They succeed in doing so, despite knowing almost nothing about the world outside. If you push on them, they push back. Their magic is not invincible: if you push hard enough, you may overwhelm them. But within their limits, they will push back against anything that would deflect them from their goal. And yet, they are not even aware that anything is opposing them. Nor do they act passively, like a nail holding something down, but instead they draw upon energy sources to actively apply whatever force is required. They do not know you are there, but they will struggle against you with all of their strength, precisely countering whatever you do. It seems that they have a sliver of that Ultimate Power of shaping reality, despite their almost complete ignorance of that reality. Just a sliver, not a whole beam, for their goals are generally simple and limited ones. But they pursue them relentlessly, and they absolutely will not stop until they are dead.
You look inside one of these devices to see how it works, and imagine yourself doing the same task...
Alien Space Bats have abducted you. You find yourself in a sealed cell, featureless but for two devices on the wall. One seems to be some sort of meter with an unbreakable cover, the needle of which wanders over a scale marked off in units, but without any indication of what, if anything, it is measuring. There is a red blob at one point on the scale. The other device is a knob next to the meter, that you can turn. If you twiddle the knob at random, it seems to have some effect on the needle, but there is no fixed relationship. As you play with it, you realise that you very much want the needle to point to the red dot. Nothing else matters to you. Probably the ASBs' doing. But you do not know what moves the needle, and you do not know what turning the knob actually does. You know nothing of what lies outside the cell. There is only the needle, the red dot, and the knob. To make matters worse, the red dot also jumps along the scale from time to time, in no particular pattern, and nothing you do seems to have any effect on it. You don't know why, only that wherever it moves, you must keep the needle aligned with it.
Solve this problem.
That is what it is like, to be one of these magical devices. They are actually commonplace: you can find them everywhere.
= 783df68a0f980790206b9ea87794c5b6)
Subscribe to RSS Feed
= f037147d6e6c911a85753b9abdedda8d)