Steve_Rayhawk comments on What is control theory, and why do you need to know about it? - Less Wrong

40 Post author: RichardKennaway 28 April 2009 09:25AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (47)

You are viewing a single comment's thread.

Comment author: Steve_Rayhawk 29 April 2009 03:27:18AM *  17 points [-]

I have not studied control theory, but I think a PID controller may be the Bayes-optimal controller if:

  • the system is a second-order linear system with constant coefficients,
  • the system is controllable,
  • all disturbances in the system are additive white noise forcing terms,
  • there is no noise in perception,
  • the cost functional is the integral of the square of the error,
  • the time horizon is infinite in both directions (no transients), and
  • the prior belief distribution over possible reference signals is the same as if the reference signal was a Brownian motion (which needs first-order control) plus an integral of a Brownian motion (which needs second-order control).

What makes it a full Bayesian decision problem is the prior belief distribution over possible reference signals. At each time, you don't know what the future reference signal is going to be, but you have a marginal posterior belief distribution over possible future reference signals given what the reference signal has been in the past. Part of this knowledge about possible future reference signals is represented in the state of the system you have been controlling, and part of it is represented in the state of the I element of the controller. You also don't know what the delayed effects of past disturbances will be, but you have a marginal posterior belief distribution over possible future delayed effects given what the perception signal has been in the past. Part of this knowledge is also represented in the system and in the I element. (Not all of your knowledge about possible future reference signals and possible future delayed effects of past disturbances is represented, only your knowledge about possible future differences between them.) This representation is related to sufficient statistics ("sufficiency is the property possessed by a statistic . . . when no other statistic which can be calculated from the same sample provides any additional information") and to updating of the parameter for a parametric family of belief distributions.

In a real engineering problem, the true belief about expected possible reference signals would be more specific than a belief of a random Brownian motion. But if a reference signal would not be improbable for Brownian motion, then a PID controller can still do well on that reference signal.

I think these conditions are sufficient but not necessary. If I knew control theory I would tell you more general conditions. If the cost functional has a term for the integral of the squared control signal, then a PID controller may not be optimal without added filters to keep the control signal from having infinite power.

Example 6.3-1 in Optimal Control and Estimation by Robert F. Stengel (1994 edition, pp. 540-541) is about PID controllers as optimal regulators in linear-quadratic-Gaussian control problems.

I see optimal control theory as the shared generalization of Bayesian decision networks and dynamic Bayesian networks in the continuous-time limit. (Dynamic Bayesian networks are Bayes nets which model how variables change over discretized time steps. When the time step size goes to zero and the variables are continuous, the limit is stochastic differential equations such as the equations of Brownian motion. When the time step size goes to zero and the variables are discrete, the limit is almost Uri Nodelman's continuous-time Bayesian networks. Bayesian decision networks are Bayes nets which represent a decision problem and contain decision nodes, utility nodes, and information arcs.)

Comment author: Steve_Rayhawk 30 April 2009 02:00:28AM *  1 point [-]

(Not all of your knowledge about possible future reference signals and possible future delayed effects of past disturbances is represented, only your knowledge about possible future differences between them.)

So this isn't a sufficient statistic, it's only a sufficient-for-policy-implications statistic. Is there a name for that?

Comment author: Eliezer_Yudkowsky 27 June 2009 05:32:26PM -1 points [-]

All "sufficient" statistics are only "sufficient" for some particular set of policy or epistemic implications. You could always care about the number of 1 bits, if you're allowed to care about anything.

Comment author: Steve_Rayhawk 28 June 2009 02:41:11AM *  2 points [-]

Then every "sufficient-for-policy-implications" statistic can become a "sufficient-for-implications-for-beliefs-about-the-future" statistic, under a coarsening of the sample space by some future-action-preserving and conditional-ratios-of-expected-payoff-differences-preserving equivalence relation?

(Would we expect deliberative thinking and memory to physically approximate such coarsenings, as linear controllers do?)

Comment author: RichardKennaway 29 April 2009 11:30:37PM 0 points [-]

Thank you for those references -- exactly the sort of thing I've been looking for.