Chaos provides value both by telling certain engineers where not to look for solutions to their problems, and by getting their bosses off their back about it. That’s a significant value add, but short of what I was hoping for when I started looking into Chaos.
I don't think it's a value-add, because this sort of proof-by-intimidation abuse is how chaos theory gets used in many places, such as here on Lesswrong as well, not just engineers fighting their managers. Remember the proof that humans can't get high scores playing pinball because 'chaos theory'? It's just an indiscriminate rhetorical weapon. It is not true in the case of playing pinball, it is probably not true of trebuchets in general (as opposed to cheap simple trebuchets constructed for contests or the Third World), and I would be surprised if all of those 6 successful manipulations were the valid exceptions. It is similar to the pervasive abuse of Godel or the Halting theorem; you doubtless could successfully convince some managers to not bother with things like typechecking or unit-tests or formal proofs because "Turing proved it is impossible to prove things about arbitrary programs" etc, but that is not a good thing, it is a bad thing.
6
3
1I don't have any experience with actual situations where this could be relevant, but it does feel like you're overly focusing on the failure case where everyone is borderline incompetent and doing arbitrary things (which of course happens on less wrong sometimes, since the variation here is quite large!). There's clearly a huge upside to being able to spot when you're trying to do something that's impossible for theoretical reasons, and being extra sceptical in these situations. (E.g. someone trying to construct a perpetual motion machine). I'm open to the argument that there's a lot to be wished for in the way people in practice apply these things.
Remember the proof that humans can't get high scores playing pinball because 'chaos theory'?
Can you point to where the post says this? Because I read it as saying "It is impossible to predict a game of pinball for more than 12 bounces in the future" and "Professional pinball players try to avoid the parts of the board where the motion is chaotic."
See my comment. The problem with the post is revealed in the fourth sentence:
To demonstrate how chaos theory imposes some limits on the skill of an arbitrary intelligence, I will also look at a game: pinball.
Note that predicting a ball is not at all the same thing as skill in manipulating a ball. It's just a giant non sequitur being slipped in before he begins the math. Which is why he is 100% wrong when he concludes
This is not a problem that is solvable by applying more cognitive effort.
It totally is solvable. The 'cognitive effort' here is 'git gud at pinball, scrub, and stop making excuses for losing', and as he admits in the footnote he didn't include in the LW version, in real life, when adequately incentivized to win rather than find excuses involving 'well, chaos theory shows you can't predict ball bounces more than n bounces out', pinball pros learn how to win and rack up high scores despite 'muh chaos'.
And that is why I don't believe your anecdotal survey responses imply anything good. I think that several or all of those cases, if we were able to investigate them adequately, would turn out to be similar to this pinball essay: a lot of browbeating intimidation-by-math, possibly completely valid insofar as it went, but ultimately, proving an irrelevant claim and the problem in fact soluble.
2and as he admits in the footnote he didn't include in the LW version, in real life, when adequately incentivized to win rather than find excuses involving 'well, chaos theory shows you can't predict ball bounces more than n bounces out', pinball pros learn how to win and rack up high scores despite 'muh chaos'.
I was confused about this part of your comment because the post directly talks about this in the conclusion.
The strategy typically is to catch the ball with the flippers, then to carefully hit the balls so that it takes a particular ramp which scores a lot of points and then returns the ball to the flippers. Professional pinball players try to avoid the parts of the board where the motion is chaotic.
The "off-site footnote" you're referring to seems to just be saying "The result is a pretty boring game. However, some of these ramps release extra balls after you have used them a few times. My guess is that this is the game designer trying to reintroduce chaos to make the game more interesting again." which is just a minor detail. AFAICT pros could score lots of points even without the extra balls.
(I'm leaving this comment here because I was getting confused about whether there had been major edits to the post, since the relevant content is currently in the conclusion and not the footnote. I was digging through the wayback machine and didn't see any major edits. So trying to save other people from the same confusion.)
The "off-site footnote" you're referring to seems to just be saying "The result is a pretty boring game. However, some of these ramps release extra balls after you have used them a few times. My guess is that this is the game designer trying to reintroduce chaos to make the game more interesting again." which is just a minor detail. AFAICT pros could score lots of points even without the extra balls.
(I'm leaving this comment here because I was getting confused about whether there had been major edits to the post, since the relevant content is currently in the conclusion and not the footnote. I was digging through the wayback machine and didn't see any major edits. So trying to save other people from the same confusion.)
Absolutely right! The main problem is how we use the theories. They need to be applied in a way that people can understand, not confused or confounded
One reason to doubt chaos theory’s usefulness is that we don’t need fancy theories to tell us something is impossible. Impossibility tends to make itself obvious.
This claim seems really weird to me. Why do you think that's true? A lot of things we accomplished with technology today might seem impossible to someone from 1700. On the other hand, you could have thought that e.g. perpetuum mobile, or superluminal motion, or deciding whether a graph is 3-colorable in worst-case polynomial time, or transmitting information with a rate higher than Shannon-Hartley is possible if you didn't know the relevant theory.
4I agree with your examples and larger point. That was a gloss I was never happy with but can't quite bring myself to remove. I was hoping to articulate the reasons better than this but so far they're eluding me.
Yeah, I agree with the general point (don't have strong opinion about chaos theory at the moment).
First, negative results are really, really important. Mostly because they let you not lose your time trying to do something impossible, and sometimes they actually point you toward an answer. In general, conservation laws in physics have this role. And knowing what is undecidable is really important in formal methods, where the trick is generally to simplify what you want or the expressive power of your programs in order to sidestep it.
Then, they are indeed quite hard to prove, at least in non-trivial cases. Conservations laws are the results of literally centuries of reframing of classical mechanics and reduction, leading to seeing the importance of energy and potential in unifying everything in physics. Undecidability is the result of 60 years of metamathetical work trying to clean formalisms enough to be able to study these kind of properties.
The really hot take is that the really important boundary is between tractable P problems and (conjectured) intractable NP and more complicated problems, not the decidability boundary, because it is both far more relevant to what you can expect a computer to do in practice, and because at a meta level above, a lot of the historical results about incompleteness and the inability to define 1st order arithmetical truth in 1st order arithmetic can be proved based on other theorems that are relevant to computability theory.
As I like to say, computability theory is complexity theory when you allow for arbitrary memory and time and want to compare one resource/oracle with another, like nondeterminism and time travel, since even in the world where you can build arbitrarily large memory computers and can think for arbitrarily long, some resources are more useful than others to compute more problems.
But you are correct that intractability/impossibility results from a certain set of assumptions is really important and useful IRL.
- If a trebuchet requires you to solve the double pendulum problem (a classic example of a chaotic system) in order to aim, it is not a competition-winning trebuchet.
Ah, this is not quite the takeaway- and getting the subtlety here right is important for larger conclusions. If a simulating a trebuchet requires solving the double pendulum problem over many error-doublings, it is not a competition-winning trebuchet. This is an important distinction.
If you start with a simulator and a random assortment of pieces, and then start naively optimizing for pumpkin distance, you will quickly see the sort of design shown at 5:02 in the video, where the resulting machine is unphysical because its performance depends on coincidences that will go away in the face of tiny changes in initial conditions. This behaviour shows up with a variety of simulators and optimizers.
An expensive but probably effective solution is to perturb a design several times, simulate it several times, and stop simulation once the simulations diverge.
An ineffective solution is to limit the time of the solution, as many efficient and real-world designs take a long time to fire, because they begin with the machine slowly falling away from an unstable equilibrium.
The chaos-theory motivated cheap solution is to limit the number of rotations of bodies in the solution before terminating it, as experience shows error doublings tend to come from rotations in trebuchet-like chaotic systems.
The solution I currently have implemented at jstreb.hgreer.com is to only allow the direction of the projectile to rotate once before firing (specifically, it is released if it is moving upwards and to the right at a velocity above a threshold) which is not elegant, but seems mostly effective. I want to move to the "perturb and simulate several times" approach in the future.
1I enjoyed doing this interview. I haven’t done too much extemporaneous public speaking, and it was a weird but wonderful experience being on the other side of the youtube camera. Thanks Elizabeth!
1It seems to me that chaos control and anti-control is another non-application.
[Handbook of Chaos Control: Schöll, Eckehard, Schuster, Heinz Georg](https://www.amazon.com/Handbook-Chaos-Control-Eckehard-Sch%C3%B6ll/dp/3527406050)
I do want to note that a lot of the claimed unpredictability from chaos only works if you can measure stuff to a finite precision only, and while this is basically always true in practice, it is worth noticing, because if you did have the ability to have an infinite memory and infinite FLOP/s computer with infinitely precise measurement, like in Newtonian physics, chaos theory doesn't matter, because in a deterministic system, if you get the exact same input, it will always have the same output, so chaos doesn't matter.
To be clear, this isn't a practical way to beat chaos, but it is an exception to the rule that chaos makes a system unpredictable.
This is a good point! As a result of this effect and Jensen’s inequality, chaos is a much more significant limit on testing CUDA programs than for example cpp programs
Huang
Previously Alex Altair and I published a post on the applications of chaos theory, which found a few successes but mostly overhyped dead ends. Luckily the comments came through, providing me with an entirely different type of application: knowing you can’t, and explaining to your boss that you can’t.
Knowing you can’t
Calling a system chaotic rules out many solutions and tools, which can save you time and money in dead ends not traveled. I knew this, but also knew that you could never be 100% certain a physical system was chaotic, as opposed to misunderstood.
However, you can know the equations behind proposed solutions, and trust that reality is unlikely to be simpler[1] than the idealized math. This means that if the equations necessary for your proposed solution could be used to solve the 3-body problem, you don’t have a solution.
[[1] I’m hedging a little because sometimes reality’s complications make the math harder but the ultimate solution easier. E.g. friction makes movement harder to predict but gives you terminal velocity.]
I had a great conversation with trebuchet and math enthusiast Hastings Greer about how this dynamic plays out with trebuchets.
Transcript
Note that this was recorded in Skype with standard headphones, so the recording leaves something to be desired. I think it’s worth it for the trebuchet software visuals starting at 07:00
My favorite parts:
Explaining you can’t
One reason to doubt chaos theory’s usefulness is that we don’t need fancy theories to tell us something is impossible. Impossibility tends to make itself obvious.
But some people refuse to accept an impossibility, and some of those people are managers. Might those people accept “it’s impossible because of chaos theory” where they wouldn’t accept “it’s impossible because look at it”?
As a test of this hypothesis, I made a Twitter poll asking engineers-as-in-builds-things if they had tried to explain a project’s impossibility to chaos, and if it had worked. The final results were:
5/36 is by no means common, but it’s not zero either, and it seems like it usually works. My guess is that usage is concentrated in a few subfields, making chaos even more useful than it looks. My sample size isn’t high enough to trust the specific percentages, but as an existence proof I’m quite satisfied.
Conclusion
Chaos provides value both by telling certain engineers where not to look for solutions to their problems, and by getting their bosses off their back about it. That’s a significant value add, but short of what I was hoping for when I started looking into Chaos.