Spaghetti Towers
A brief authorial take - I think this post has aged well, although as with Caring Less (https://www.lesswrong.com/posts/dPLSxceMtnQN2mCxL/caring-less), this was an abstract piece and I didn't make any particular claims here.
I'm so glad that A) this was popular B) I wasn't making up a new word for a concept that most people already know by a different name, which I think will send you to at least the first layer of Discourse Hell on its own.
I've met at least one person in the community who said they knew and thought about this post a lot, well before they'd met me, which was cool.
I think this website doesn't recognize the value of bad hand-drawn graphics for communicating abstract concepts (except for Garrabrant and assorted other AI safety people, whose posts are too technical for me to read but who I support wholly.) I'm guessing that the graphics helped this piece, or at least got more people to look at it.
I do wish I'd included more examples of spaghetti towers, but I knew that before posting it, and this was an instance of "getting something out is better than making it perfect."
I've planned on doing followups in the same sort of abstract style as this piece, like methods I've run into for getting around spaghetti towers. (Modularization, swailing, documentation.) I hopefully will do that some day. If anyone wants to help brainstorm examples, hit me up and I may or may not get back to you.
I would call this a good visual representation of technical debt. I like to think of it as chaining lots of independently reasonable low order approximations until their joint behavior becomes unreasonable.
It's basically fine to let this abstraction be a little leaky, and it's basically reasonable to let that edge case be handled clumsily, and it's basically acceptable to assume the user won't ever give this pathological input, etc., until the number of "basically reasonable" assumptions N becomes large enough that 0.99^N ends up less than 0.5 (or some other unacceptably low probability of success). And even with a base as high as 0.99, the N that breaks 50% is only ~70!
The visual depiction of this as parts being stacked such that each additional part is placed in what looks to be a reasonable way but all the parts together look ridiculously fragile is excellent! It really emphasizes that this problem mode can only be understood with a global, rather than a local or incremental, view.
See https://en.wikipedia.org/wiki/Big_ball_of_mud
There is a rich literature on design anti-patterns and the reasons they exist and survive.
Agree that biology looks like a classic legacy system, but worse:
3.5B years old
No documentation
No source code
No source code
I get the intended meaning, but I would like to made the words a little more precise. While we can find the executable source code (DNA) for an organism, that DNA is far from a high-level language.
Some more examples here: https://mikehadlow.blogspot.com/2014/12/the-lava-layer-anti-pattern.html
“Can you just straighten out the yellow one without touching any of the others? Thanks.”
https://en.wikipedia.org/wiki/Recurrent_laryngeal_nerve#Evidence_of_evolution
This idea has become part of my conceptual toolkit for discussing / describing a key failure mode.
(Note: the below part of the comment is from my Facebook comment about the article when it came out.)
There's a great connection you make to bureaucracy, and it's definitely worth exploring.
This gives me a good language to discuss something I've noted a number of times. I'd posit that selection pressure for bureaucracy limits how stupid the system gets as a function of the best simple alternative, and the difficulty of transitioning to it without turning off the system. This means that for critical systems where there is no incremental pathway to improve, it's near-permanent even if there are better alternatives - see the US healthcare system. For less critical systems, once an alternative is found, as long as the transition isn't too long/too expensive, and there are incentives that actually promote efficiency, it will happen. The critical fact is that the incentives need to exist for everyone that is involved - not just the end user. So if bob in accounting doesn't like the change, unless someone else can induce cooperation (like senior management,) it never happens.
I wonder to what extent tax codes are spaghetti towers - every time someone finds a loophole a new bit of tax code gets added to close it without considering how to make a coherent whole. This would explain how the uk tax code runs to >10,000 pages
Having looked into this, it's only partly an issue of spaghetti towers. The more important reason that you get absurd page counts, however, is that tax codes are written in legalese. I saw the same thing happen when I worked in equity derivatives pricing, where a simple options contract for a call can easily be described in (at most) 10 lines of code, or expressed as a one-line equation. Instead, however, the legal terms are actually written as this 188 page pamphlet; https://www.theocc.com/components/docs/riskstoc.pdf which is (technically but not enforced to be a) legally required reading for anyone who wants to purchase an exchange traded option. And don't worry - it explicitly notes that it doesn't cover the actual laws governing options, for which you need to read the relevant US code, or the way in which the markets for trading them work, or any of the risks.
I grant that legalese increases the total page count, but I don't think it necessarily changes the depth of the tree very much (by depth I mean how many documents refer back to other documents).
I've seen spaghetti towers written in very concise computer languages (such as Ruby) that nevertheless involve perhaps 50+ levels (in this context, a level is a function call).
Agree that it's possible to have small amounts of code describing very complex things, and I said originally, it's certainly partly spaghetti towers. However, to expand on my example, for something like a down-and-in European call option, I can give you a two line equation for the payout, or a couple lines of easily understood python code with three arguments (strike price, min price, final price) to define the payout, but it takes dozens of pages of legalese instead.
My point was that the legal system contains lots of that type of what I'd call fake complexity, in addition to the real complexity from references and complex requirements.
I also thought of tax, and I think it is probably a good example. What is especially confusing about tax is that, for some reason, whenever anyone has an idea they add on a new bit, rather than removing an old one.
ISAs are an obvious example of this. They basically introduce a subsystem that bypasses the normal taxation system in a limited way. But a similar effect could have been introduced much more simply by just raising the tax threshold in the main system.
Extending that the legislation and law in general seems appropriate as well. I wonder if one contrasts and compares slice of time views of both common law regimes and statutory law regimes if there would be any differences.
Suddenly very inspired with the idea of a programming language where even the most carelessly constructed spaghetti towers are fairly easy to refactor.
I want to design a medium that can be assembled into a fairly decent set of shelves by just progressively throwing small quantities of it at a wall.
I want to design tools that do not require foresight to be used well, because if you're doing something that's new to you, that challenges you- and I want to live in a world where most people are- foresight is always in scarce supply.
In my experience, programming languages with {static or strong} typing are considerably easier to refactor in comparison to languages with {weak or dynamic} typing.*
* The {static vs dynamic} and {strong vs weak} dimensions are sometimes blurred together, but this Stack Overflow Q&A unpacks the differences pretty well.
Promoted to curated: I've wanted to curate this for a while, mostly because it introduces a concept in a nice and concise way, while also connecting it to existing concepts and scientific disciplines. I've also found the images pretty useful and good, and I do think we have a lack of images, and see this as a good example of low-effort images increasing the quality of a post by a lot.
Also "Normal accidents" by Perrow may be relevant: spaghetti towers have some natural rate of crashes, not too often, as this will not allow them to be built so high, but also not very rare, as they are not designed to be stable.
This has been a useful concept, one which I've continued referring to afterwards, e.g. in the context of my multi-agent mind posts.
(somewhat annoying request: can you post this as a top level "official nomination" linking to this comment, if that's your intent? I realize it's a bit repetitive, but I'd rather spend marginal dev-time working on the UI for the Review Phase than improving the nomination UI, since it'll only be used for a week, and meanwhile this will make it easier to check how many nominations each post has)
Where it would theoretically be nice if every function was neatly ordered
...but another problem is this: when you build a bridge, you know in advance what you need and what you don't. Nobody (sane) will demand that you add a platform for selling Coca-Cola and a pier at the bottom with an elevator to ground level,and whatnot mid-construction. In programming, you either work for military where you will get proper specification several inches thick, or you start developing and try to accommodate for sudden requirement changes on the fly. Good programmer is better at predicting which changes are more likely and designs their code so that those are relatively easy to implement. But when redesigning from scratch is not an option and refusing to change is not an option - spaghetti tower becomes the most viable option left.
Let's say we want to create a program to perform a particular task T. Imagine you are constructing a program A by just throwing random things on the wall. The output of A is a program. So we run A and get a program B. We now see how well B performs on T. That tells us how well a change to A sticks.
Now assume that T is quite complicated and that A is random, i.e. it has no structure to it. Would you expect in that circumstance that B has no structure to it? To me, it seems quite likely that B will have a lot of regularity to it. It will not be good code from the human perspective, but there will be a lot of structure I think, simply because that structure is in T and the environment. If there are two obvious actions in the environment A and B, and a simple solution to T is to perform a sequence of these two actions depending on the context, then it seems kind of likely that partial solutions to this factorization would stick better to A.
If you try random things you are not actually accepting all random changes. You only accept changes that match the structure of the problem you are solving. You might often find a matching structure that no human designer would choose because it is not the easiest understandable thing. But to me, it seems that the program that is produced by the spaghetti test will be closer to a program a human designer would come up with than with a program sampled randomly. If we only consider all the programs that solve T, we will still be closer to a human interpretable program, than the "average program" that solves T.
That is somewhat vague, but I have a semi-strong intuition that something like this holds. Even in some non-trivial cases. In the case where we just consider all possible programs that solve T it kind of holds trivially. Then we would obviously be very close to the human-designed program, as there are programs of arbitrary size without dead code (in a weak sense) that solve T.
Now I made the assumption that A has no structure to it. But I am not sure that this actually holds in general. Throwing random things on the wall, and seeing if they stick to generate a program of size , doesn't seem equivalent to just randomly sampling from all programs of size .
That's true, but Program B will still be worse than a human-written program, so we aim to avoid spaghetti towers.
Spaghetti towers work especially poorly in changing environments: if evolution were reasonable, it would force us to try to maximize the number of our genes in the next generation. But instead, she created several heuristics like hunger and desire for groin friction. So when people came up with civilization, we started eating fast food and having sex with condoms.
First, thank you for writing this. I would ask that you continue to think & refine and share back what you discover, prove, or disprove.
To me, it seems quite likely that B will have a lot of regularity to it. It will not be good code from the human perspective, but there will be a lot of structure I think, simply because that structure is in T and the environment.
I'm interested to see if we can (i) do more than claim this is likely and (ii) unpack reasons that might require that it be the case.
One argument for (ii) would go like this. Assume the generating process for A has a preference for shorter length programs. So we can think of a A as a tending to find shorter description lengths that match task T.
Claim: shorter (and correct) descriptions reflect some combination of environmental structure and compression.
- by 'environmental structure' I mean the laws underlying the task.
- by 'compression' I mean using information theory embodied in algorithms to make the program smaller
I think this claim is true, but let's not answer that too quickly. I'd like to probe this question more deeply.
- Are there more than two factors (environmental structure & compression)?
- Is it possible that the description gets the structure wrong but makes up for it with great compression? I think so. One can imagine a clever trick by which a small program expands itself into something like a big ball of mud that solves the task well.
- Any expansion process takes time and space. This makes me wonder if we should care not only about description length but also run time and space. If we pay attention to both, it might be possible to penalize programs that expand into a big ball of mud.
- However, penalizing run time and space might be unwise, depending on what we care about. One could imagine a program that start with first principles and derives higher-level approximations that are good enough to model the domain. It might be worth paying the cost of setting up the approximations because they are used frequently. (In other words, the amortized cost of the expansion is low.)
- Broadly, what mathematical tools can we use on this problem?
I seem to work (and possible live) in such a setting ;-)
On one level I wonder how much we need to worry. The idea behind that comment is the problem we all face with the asymmetric nature of our knowledge when facing the future. The distinction here is not to say that sloppy planning and maintenance should be consider acceptable but that we can only avoid the spaghetti to some extent. This is not to suggest the author was suggesting that. Only saying we might want to think about where the margins of efficiency are.
I was also wondering if some of the innovations in graph theory might help manage decision-making in a spaghetti world. The idea of a programming language that can help refactor the spaghetti might use such tools.
The last (actually first but...) thought I had was "Hello Gordian Knot."
Sometimes changing such spaghetti towers can become a beautiful art all in itself. Like a combination of Jenga, Mikado and those mind boggling topology riddles.
Take for example the rules of D&D. They started simple, then new rules was added in the most spaghetti like way imaginable (Okay, you can play a wizard. But then you are not allowed to wear any armor! You can still wear a backpack because otherwise it is inconvenient. And no Backstabbing with a longsword!) The problem is that for every arbitrary spaghetti rule, somebody will have build a beloved around it. So you got to admire the game designers who untangled the spaghetti tower of rules version 3.5 into the slightly less tangled and more playable version 5 without hurting anybody's feelings too much.
See also Nomic, a game by Peter Suber where a move in the game is a proposal to change the rules of the game.
It seems to me you are talking about levels of entropy. High entropy systems still rely upon the low entropy of universal laws. A computer program routines may be itertwined, wrapped around each other, undocumented, may be a nightmare to understand, but if it works, why fix it? It conforms to the requirements to compile it into a useable executable file. A star that is exploding is like a spaghetti structure. It falls into a state of high entropy but only in as much as universal laws dictate how it will do that. Correct me if I am wrong but are you talking about levels of entropy?
Edited in later:
The kind of entropy I am talking about is a physical state. Imagine you make a building of bricks with a nice kitchen, dining room, etc etc. The building is left to stand for a 1000 years. At first it has low entropy. It is a very definite structure. Over time, roof tiles fall off, the rain gets in, wood rots, bricks crumble. By 500 years it has lost most of its original defining characteristics, it is falling into a state of high entropy. It is becoming a mound of rubble. Pour sand out of a 1000 buckets. At first the sand is shaped by the shape of the bucket, but then when it leaves the bucket that structure is lost. Pouring more sand on the floor, on top of the other sand does nothing to lower the state of entropy. Systems fall into decay. Unravelling the decay, creating some semblance of order brings about a different structure a different level of entropy
Many automobiles are spaghetti towers. The longer a model or drivetrain is in production, the more spaghetti. I drive a 1985 Volkswagen Vanagon. Its ancestors had horizontally opposed four cylinder air-cooled engines in the rear, like all ancestral VWs. In 1984 the Vanagon got water cooled, but still horizontally opposed four cylinder rear-engine. The plumbing is, as they say, like a submarine. Some later Vanagon models layered onto this concept four-wheel drive and air conditioning. Talk about plumbing.
Of course, most every automobile wiring system grew from simple ignition-lights-radio-heater-fan-windshield-wipers to include airplane-like control panels, wireless entry locks, theft alarms, seatbelt buzzers, seat warmers, electric windows, seats, hatches, sun roofs, and steering-wheel adjusters. I think four fuse panels is the norm now. My modern diesel pickup truck had two batteries, two alternators, and every gizmo in the catalog, layered onto the basic automotive wiring system.
The general patterns is
Systems in general work poorly or not at all.
-- https://en.wikipedia.org/wiki/Systemantics
Which also has lots of examples but needs to be taken not too serious.
Meta note: the actual link URL (https://www.lesswrong.com/out?url=eukaryotewritesblog.com%2F2018%2F12%2F21%2Fspaghetti-towers%2F) results in an error when I click on it.
Fixed it manually. The url specified didn't have a www or an http at the beginning, which means it was interpreted as a relative URL. Not sure what I want the behavior to be here, since sometimes we do want relative URLs. Will probably just start paying attention to it and fix it whenever I see someone accidentally adding a relative URL, and if it starts happening often will change the default behavior.
Wasn't the 2008 credit crunch due to this (at least partly)? Numerous dependencies between different financial institutions which they only partly understood so they all stopped trusting each other?
I've heard that there has been a similar situation for many years in derivatives - a vast complex largely uncharted edifice of dependencies between complex financial instruments that are based on other financial instruments e.g. share prices. If parts of it collapse no-one knows which other bits will collapse too. Its size depends how you look at it, but it's certainly in the trillions of $, and by some accounts it's far larger than the entire real-world economy. https://www.investopedia.com/ask/answers/052715/how-big-derivatives-market.asp
If this is the case, would it be possible and/or useful to trace the "dependency tree", determine what instrument lay most precarious atop the mound of interconnected dependencies, and determine a good bet on its likelihood of failure as compared to that of less dependent instruments? [1]
- ^
...Or did I just describe something that has already been done or dismissed/invalidated/etc? It gives me the same vague suspicion I got as a fresh undergrad having just independently theorized file compression for the umpteenth time (many students having learned some basics of information theory do this if they don't already know how compression works.)
Here’s a pattern I’d like to be able to talk about. It might be known under a certain name somewhere, but if it is, I don’t know it. I call it a Spaghetti Tower. It shows up in large complex systems that are built haphazardly.
Someone or something builds the first Part A.
Later, someone wants to put a second Part B on top of Part A, either out of convenience (a common function, just somewhere to put it) or as a refinement to Part A.
Now, suppose you want to tweak Part A. If you do that, you might break Part B, since it interacts with bits of Part A. So you might instead build Part C on top of the previous ones.
And by the time your system looks like this, it’s much harder to tell what changes you can make to an earlier part without crashing some component, so you’re basically relegated to throwing another part on top of the pile.
I call these spaghetti towers for two reasons: One, because they tend to quickly take on circuitous knotty tangled structures, like what programmers call “spaghetti code”. (Part of the problem with spaghetti code is that it can lead to spaghetti towers.)
Especially since they’re usually interwoven in multiple dimensions, and thus look more like this:
“Can you just straighten out the yellow one without touching any of the others? Thanks.”
Second, because shortsightedness in the design process is a crucial part of spaghetti machines. In order to design a spaghetti system, you throw spaghetti against a wall and see if it sticks. Then, when you want to add another part, you throw more spaghetti until it sticks to that spaghetti. And later, you throw more spaghetti. So it goes. And if you decide that you want to tweak the bottom layer to make it a little more useful – which you might want to do because, say, it was built out of spaghetti – without damaging the next layers of gummy partially-dried spaghetti, well then, good luck.
Note that all systems have load-bearing, structural pieces. This does not make them spaghetti towers. The distinction about spaghetti towers is that they have a lot of shoddily-built structural components that are completely unintentional. A bridge has major load-bearing components – they’re pretty obvious, strong, elegant, and efficiently support the rest of the structure. A spaghetti tower is more like this.
Image from the always-delightful r/DiWHY.
(The motto of the spaghetti tower is “Sure, it works fine, as long as you never run lukewarm water through it and unplug the washing machine during thunderstorms.”)
Where do spaghetti towers appear?
I think there are probably a lot of examples of spaghetti towers, and they probably have big ramifications for things like, for instance, what systems evolution can and can’t build.
I want to do a much deeper and more thoughtful analysis about what exactly the implications here are, but this has been kicking around my brain for long enough and all I want to do is get the concept out there.
Does this feel like a meaningful concept? Where do you see spaghetti towers?