Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

Say Not "Complexity"

34 Post author: Eliezer_Yudkowsky 29 August 2007 04:22AM

Once upon a time...

This is a story from when I first met Marcello, with whom I would later work for a year on AI theory; but at this point I had not yet accepted him as my apprentice.  I knew that he competed at the national level in mathematical and computing olympiads, which sufficed to attract my attention for a closer look; but I didn't know yet if he could learn to think about AI.

I had asked Marcello to say how he thought an AI might discover how to solve a Rubik's Cube.  Not in a preprogrammed way, which is trivial, but rather how the AI itself might figure out the laws of the Rubik universe and reason out how to exploit them.  How would an AI invent for itself the concept of an "operator", or "macro", which is the key to solving the Rubik's Cube?

At some point in this discussion, Marcello said:  "Well, I think the AI needs complexity to do X, and complexity to do Y—"

And I said, "Don't say 'complexity'."

Marcello said, "Why not?"

I said, "Complexity should never be a goal in itself.  You may need to use a particular algorithm that adds some amount of complexity, but complexity for the sake of complexity just makes things harder."  (I was thinking of all the people whom I had heard advocating that the Internet would "wake up" and become an AI when it became "sufficiently complex".)

And Marcello said, "But there's got to be some amount of complexity that does it."

I closed my eyes briefly, and tried to think of how to explain it all in words.  To me, saying 'complexity' simply felt like the wrong move in the AI dance.  No one can think fast enough to deliberate, in words, about each sentence of their stream of consciousness; for that would require an infinite recursion.  We think in words, but our stream of consciousness is steered below the level of words, by the trained-in remnants of past insights and harsh experience...

I said, "Did you read A Technical Explanation of Technical Explanation?"

"Yes," said Marcello.

"Okay," I said, "saying 'complexity' doesn't concentrate your probability mass."

"Oh," Marcello said, "like 'emergence'.  Huh.  So... now I've got to think about how X might actually happen..."

That was when I thought to myself, "Maybe this one is teachable."

Complexity is not a useless concept.  It has mathematical definitions attached to it, such as Kolmogorov complexity, and Vapnik-Chervonenkis complexity.  Even on an intuitive level, complexity is often worth thinking about—you have to judge the complexity of a hypothesis and decide if it's "too complicated" given the supporting evidence, or look at a design and try to make it simpler.

But concepts are not useful or useless of themselves.  Only usages are correct or incorrect.  In the step Marcello was trying to take in the dance, he was trying to explain something for free, get something for nothing.  It is an extremely common misstep, at least in my field.  You can join a discussion on Artificial General Intelligence and watch people doing the same thing, left and right, over and over again—constantly skipping over things they don't understand, without realizing that's what they're doing.

In an eyeblink it happens: putting a non-controlling causal node behind something mysterious, a causal node that feels like an explanation but isn't.  The mistake takes place below the level of words.  It requires no special character flaw; it is how human beings think by default, since the ancient times.

What you must avoid is skipping over the mysterious part; you must linger at the mystery to confront it directly. There are many words that can skip over mysteries, and some of them would be legitimate in other contexts—"complexity", for example.  But the essential mistake is that skip-over, regardless of what causal node goes behind it.  The skip-over is not a thought, but a microthought.  You have to pay close attention to catch yourself at it.  And when you train yourself to avoid skipping, it will become a matter of instinct, not verbal reasoning.  You have to feel which parts of your map are still blank, and more importantly, pay attention to that feeling.

I suspect that in academia there is a huge pressure to sweep problems under the rug so that you can present a paper with the appearance of completeness.  You'll get more kudos for a seemingly complete model that includes some "emergent phenomena", versus an explicitly incomplete map where the label says "I got no clue how this part works" or "then a miracle occurs".  A journal may not even accept the latter paper, since who knows but that the unknown steps are really where everything interesting happens?  And yes, it sometimes happens that all the non-magical parts of your map turn out to also be non-important.  That's the price you sometimes pay, for entering into terra incognita and trying to solve problems incrementally.  But that makes it even more important to know when you aren't finished yet.  Mostly, people don't dare to enter terra incognita at all, for the deadly fear of wasting their time. 

And if you're working on a revolutionary AI startup, there is an even huger pressure to sweep problems under the rug; or you will have to admit to yourself that you don't know how to build an AI yet, and your current life-plans will come crashing down in ruins around your ears.  But perhaps I am over-explaining, since skip-over happens by default in humans; if you're looking for examples, just watch people discussing religion or philosophy or spirituality or any science in which they were not professionally trained.

Marcello and I developed a convention in our AI work: when we ran into something we didn't understand, which was often, we would say "magic"—as in, "X magically does Y"—to remind ourselves that here was an unsolved problem, a gap in our understanding.  It is far better to say "magic", than "complexity" or "emergence"; the latter words create an illusion of understanding.  Wiser to say "magic", and leave yourself a placeholder, a reminder of work you will have to do later.


Part of the sequence Mysterious Answers to Mysterious Questions

Next post: "Positive Bias: Look Into the Dark"

Previous post: "The Futility of Emergence"

Comments (48)

Sort By: Old
Comment author: Felix2 29 August 2007 07:23:23AM 3 points [-]

Quote: "We think in words, "

No we don't. Apparently you do, though. No reason to believe otherwise. :)

Please keep up these postings! They are very enjoyable.

Going back to "explaining" something by naming it (from a couple of your earlier posts):

e.g. Q: Why does this block fall to the floor when I let go of it? ... A: Gravity!

I always thought that such explanations were common side-effects of thinking in words. Sort of like optical illusions are side-effects of how the visual system works. Perhaps not. One does not need to use words to think symbolically. There are, after all, other ways to do lossy compression than with symbols.

Anyway, I'll still assert that it's easier to fall for such an "explanation" if you think in words. ... An easy assertion, given how hard it is to count the times one does it!

Comment author: Rick_Smith 29 August 2007 08:25:58AM 1 point [-]

Aren't we understating the role of labels in brevity here?

Where the labelled thing is understood well enough by the labeller and listener or of trivial importance to the problem domain, don't labels contribute to cognitive economy?

I'd have said when you need to get things done, fear of wasting time is desirable rather than deadly.

Comment author: Valter 29 August 2007 08:34:35AM 7 points [-]

Actually, the "emergence" and "complexity" pseudo-causal explanation are much worse than Felix's "gravity" example: the answer "Gravity!" does explain the fact that the block falls to the floor by noting that it is a specific instance of a general phenomenon for which we have very precise information on how it works (attraction force is constant x m1 x m2 /d^2). We may not know why gravity exists, but that is a different (higher level?) problem.

In the case of "emergence" and "complexity", we just don't know.

P.S. I do think that "emergence" is a useful concept to describe situations where modelling is more conveniently done at a (more) aggregate level, but that's yet another story.

Comment author: Hopefully_Anonymous 29 August 2007 11:20:23AM 2 points [-]

I don't think these parable posts convey information efficiently to the overcomingbias audience, but I like your point at the end. Specifically, I agree it's better to use placeholders that make lack of knowledge/understanding clear, rather than placeholders that seem to cover up such lack of knowledge/understanding.

Comment author: Nato_Welch 29 August 2007 07:19:38PM 0 points [-]

"Then a miracle occurs..."


I wonder if memetics would serve as a good candidate for the category of things that satisfy without explaining or predicting anything, along with phlogiston, emergence, and complexity. The analogy to biology seems interesting and fun, but is it more useful than as just a way to re-formulate our perspective?

Comment author: timtyler 04 March 2011 08:06:26PM *  0 points [-]

I don't know where you get the idea that memetics doesn't explain or predict things from.

We know a lot about what factors influence cultural virulence. Marketing and advertising folk make use of that knowledge on a daily basis. They know which jingles are catchy, which catchphrases are likely to be repeated, which images are more likely to be shared - and so on. We know which ideas play well with which other ones well enough to know that we should not target our condom commercials at the catholic demographic.

Check out Dan Zarella for some of the recent material: http://danzarrella.com/

He views his work as being memetics: http://danzarrella.com/what-is-a-meme.html

Comment author: Gray_Area 29 August 2007 10:29:22PM 15 points [-]

In computer science there is a saying 'You don't understand something until you can program it.' This may be because programming is not forgiving to the kind of errors Eliezer is talking about. Interestingly, programmers often use the term 'magic' (or 'automagically') in precisely the same way Eliezer and his colleague did.

Comment author: Bernard_Guerrero 29 August 2007 10:43:13PM 5 points [-]

Step 1: Steal Underpants Step 2: ????? Step 3: Profits!!!!

Comment author: Eliezer_Yudkowsky 30 August 2007 12:42:18AM 8 points [-]

Programming is not forgiving to the kind of errors Eliezer is talking about.

But it's a lot better to be unforgiving of yourself than to wait for reality to hit you over the head with it. It's better to notice in 10 seconds that you don't understand something, than to realize this only after 20 people spend 5 years and $10 million of venture capital and the "emergent behavior" you pinned your hope on fails to materialize. It's all too easy to program "chaos", "complexity", or "emergence", so long as you tell yourself that you need to program more of it before you reach Step 3 and Profit.

Comment author: Tom_McCabe 30 August 2007 02:27:46AM 1 point [-]

"That was when I thought to myself, "Maybe this one is teachable.""

*How* many people have asked you about becoming an AGI designer? It sounds like you have a good deal of experience with rejection, even after weeding out the obvious crackpots.

Comment author: Barkley_Rosser 30 August 2007 07:15:38AM 0 points [-]

Well, this is partly a matter of what discipline one is dealing with. So, sure, for AI or computer science more generally, Kolmogorov or Chaitin or Rissanen measures are more useful and reasonably well defined. For other disciplines, other definitions may be more suitable. Thus for economics, I have (following Richard Day) defined complexity in a dynamic way based on erratic dynamics appearing endogenously out of the system (with "erratic" defined more specifically). I laid this out in a paper in 1999 in the Journal of Economic Perspectives, and have a more recent paper up on my website ("Computational and Dynamic Perspectives on Economic Complexity") comparing the two approaches, at http://cob.jmu.edu/rosserjb.

Comment author: Benjamin_Zivan 30 August 2007 01:46:16PM 4 points [-]

As a current student, I can confirm your suspicions about a seemingly complete paper being preferred over one that addresses all information about the topic. "I don't know" still is not an acceptable answer in many circles and I regard it as an unfortunate phenomenon.

Comment author: NevilleSandiego 21 October 2012 04:39:24AM 1 point [-]

In my second year uni course, I have an outline for writing lab reports that says 'include in your discussion anything you feel is out of place, or that you don't understand in this experiment. You will not be marked down for such admissions'. And I thought 'NO-ONE is going to take you up on that.'. I hate having to bullshit science papers - I tend to compromise, with a hashed together explanation that I express doubt in, and take the marks hit. Bullshitting is great fun in English courses, but in science it feels like shooting myself in the foot.

Comment author: Bill_Harris 31 August 2007 04:09:54PM 0 points [-]

Gray Area wrote, "You don't understand something until you can program it." As somewhat of an aside, Randy MacDonnell has written about APL and J (http://facilitatedsystems.com/weblog/2007/07/if-you-can-say-it-its-done.html), "If you can say it, it's done."

On a different note, I took a pair of summer high school mathematics courses sponsored by the NSF years ago at the University of Miami. One professor, a Dr. Hermann, I seem to recall, said he often imagined himself wearing "worry beads" and fingering them as he spoke. If he fingered the beads nearer his neck, he was speaking more precisely; if he fingered those nearer his waist, he was speaking less precisely. In reality, he wore no beads, but he did, on occasion, finger imaginary beads as he was explaining certain concepts.

Perhaps the same thing can be adapted here to indicate the level of magic in claims we make. If we finger imaginary beads near our necks, we claim we know what's going on; if we finger those nearer our waists, we admit there's magic here.

Comment author: Nathan2 31 August 2007 07:20:30PM 0 points [-]

Forgive me for latching onto the example, but how would an AI discover how to solve a Rubik's cube? Does anyone have a good answer?

Comment author: DanielLC 27 December 2009 07:49:43PM 0 points [-]

I had the same problem.

I think it would need some genetic algorithm in order to figure out about how "close" it is to the solution, then make a tree structure where it figures out what happens after every combination of however many moves, and it does the one that looks closest to the solution.

It would update the algorithm based on how close it is to the closest solution. For example, if it's five moves away from something that looks about 37 moves away from finishing, then it's about 42 moves away now.

The problem with this is that when you start it, it will have no idea how close anything is to the solution except for the solution, and there's no way it's getting to that by chance.

Essentially, you'd have to cheat and start by giving it almost solved Rubik's cubes, and slowly giving it more randomized ones. It won't learn on its own, but you can teach it pretty easily.

Comment author: CronoDAS 28 December 2009 06:50:33AM 2 points [-]

A less cheating-ish solution is to use some reasonable-seeming heuristic to guess how close you are to a solution. For example, you could just count the number of squares "in the right place" after a move sequence.

Comment author: xfc 20 March 2010 01:00:28PM 0 points [-]

(First post, bear with me.. find the site very interesting :)

I do agree!

But actually I would model the problem with what is known in some circles as a closed-loop controller, and specifically with a POMDP. Then apply RealTime Dynamic Prog. by embedding an heuristic without having to visit all the states in order to compute the rough but optimal h*.

Another way could be done by means of a graphical model, and more specifically a DAG would be quite nicely suited to the problem. Apply a simulated annealing approach (Ising model!) and when you reach "thermal equilibrium" by having minimized some energy functional you get the solution. Obviously this approach would involve learning the parameters of the model, instead of modelling the problem as in my first proposed approach.

Quite geeky, excuse me!

Comment author: CG_Morton 14 June 2011 04:47:48PM 0 points [-]

Exactly the difficulty of solving a Rubik's cube is that it doesn't respond to heuristics. A cube can be 5 moves from solved and yet look altogether a mess, whereas a cube with all but one corner correct is still some 20 moves away from complete (by the methods I looked up at least). In general, -humans- solve a Rubik's cube by memorizing sequences of moves with certain results, and then string these sub-solutions together. An AI, though, probably has the computational power to brute force a solution much faster than it could manipulate the cube.

The more interesting question (I think) is how it figures out a model for the cube in the first place. What makes the cube a good problem is that it's designed to match human pattern intuitions (in that we prefer the colors to match, and we quickly notice the seams that we can rotate through), but an AI has no such intuitions.

Comment author: DanielLC 15 June 2011 02:28:04AM 0 points [-]

Exactly the difficulty of solving a Rubik's cube is that it doesn't respond to heuristics. A cube can be 5 moves from solved and yet look altogether a mess, whereas a cube with all but one corner correct is still some 20 moves away from complete (by the methods I looked up at least).

I don't know the methods you used, but the only ones I know of have certain "steps" where you can easily tell what step it's on. For example, by one method, anything that's five moves away will have all but two sides complete.

Comment author: danlowlite 20 August 2010 02:24:29PM 5 points [-]

Wouldn't the AI have to discover that it is something to be solved, first? Give a kid such a puzzle and she's likelier to put it in her mouth then even try.

Unless I'm being obtuse.

Comment author: NickiH 18 December 2010 05:32:28PM 2 points [-]

You're right, and I think that this is a mistake a lot of people make when thinking about AI - they assume that the fact that they're intelligent means they also know a lot. Like the child, their specific knowledge (such as the fact that there is something to solve), is something they have to learn, or be taught, over time.

Comment author: bigjeff5 30 January 2011 09:51:45PM -1 points [-]

Curiosity could be built-in, I don't see the problem with that.

It seems to be built-in for humans - we don't learn to be curious, though we can learn not to be.

Comment author: danlowlite 31 January 2011 02:27:52PM 1 point [-]

It could be built in. I agree. But the child is curious about it's texture and taste than how the pieces fit together. I had to show my child a puzzle and solve it in front of her to get her to understand it.

Then she took off with it. YMMV.

Good point, though.

Comment author: bigjeff5 31 January 2011 05:37:12PM 0 points [-]

But the child is curious about it's texture and taste than how the pieces fit together.

But as you see, there was an initial curiosity there. They may not be able to make certain leaps that lead them to things they would be curious about, but once you help them make the leap they are then curious on their own.

Also, there are plenty of things some people just aren't curious about, or interested in. You can only bring someone so far, after which they are either curious or not.

It would be very interesting to do the same thing with an AI, just give it a basic curiosity about certain things, and watch how it develops.

Comment author: CCC 21 October 2012 10:37:54AM -1 points [-]

Consider how this could be tested. One would write a program that generates a virtual rubik's cube, and passes this on to the AI to be solved (this avoids the complexity of first having to learn how to control robotic hands). It can't just randomly assign colours to sides, lest it end up with an unsolveable cube. Hence, the preparatory program starts with a solved cube, and then applies a random sequence of moves to it.

This will almost certainly be done on the same computer as the AI is running on. A good AI, therefore, should be able to learn to inspect its own working memory, and observe other running threads on the system - it will simply observe the moves used to shuffle the cube, and can then easily reverse them if asked.

It is possible, of course, for test conditions to be altered to avoid this solution. That would, I think, be a mistake - the AI will be able to learn a lot from inspecting its own running processes (combined with the research that led to its development), and this behaviour should (in a known Friendly AI) be encouraged.

Comment author: stack 19 September 2016 11:02:17PM 0 points [-]

the problem with this is the state space is so large that it cannot explore every transition, so it can't follow transitions backwards in a straight forward manner as you've proposed. It needs some kind of intuition to minimize the search space, to generalize it.

Unfortunately I'm not sure what that would look like. :(

Comment author: CCC 20 September 2016 10:24:09AM 0 points [-]

(Wow, this was from a while back)

I wasn't suggesting that the AI might try to calculate the reverse sequence of moves. I was suggesting that, if the cube-shuffling program is running on the same computer, then the AI might learn to cheat by, in effect, looking over the shoulder of the cube-shuffler and simply writing down all the moves in a list; then it can 'solve' the cube by simply running the list backwards.

Comment author: stack 20 September 2016 01:12:25PM 0 points [-]

Oh I see: for that specific instance of the task.

I'd like to see someone make this AI, I want to know how it could be done.

Comment author: Nate 29 May 2008 07:18:29AM 12 points [-]

"We think in words, "

Correction: We think by magic!

Comment author: thomblake 20 August 2010 02:35:58PM 1 point [-]
Comment author: TheatreAddict 08 July 2011 05:42:27PM 2 points [-]

I think I just thought of an insanely over-simplified analogy.

Say I'm not invited to my best friend's sleepover and I don't understand why. I call her, and the answer she gives me is: "It's complicated."

The situation might indeed be complicated, but the word complicated is just a fake explanation... :D Amiright, guys?

Comment author: DanielLC 06 August 2011 04:45:50AM 1 point [-]

That sounds to me more like a reason not to explain. If it's complicated, it will take a while.

Comment author: TheStevenator 01 August 2011 03:27:06AM 3 points [-]

I've been working my way through the sequences in order for the last few weeks and trying to read all of the links. I love this blog and tell people about whenever I can.

Reading these entries has helped me realize some of the ways in which I tend to think incorrectly, and I hope I am taking it slow enough to reflect enough and make myself think better. :)

I suppose I should comment about at least one thing relevant to this article in particular. Posted at 4:22 am?! When do you sleep, Eliezer?

Comment author: Thering 15 August 2011 04:31:52AM 1 point [-]

I think Many of the commanders have misread her question; it's not, how would you make a program to solve a rubiks cube (which is brute forceable as there are a finite number of states for a rubiks cube), but how do you program an ai so it can work out how to do so. The ai has to study the cube and determine the possible states of the cube (and how to write them down mathematically) and the operators available to change the state of the cube. This means it has to know what a state and an operator are (if not by name). It then has to work out how to combine the operators to change the state to a predetermined state, which we happen to call a solved cube. The ai has to be able to do this with you never having coded into the ai anything specific to do with a rubiks cube, or any methods to solve the rubiks cube. The next step would be to code an ai where it ha to work out for itself that it must combine operators to change the state of the cube. I don't think I could manage to code the simpler version; for it to have been coded well, it must be able to solve any problem, not just the rubiks cube.

Comment author: buybuydandavis 22 September 2011 09:46:30AM *  2 points [-]

we would say "magic" - as in, "X magically does Y"

That's a nice bit of semantic hygiene. I hope to remember it.

Comment author: Dmytry 06 April 2012 09:51:48PM *  1 point [-]

The solution (I posted it elsewhere also):

To solve Rubik's cube, you can just do hill climbing, with breadth-first-ish search for the higher hill point (i.e. you find higher point even if it is several moves away). This discovers the sequences. Cache the sequences.

It's a very general problem solving method, hill climbing with N move look-ahead. One does try maximizing various metrics, that are maximal in the final state, and finds one that works for you without getting you stuck in local maximum for too long. You also try various orders of iterating the moves (e.g. one could opt for repetitive sequences).

This works for chess as well, and for pretty much all puzzles. This is how I solve puzzles when I get a puzzle for first time, except of course I have terabytes worth of tricks that I can try, and 10^15 - ish operations per second; parallel, of course, but parallel works. Pre-generating sequences is not necessary. You arrive at them when hill climbing with breadth-first search, and cache them. You also tell them to other people whom you want to make into rubik-cube-solvers. The important thing that can't be stressed enough - try to figure out a good metric to climb. Some sides of hill are smoother than others.

One could hill climb some sort of complexity metric - evolution did that to arrive at humans, even though the bacteria is a better solution to 'reproduction'. You only need a comparator for climbing. Comparators are easy. You can make agents fight (or you can make agents cooperate). You don't need mapping to real number. You can do evolutionary hill climbing with n-move look ahead. edit: note that you do NOT need good ordering for hill climbing either. If sometimes a>b and b>c and c>a it is okay if you remember where you already been and avoid looping. That may still get you to the top of the hill.

Comment author: orthonormal 06 April 2012 11:17:33PM 1 point [-]

One could hill climb some sort of complexity metric - evolution did that to arrive at humans, even though the bacteria is a better solution to 'reproduction'.

I can't understand what you mean. Surely you don't mean that natural selection rewarded something besides inclusive genetic fitness.

Comment author: Dmytry 07 April 2012 06:38:42AM *  0 points [-]

It of course didn't reward anything other than fitness. And the universe is not made of anything other than quarks etc (or smaller yet things). Hello fake-reductionist nihilism.

It, however, so happened that rewarding it resulted in growing complexity of behaviours of most complex organisms. You can hill climb by pouring liquid into a valley, if all you care for is some liquid on the top of the hill; liquid behaves in a very complicated way, minimizing a very complicated metric, such that it ends up on the tops of the hills by surface tension even though most of it is in the valleys, and a single molecule would be seeking valleys. The evolution doesn't just lead to mankind. The evolution, for the most part, leads to better bacteria. Mankind is a side effect from niche-filling. Remove all bacteria and single celled organisms, and they will re-evolve from a human (the canine infectious cancer was once a dog).

Comment author: orthonormal 07 April 2012 03:52:12PM 0 points [-]

I think it would be less misleading to say that many of our complex characteristics were instrumental goals for the evolutionary process as it hill-climbed the inclusive genetic fitness metric.

Comment author: Dmytry 07 April 2012 03:58:32PM *  0 points [-]

It's hard to put it in a non misleading way. If you simulate evolution as is you are wasting almost all of your time on bacteria. Evolution didn't as much hill climb as just flooded the entire valley. edit: or rather, it predominantly wasn't going towards human. If you want to optimize, you look at how it got to human, and think how you avoid doing the rest of it.

Comment author: TheOtherDave 07 April 2012 04:51:22PM 1 point [-]

To clarify: are you actually suggesting that simulating just that subset of the evolutionary process that evolved humans and not the subset that evolved bacteria is a worthwhile strategy to explore towards achieving some goal? (If so, what goal?) Or do you mean this just as an illustration of a more general point?

Comment author: Dmytry 07 April 2012 08:49:49PM *  0 points [-]

As illustration, with a remark on practical approach. Seriously, the thing about the evolution, it doesn't "reward fitness" either.

The agents compete, some are eliminated, some are added after modification; it's a lousy hill climbing, with really lousy comparator (and no actual metric like 'fitness' - just a comparator which aren't even climbing properly - where A may beat B, B beat C, and C beat A), but it makes for a variety, where the most complex behaving agent behaves in more and more complex ways all the way until it starts inventing puzzles and solving them. When one has a goal in mind, one can tweak the comparator to get to it more efficiently. The goal can be as vague as "complex behaviour" if you know what sort of "complex" you want or have an example. Problem solving doesn't require defining stuff very precisely first.

Comment author: TheOtherDave 07 April 2012 09:12:04PM *  0 points [-]

A few things:

  • Agreed that given a process for achieving a goal that involves a comparator with that goal as a target, one can often start with a very fuzzy comparator (for example, "complex behavior") and keep refining it as one goes. That's especially true in cases where the costs of getting it not-quite-right the first time are low relative to the benefits of subsequently getting it righter... e.g., this strategy works a lot better for finding a good place to have dinner than it does for landing a plane. (Though given a bad enough initial comparator for the former, it can also be pretty catastrophic.)

  • I infer that you have a referent for 'fitness' other than whatever it is that gets selected for by evolution. I have no idea what that referent is.

  • I think it's misleading to refer to evolution having a comparator at all. At best it's true only metaphorically. As you say, all evolution acts on is the result of various competitions.

  • You seem to be implying that evolution necessarily results in extremely complex puzzle-inventing systems. If I've understood that correctly, I disagree.

Comment author: NevilleSandiego 21 October 2012 04:41:24AM 0 points [-]

'Mercury's gravitational pull has long since been destroyed by solar flares, which is why is has no atmosphere'. Something I read today- seems appropriate. Apparently they'd been watching a documentary, and I think they put the components of the explanation together incorrectly in their head.

Comment author: Houshalter 06 May 2013 10:45:44PM 0 points [-]

I think when your friend was talking about "complexity" he didn't mean the word literally. He may have meant that you would have to create a complicated solution, as opposed to finding a nice and elegant solution. The difference is you try to hammer out every detail and special case, one at a time, and adding "complexity" as you go, as opposed to just thinking about a single solution which would handle every case.

This is what I think most people mean when they talk about "complexity" as a solution to their problem. They don't literally mean that adding more complexity will solve the problem. It is just a different approach to problem solving. And sometimes that approach is easier and gets things done, even if it is more messy. Sometimes it is not.

Different approaches to solving problems is an interesting subject in itself. I've seen it create huge divisions in both artificial intelligence and politics. I tend to prefer nice elegant solutions. But when the problem seems complicated, it's tempting to run to the complexity side of things. There is no guarantee you will ever find an elegant solution, but if you just handle special case after special case, you can make progress over time for sure.

Comment author: Timo 02 January 2015 01:28:16PM *  0 points [-]

I suspect that counts as a useful thinking-tool. Whenever I notice incomplete steps in my reasoning, I'll say "by magic!" and then I can worry less about fooling myself.

A thinking-tool needs a name in order to properly install it into memory. "Magic-markers" could work.

Comment author: slicedtoad 06 July 2015 02:22:35PM *  3 points [-]

"Technical explanation of technical explanation" link is broken.

Here's a working one: http://www.yudkowsky.net/rational/technical/