This reminded me of the lord of the rings - first you get 200 pages about the genealogy of Meriadoc Brandybuck and how one of his paternal great-grandfathers had big hairy feet, then it turns out the story is about some other dude who stares into space a lot.
I enjoyed it because it also reminded me of my math study days in <redacted> city, where all the boys were smart and all the girls were ... also smart.
I am however confused about the title: How does solving an inconsequential puzzle in the most inefficient way possible showcase "returns to intelligence".
How does solving an inconsequential puzzle in the most inefficient way possible showcase "returns to intelligence".
I suppose 'returns' is used to mean 'expansions to the frontier of things you are capable of doing'. Similar to how the ability to lift a 400 pound weight is a "return to muscle mass".
I've heard a lot of people say: "if only we had smarter AI, we could solve [problem]" - but there are already smart people who could work on [problem] if society wanted them to and aren't currently doing something more important. The biggest problem is distinguishing them from fake experts, and some people seem to think AI experts solve that problem, but I think people are starting to realize the nature of AI sycophancy a bit now.
The biggest problem is distinguishing them from fake experts, and some people seem to think AI experts solve that problem, but I think people are starting to realize the nature of AI sycophancy a bit now.
Human sorting used to be a lot better, to the point where this wasn't nearly as big of a problem. IQ testing in the workplace was legal until a judge ruled otherwise in 1971. LLMs don't have the same regulations applied to them, and can be evaluated as much as we like. Certainly GPT-4o can convince reddit users that it's discovered the key to the universe, but the benchmarks put that claim to rest.
Besides that, humans allocate their upskilling time to plenty of different things - networking, public speaking, and so on. There's a tradeoff between being the world's best engineer and getting yourself in a position to use that skill. Assuming the current paradigm of throwing hard RLVR problems at LLMs holds, the problem of an unskilled human that spends his/her studying time getting to know its future bosses and hiring managers doesn't seem like it naturally carries over.
It feels like there actually are significant differences in "intelligences", where different roughly-equally-intelligent people are actually really different in their ability on specific tasks. For example, verbal intelligence vs. spatial intelligence.
(Or Drake is much much smarter than Peter)
Drake doesn't seem obviously smarter than Peter in general, rather than in the specific domain of (literal) shape rotation.
Your published post version had the widget inside of a collapsable block. Unfortunately I think that was flying to close to the sun for our editor tooling and did indeed not actually render, so I moved the widget out of the collapsible block, thinking that you probably prefer that over not being to access it at all.
Oh, alas, it worked in the editor but didn't think to check in the published version. I do indeed prefer that.
This in isolation doesn't seem like evidence about returns in the sense of what intelligence lets you accomplish that's of strategic importance. It is evidence that humans vary a lot in cognitive ability as can be pretty directly measured by a simple puzzle, but not about the mapping of cognitive ability -> strategic outcomes. You'd need to incorporate other bits of evidence for that.
Yeah, I agree; I wanted a clear intuition pump for at least the first step and it seems pretty hard to demonstrate both at once in a way that people will find convincing.
inspired, i had a go at this.
well, to jump to the ending, i got the chiral pentominoes wrong at the outset (replaced ../xx/x.//.x/.x/.. with the non-chiral x./../..//xx/x./x.). i suspect this makes the problem considerably easier, since the pieces are a bit more regular.
i was able to solve this version!
one note: you gotta put a warning before a snipe so accurate :-)
overall, i am nervous about the tendency of these tales to create a myth of uniqueness. knowing that it's solvable should make it easier, not harder! i feel similarly about the von Neumann fly train story: the correct lesson is "this is how good you should be at geometric series," not -- as often seems to be the implication -- "give up now, mortal!"
i know, i know. bold words from someone who didn't even solve the puzzle...
I read an anecdote about a nobel prize winner (maybe Nils Bohr?) who went on walks with his two sons and they played battleship against each other in their head, while he was the arbiter keeping both setups in his head. That always sounded impressive to me because it seems so off the cuff.
When I went on walks with my girlfriend and my roommate we used to play chess against each other. Also just in our heads each against each, so each playing two games at once. Despite this being much more complex than the battleship example, it doesn't impress me much, because I know that this ability comes from playing lot's of chess and is the norm in strong chess players.
My roommate could also remember long random strings within seconds, very impressive, but based on memo-techniques, i.e. learnable.
When I had dinner with a famous scientist, he mentioned something growing at x%. I said, ok, so it doubles every y years. He gave me a surprised look, but it's just a trick (dividing log(2) by the growthrate gives the approximation of the doubling time).
Point is: It is hard to use anecdotes as evidence for unusual intelligence. People can learn many impressive seeming abilities. Everything loses luster once you know how the sausage is made.
If you want to play around with the puzzle yourself, here is a widget.
I suspect I am confused, but which shape in the widget corresponds to the A block in Drake's solution?
No, it's definitely not, those are b and B.
(I confused myself and then unconfused myself. I think B, J, L, and S in the widget correspond to 1, 3, 5, and 6 out of the pentacube pairs, but I don't think Z and T correspond to 2 or 4. a / A is 2, which is why I couldn't find the correspondence. ZR is just SL.)
I'm going to tell you a story. For that story to make sense, I need to give you some background context.
I have some pretty smart friends. One of them is Peter Schmidt-Nielsen. Peter has an illustrious line of descent. His paternal grandfather was Knut Schmidt-Nielsen, regarded as one of the great animal physiologists of his time. His paternal grandmother was Bodil Schmidt-Nielsen, who became the first woman president of the American Physiological Society. Bodil's father was August Krogh, who won the 1920 Nobel Prize in Physiology "for the discovery of the mechanism of regulation of the capillaries in skeletal muscle", and later went on to found the company that would eventually become Novo Nordisk.
Peter himself is no slouch. He was homeschooled for most of his childhood. When it was time to go to university, he simultaneously applied to MIT's undergrad and grad programs, and was accepted to both. (He decided to go to undergrad.) He went on to do some startup stuff, then was an early employee at Redwood. While there, he broke Meow Hash for fun. (He's not a security guy.)
My point here is: Peter is very, very smart.
Ok, here's the story.
One day I was in a room with Peter and Drake Thomas. Peter was telling us a story about a puzzle he'd grown up with, but never solved. Peter's father also went to MIT. While there, he decided to come up with a new "cube" puzzle, finding traditional cube puzzles like the Soma Cube too easy. Knowing that people often struggled with chirality, he decided to start with the six chiral pentacube pairs, but that left four cube units to complete a four-sided cube. Thinking that four 13 pieces would make it too easy, he decided to fill out the remainder with two 1x1x2 pieces (i.e. dominos).
The six chiral pentacube pairs. Source: https://sicherman.net/c5nomen/index.html
He then cut the puzzle out of wood and spent some time trying to solve it. Not having any success, he left it overnight in the grad student lounge, and came back to find it solved the next morning.
Drake, hearing this story, said something to the effect of, "I think I can probably solve this puzzle in my head."
Impossible, right? No way a human can do that in their head in any reasonable time frame?
If you want to play around with the puzzle yourself, here is a widget.
After that, I watched Drake lie down on a couch and stare into space for two hours. Then he went to sleep. He came back downstairs the next morning and stared into space for another hour. Then he took a piece of paper and pen and wrote this down (in a spoiler block, in case you want to avoid any hints):
We didn't have a copy of the puzzle handy to make extra-sure, so they 3d printed one out and confirmed that the solution was correct.
Here are a couple of the original puzzles from the 70s:
The distribution of what unassisted human brains can accomplish is extremely wide. Human brains are squishy meat sacks. Better things are possible. Alas.