Though not meant as derision, it is absolutely wild to read “Though I don't know that much about orcas” and “50% that orcas could do superhuman scientific problem solving” in the same paragraph.
My uneasiness with this post is that I am not sure how serious/joking the post is. It has some of the hallmark of a relatively lighthearted post written in a serious way. (The interaction with the IP, for example) And tones of conversation is light at parts. Yet the call to actions are confusing - it is not really motivating and seems to offload responsibility too eagerly for someone that actually believes what they are writing about.
I am very confused about the post and not sure what to think about it.
I would recommend reading the original reddit post that motivated it: https://www.reddit.com/r/biology/comments/16y81ct/the_case_for_whales_actually_matching_or_even/.
It is meant seriously, but the author is rightly acknowledging how far-fetched it sounds.
I did read the original. It was long and I skimmed it. It was better in the coherence-sense that the OOP didn’t post a probability on whether it is true or not. Hell, the OOP hedged it by saying “ Do I believe what I’m saying? Well, yes and no”.
I guess the core of my confusion is the radical mismatch in confidence projection in its explicit form and implicit form (through tone and context setting). [Note: the updated wording definitely tempers the expectations in the right direction, thou still a bit bonkers at first glance.]
50% is extremely high. And lighthearted tones are often used to convey a sense of “I know this is farfetched theory. But I hold this strong claim very/appropriately weakly”.
1The post is not joking. (But thanks for feedback that you were confused in this way!)
I basically didn't know much about orcas before I learned that they have 2.05 times 3 days ago as many neurons in the neocortex than humans, and then yesterday and the day before I spent looking into how smart orcas might be and evaluating the evidence. So I'm far from an expert but I also didn't run over strong evidence that they are dumber than hunter-gatherer humans and some weak-medium strong evidence that they are might at least as smart as 15 year olds. But it's still possible that orca researchers have observed orcas not finding some strategies they would've thought of or sth.
But yeah the only piece of evidence that they might be significantly smarter than humans is their brains. I consider it reasonably strong though.
ADDED:
I edited "Though I don't know that much about orcas" to "Though I only tried to form a model of orca intelligence for 1-2 days". thanks!
I appreciate you being relatively clear about this, but yeah, I think it's probably better to spend more time learning facts and thinking stuff through, compared to writing breathless LW posts. More like a couple weeks rather than a couple days. But that's just my stupid opinion. The thing is, there's probably gonna be like ten other posts in the reference class of this post, and they just... don't leave much of a dent in things? There's a lot that needs serious thinking-through, let's get to work on that! But IDK, maybe someone will be inspired by this post to think through orca stuff more thoroughly.
I don't think I'm the right person to look into this.
I just updated quickly via conservation of expected probability. (I agree though that I'd be a bit concerned about most people updating that quickly. If you think I've gone slightly psychotic please bet with me so I update harder if I notice you're right.)
(EDIT: actually it's sorta shitty because we might not get more evidence because I have even more important things to do and probably don't have time to look into it myself, but i'm happy to bet, though I'd probably want to revise my betting probability.)
I'm happy to bet on "By the end of 2034, does Tsvi think that it's >60% likely that orcas could do superhuman science if they had similar quality and quantity of science education as scientists and were motivated for this, conditional on Tsvi having talked to me for at least 2 hours about this {sometime in 2030-2034}/{when I might have more evidence}?"
I'd currently be at like 30%26% on this, though if you take more time to think about it I might adjust this estimate I am willing to bet on.
I'm happy to bet up to 200$ per bit (or maybe more but would have to think about it). Aka if it's resolved "Yes", money flowing from you to me would be (and if it's resolved "No" it would be ). (Where negative money flow indicates flow into the other direction.)
(Also obviously you'd need to commit to talking to me for 2h sometime when i have more evidence, and not just avoid resolution by not talking to me.)
I don't know what you mean by this:
The thing is, there's probably gonna be like ten other posts in the reference class of this post, and they just... don't leave much of a dent in things?
I'm not gonna read the reddit post because
- it's an eyebleed wall of text,
- the author spent hours being excited about this stuff without bothering to learn that we have ~20 billion cortical neurons, not 20 trillion,
- yeah.
I don't know whether orcas are supersmart. A couple remarks:
- I don't think it makes that much sense to just look at cortical neuron counts. Big bodies ask for many neurons, including cortical motor neurons. Do cetaceans have really big motor cortices? Visual cortices? Olfactory bulbs? Keyword "allometry". Yes, brains are plastic, but that doesn't mean orcas are actually ever doing higher mathematics with their brains.
- Scale matters, but I doubt it's very close to being the only thing! Humans likely had genetic adaptations for neuroanatomical phenotypes selected-for by some of: language; tool-making; persisting transient mental content; intent-inference; intent-sharing; mental simulation; prey prediction; deception; social learning; teaching; niche construction/expansion/migration. Orcas have a few of these. But how many, how much, for how long, in what range of situations and manifestations? Or do you think a cow brain scaled to 40 billion neurons would be superhuman?
- Culture matters. The Greeks could be great philosophers... But could a kid living in 8000 BCE, who gets to text message with an advanced alien civilization of kinda dumb people, become a cutting edge philosopher in the alien culture? Even though almost everyone ze interacts with is preagricultural, preliterate? I dunno, maybe? Still seems kinda hard actually?
- Regardless of all this, talking to orcas would be super cool, go for it lol.
- Superbabies is good. It would actually work. It's not actually that hard. There's lots of investment already in component science/tech. Orcas doesn't scale. No one cares about orcas. There's not hundreds of scientists and hundreds of millions in orca communications research. Etc. The sense of this plan being weird is a good sense to investigate further. It's possible for superficial weirdness to be wrong, but don't dismiss the weirdness out of hand.
2
- I don't think it makes that much sense to just look at cortical neuron counts. Big bodies ask for many neurons, including cortical motor neurons. Do cetaceans have really big motor cortices? Visual cortices? Olfactory bulbs? Keyword "allometry". Yes, brains are plastic, but that doesn't mean orcas are actually ever doing higher mathematics with their brains.
See this comment.
- Scale matters, but I doubt it's very close to being the only thing! Humans likely had genetic adaptations for neuroanatomical phenotypes selected-for by some of: language; tool-making; persisting transient mental content; intent-inference; intent-sharing; mental simulation; prey prediction; deception; social learning; teaching; niche construction/expansion/migration. Orcas have a few of these. But how many, how much, for how long, in what range of situations and manifestations?
I already considered this. (I just posted a question about this.) I don't have good information on to what extent orcas have those, but my guesses are already reflected in my overall guess in the post.
Why do you think orcas have few of those? For me it seems plausibe that orcas have everything except tool use and niche construction.
I do think there was some significant selection for some kind of intelligence in dolphins and orcas - the main question here is whether being optimized on tool use (IF that was a significant driver in what selected humans for intelligence) would be significantly more useful for having the brain potential generalize to doing science than if the brains were optimized because of social dynamics or hunting strategies.
But of course there are other considerations like "maybe you need fully recursive language to be able to have the abstract reasoning take off, and this might very well come from some adaptations that are not just about neoron counts, and maybe orcas don't have that".
I already took all my current uncertain consideration on this into account when I said "50% that they would be superhuman at science if they had similar quality and quantity of science education as scientists and were motivated for this".
Or do you think a cow brain scaled to 40 billion neurons would be superhuman?
I don't know what you're asking for here. I don't think organisms end up with 40 billion cortical neurons without either some strong selection for at least some sub-dimensions of intelligence, or being as big as Godzilla.
I'm not really excited about just smushing together more brain tissue without it having been optimized to work well together, but orca brains were optimized.
- Culture matters. The Greeks could be great philosophers... But could a kid living in 8000 BCE, who gets to text message with an advanced alien civilization of kinda dumb people, become a cutting edge philosopher in the alien culture? Even though almost everyone ze interacts with is preagricultural, preliterate? I dunno, maybe? Still seems kinda hard actually?
Yep that's why I'm only at like 15% that we get very significant results out of it in the next 30 years even if we tried hard. (aka 30% conditional on orcas being smart enough.)
- Superbabies is good. It would actually work. It's not actually that hard. There's lots of investment already in component science/tech. Orcas doesn't scale. No one cares about orcas. There's not hundreds of scientists and hundreds of millions in orca communications research. Etc. The sense of this plan being weird is a good sense to investigate further. It's possible for superficial weirdness to be wrong, but don't dismiss the weirdness out of hand.
I mean if Orcas are smarter they might be super vastly smarter so you wouldn't need that many.
Superbabies would work well given multiple generations but also only like 30% that we'd get +7std humans born within 10 years even if we tried similarly hard[1], and I think it's pretty unlikely we have more than 40 years left without strong governance success. (E.g. afaik we still have problems cloning primates well (even though it's been a thing for long) and those are just sub-difficulties[2] of e.g. creating superbabies through repeated embryo selection.)
(e.g. 1 billon dollars and a few very smart geniuses going into trying to make communication with orcas work well)
That would give more like a 90% chance of superbabies born in <10 years.
Yeah I pretty much agree with this assessment. I think you could probably get to 80% with 100 million and ten years and maybe 50% with 30 million and 7 years. Perhaps I'm optimistic, but right now the entire field is bottlenecked by the need for $4 million to do primate testing.
(For reference I think Tsvi and GeneSmith have much more relevant knowledge for evaluating the chance of superbabies being feasible and I updated my guess to like 78%.)
(As it happens I also became more optimistic about the orca plan (especially in terms of how much it would cost and how long it would take, but also a bit in how likely I think it is that orcas would actually study science) (see footnote 4 in post). For <=30y timelines I think the orca plan is a bit more promising, though overall the superbabies plan is more promising/important. I'm now seriously considering pivoting to the orca plan though.) (EDIT: tbc I'm considering pivoting from alignment research, not superbaby research.)
Justification for this:
I don't think organisms end up with 40 billion cortical neurons without either some strong selection for at least some sub-dimensions of intelligence, or being as big as Godzilla.
One could naively expect that the neuron count (especially touch and motor) sensory processing modules are proportional to the surface area of an organism. However I think this is unrealistic: Bears don't need nearly as fine precision on what square centimeter of skin was touched (or what millimeter the paw moves) than mice, and generally this is because precision gets less relevant given body size.
So let's say the precision an organism needs is proportional to the square root of the 1-dimensional-size (aka sqrt(surface_area)) of the organism. Aka if a mice is 5cm tall and a bear 2m, the spacing between sensors on the mouse skin vs on the bear skin would be sqrt(0.05) vs sqrt(2). The number of sensors on the skin surface is proportional to the square of the distancing between sensors, so the overall number of sensors is proportional to the 1-dimensional-size (aka sqrt(surface_area)).
A brown bear has 250million neorons in the neocortex and is maybe 2m tall. So to get just by scaling size to 40billion neorons an organism would have to be 40/0.25 * 2m = 320m tall. So actually bigger than godzilla.
Really interesting post! Out of curiosity, have you ever read Star Maker? This is basically one of the civilizations he came up with, a symbiotic setup where the land-based crab-like animals provide the tool using intelligence and the aquatic whale-like animals provide the theoretical intelligence.
It seems unlikely to me that there is potential to make large brain based intelligence advancements beyond the current best humans using human evolved biology. There will be distance scaling limitations linked to neural signal speeds.
Then there is Jeff Hawkins 'thousand brains' theory of human intelligence that our brains are made up of thousands of parallel processing cortical columns of a few mm cross section and a few mm thick with cross communication and recursion etc, but that fundamental processing core probably isn't scalable in complexity, only in total number - your brain could perhaps be expanded to handle thinking about more things in parallel at once, but not at much higher levels of sophistication without paying a large coordination speed price (and evolution places a premium on reaction speed for animals that encounter violence)
I look at whales and other mammals with much much larger than human brains and wonder why they are not smarter - some combination of no evolutionary driver and perhaps a lot of their neurons are dedicated to delay-line processing needed for processing sonar and controlling large bodies with long signaling delays.
Regardless, if AI is a dominant part of our future then it seems likely to me that regardless of whether the future is human utopia or dystopia, non-transhuman humans will not exist in significant numbers in a few hundred years. Neural biology and perhaps all biology is going to be superseded as maladapted to the technological future.
I think number of neurons in neocortex (or even more prefrontal cortex - but unfortunately i didn't quickly find how big the orca prefrontal cortex is - though I'd guess it to still be significantly bigger than for humans) is a much much better proxy for intelligence of species than brain size (or encephalization quotient). (E.g. see the wikipedia list linked in my question here.)
(Also see here. There are more examples, e.g. a Blue and yellow macaw has 1.9 billion, whereas brown bears have only 250million.)
EDIT: Tbc I do think that larger bodies require more neurons in touch-sense and motor parts of the neocortex, so there is some effect of how larger animals need a bit larger brains to be similarly smart, but I don't think this effect is very strong.
But yeah there are other considerations too, which is why I am only at 50% that orcas could do science significantly better than humans if they tried.
A third approach to superbabies: physically stick >10 infant human brains together while they are developing so they form a single individual with >10x the neocortex neurons as the average humans. Forget +7sd, extrapolation would suggest they are >100sd intelligence.
Even better, we could find some way of networking brains together into supercomputers using configurable software. This would reduce potential health problems and also allow us to harvest their waste energy. Though we would have to craft a simulated reality to distract the non-useful conscious parts of the computational substrate, perhaps modeled on the year 1999...
2
(Note: This post might be slightly funnily written but it is not a joke but serious and important.)
Me: I think it's possible that we might not need to figure out biotechnological advances for how we can create superbabies through embryo selection or so, but that actually they might already be around 1000 or more superbabies born on earth per year, only they happen to be born into an environment where they don't get educated and don't learn the necessary skills to advance science. By superbabies I mean genetic potential for intelligence of >=+7std on the human distribution.
Imaginary person (IP): What, how could that be true? There being some hunter tribe of genetical supergeniuses which are almost not communicating with the rest of humanity?
Me: Sorta yeah.
IP: And where would those superbabies be?
Me: Here:
.
.
.
.
.
IP: Are you joking?
Me: No. See this excellent reddit post (or my wrapper post around it) for why it's IMO a pretty reasonable guess. Actually just read this right now before continuing. It takes only 5min. (ADDED: You could now also read this, but I still strongly recommend reading the reddit post first.)
IP: Ok but it's actually just a fun relatively unlikely possibility right?
Me: No. My current guess would 50% that orcas could do superhuman scientific problem solving (aka >=+7std[1]) if they actually trained themselves at it for human equivalent amounts and had human equivalent interfaces for research (e.g. BCI for using a computer). Though I only tried to form a model of orca intelligence for 2 days[2], so please tell me more evidence and considerations.
Though even if we tried significantly[3] I'd only give it like 15% chance that within the next 30 years, orcas could in one year solve theory-bottlenecked problems for which science would've taken 20 years.[4] (Yeah I know 15% is still ridiculously high considering how crazy the plan sounds.)
E.g. orcas might want to rather do orca stuff, or there might be orca-cultural pressure against spending a large chunk of a day studying, or communicating with orcas turns out to be much harder than my median expectation[5]. (Please comment if you have more considerations on why it might (not) work.)
Can someone please look into this?
I think conditional on orcas being >=+7std intelligent, I'd be about as excited about this as about trying to get human superbabies soon.[6]
I'd guess it might not be all that hard to get a better guess on how smart orcas are in comparison to humans. E.g. get a guess on how sophisticated their language is, see how good they are at pattern recognition or learning to solve simple math or other problems[7]. (Though in the cases where orcas perform well we'd still have some uncertainty about how well it's going to generalize to hard science problems I guess.)
Can people (you?) please look into this, and if it seems like orcas might be superhumanly smart work to become able to communicate well with them etc?[8]
(ADDED: Please let me know if you're planning to look into it more thoroughly (and also when you changed your mind and not planning it anymore).)
Cautionary notes
There's a lot of important questions to think through which I haven't thought through at all. Here's a very non-exhaustive list:
I feel sorta relatively optimistic here though.
Also the the plan here would obviously NOT be to keep orcas in captivity and try to train and extract useful cognitive work from them, but to build study places for orcas where they can come to of their own accord and communicate with humans and be taught.
Sidenote on how orcas (and pilot whales) could be useful datapoints for AI alignment
(feel free to skip this if you're not interested in alignment.)
Steven Byrnes is trying to reverse engineer the steering subsystem in human brains. My uncertain guess is that this probably is not directly amazingly useful for aligning AI, but I'm still a big fan of his agenda. A major reason for this is that understanding the steering subsystem might help us to better understand how human values form, and that we might be able to generalize the understanding a bit to see how we might align AIs.
If there are other intelligent species on earth, we could reverse their steering subsystems too and observe and understand how their values form, and we'd thereby have multiple datapoints which seems like a lot better basis for forming good generalizing models of what alignment-machinery we need for shaping the (values of the) AI as we want.
where most of the 50% probability mass is actually on orcas being even vastly smarter than +10std humans would be
Though FWIW I think my considerations are probably more thorough than you might expect given the 2 days. E.g. see this comment.
(e.g. 1 billon dollars and a few very smart geniuses going into trying to make communication with orcas work well)
UPDATE 2024-11-10: I've gotten very slightly more optimistic that orcas are smart enough, and significantly more optimistic on how feasible it would be, especially in terms of how much it would cost (and how long it would take). Currently I'm at like 30% that I could do it with $50M in 25 years (aka like 55% conditional on orcas being smart enough). (Though tbh for almost anyone else I'd assign a chunk lower probability.) My estimate will likely change further but I probably won't update this post further, but feel free to ask me in the future on my updated estimate via comment or PM.
(where tbc my median guess still says it will be very difficult - but perhaps feasible given a highly selected group of geniuses)
Though with better understanding we might of course be able to predict much better what approach is more promising, rather than thinking they are similarly promising.
Tbc, I'd not expect current orcas to be able to compete with current humans in abstract problem solving, because orcas are probably basically not trained in it at all. But we could see how fast they learn it and maybe compare how fast a human who didn't get education can learn it.
I'm currently not quite yet excited enough myself / I am more excited about my current agenda (which I am unusually excited about - the orca thing is still among the top contenders). Also I have much higher irreplaceability on my agenda whereas I might not have that great of a competetive advantage on the orca stuff.