TL;DR of the article:

This piece describes a lot of why Elon Musk wanted to start Neurolink, and how Brain-Computer Interfaces (BCIs) currently work, and how they might be implemented in the future. It's a really, really broad article, and aims for breadth while still having enough depth to be useful. If you already have a grasp of evolution of the brain, Dual Process Theory, parts of the brain, how neurons fire, etc. you can skip those parts, as I have below.

AI is dangerous, because it could achieve superhuman abilities and operate at superhuman speeds. The intelligence gap would be much smaller if we also had access to such abilities. Therefore, we should attempt this if possible.

This might be possible, despite how extremely limited and highly invasive existing BCIs are. Opening the skull is obviously way too invasive for most people, but the blood vessels offer a possible minimally invasive solution. They are essentially a highway which goes directly to every neuron in the brain. Current methods monitor at most ~100 neurons, or have low temporal resolution. 1,000,000 neurons is probably the tipping point, where it would stop being an alternative to a keyboard/screen input/outputs, and start being transformative.

Neuralink is exploring many possibilities, and probably won't narrow to just one any time soon. However, options might include "neural dust", or stints in the blood vessels. Just as dies have made fine cell structures visible under microscopes, and genetically engineering bioluminescent genes into living animals has made cells glow when active, Neuralink would need a way for such a device to detect individual neuron firings on a large scale.

To do this, the inserts themselves only need to be able to:

1. React differently to electrical discharge associated with a nearby neurons firing, or to other changes associated with neurons firing, like sodium and potassium levels.

2. Have that difference be detectable from outside the skull. (I'd divide this into active methods, like emitting light in a wavenelgth which penetrates the skull, or passive changes in properties detectable from the outside, like radioactive isotopes which cluster together based on variables in blood flow.)

(The piece doesn't make this distiction, but I thought it would be useful for better discussion and understanding.)

Neuralink, of course, hasn't narrowed the specifics down very much (and will probably pivot several times, in my opinion). However, they will start out offering something largely similar to the sorts of BCIs available to people with paralysis or sensory problems. Elon hopes that if everything goes smoothly, in a decade they would have something which could provide a useful feature to someone without such disabilities, if the FDA would allow it.

They also hope to be able to eventually influence neural firings, so that we could supply information to the brain, rather than just reading information out. This would require something which could be influenced from the outside, and then influence nearby neurons. We can already put an electric field through the whole brain, to minimize seizures, but for meaningful inputs this would also have to be done at the neuron leven.

Why you should read it anyway:

It's >35,000 words. (For comparison, the cutoff for "short novel" is 40,000.) That said, it's a good read, and I recommend it if you want to understand why Elon Musk might think a BCI might increase our odds of surviving an AI takeoff scenario.

A lot of it is still hand-waving, and doesn't make it clear that we don't necessarily need full self-replicating autonomous nanobots or whatever. Since it doesn't provide a specific architecture, but just surveys what might be possible, I think it's easy to give an uncharatable reading. I've tried to steel-man the phrasing here, but I think if we focus on tangible, near-term concepts, it can be illustrative of what is possible.

If you read this with a critical eye, you'll just note that they haven't narrowed down to one architecture yet, and complain that their lack-of-an-architecture can't possibly work. The point is to convince lay people that this might even be possible, not to convince them that Neurolink will succeed, but the comments I've seen so far have just been skepticism of Neurolink.

Instead, I'd encourage you to read with an eye toward what could be done with a stint or neural dust, and then critically examine the more tangible challenge of how small each of those possible capabilities could be made. What could be done passively? What could be done if inductively powered? How small of blood vessels could various devices fit through? Will those shrink with Moore's law, or are they physics-constrained?

Such questions will generate the possible concrete architectures which you can then apply a critical lens to. Don't bother reading if you just want to be critical of the exploratory activity itself. It won't even put up a fight.

Here are some thoughts on the viability of Brain Computer Interfaces. I know nothing, and am just doing my usual reality checks and initial exploration of random ideas, so please let me know if I'm making any dumb assumptions.

They seem to prefer devices in the blood vessels, due to the low invasiveness. The two specific form factors mentioned are stents and neural dust. Whatever was chosen would have to fit in the larger blood vessels, or flow freely through all of them. Just for fun, let's choose the second, much narrower constraint, and play with some numbers.

Wikipedia says white blood cells can be up to 30 μm in diameter. (Also, apparently there are multiple kinds of white blood cells. TIL.) I'd guess that we wouldn't want our neural dust to be any larger than that if we want to be able to give it to someone and be able to reverse the procedure later without any surgery. The injection should be fine, but if you wanted to filter these things back out of your blood, you'd have to do something like giving blood, but with a magnet or something to filter out the neural dust. So, what could we cram into 30 μm?

Well, my first hit when searching "transistors per square mm" is an article titled "Intel Now Packs 100 Million Transistors in Each Square Millimeter", so let's go with that. I realize Elon's ~10 year time horizon would give us another ~6 Moore's law doublings, but if they did an entire run of a special chip just for this, then maybe they don't want to pay top dollar for state of the art equipent, so let's stick with 100m/mm^2. That'd give us on the order of 10k-100k transistors to work with, if we filled the entire area with transistors and nothing else.

But, looking at most electronics, they are more than just a chip. Arduinos and cellphones and motherboards may be built around a chip, but the chip itself has a relatively small footprint on the larger PCB. So, I'm probably missing something which would be incredibly obvious to someone with more hardware experience. (Is all the other stuff just for interfacing with other components and power supplies? In principle, could most of it be done within the chip, if you were willing to do a dedicated manufacturing run just for that one divice, rather than making more modular and flexible chips which can be encorporate into a range of devices?)

If we assume it'd be powered and transmit data electromagnetically, it'd also need an antenna, and an induction coil. I have a hunch that both of these suffer from issues with the square-cube law, so maybe that's a bad idea. The neural dust article mentioned that the (mm scale) devices both reported information and received power ultrasonically, so maybe the square-cube law is the reason. (If not, we might also run into the diffraction limit, and not have any wavelengths of light which were short enough to effect antenas that size, but still long enough to penetrate the skull without ionizing atoms.)

I like the idea of ultrasonic stuff because acoustic waves travel through tissue without depositing much energy. So, you get around the absorption problem photons have, and don't have to literally x-ray anyone's brain. Also, cranial ultrasounds are already a thing for infants, although they have to switch to transcranial Doppler for adults, because our skulls have hardened. Nearby pieces of neural dust would be monitoring the same neurons, and so would give off their signals at about the same time, boosting the signal but maybe smearing it out a little in time.

So, let's play with some numbers for piezoelectric devices instead. (I assume that's what their ultrasonic neural dust must be using, at least. They are switching between electricity and motion somehow, and piezoelectrichttps are the name for the solid state way of doing that. I can't picture them having tiny speakers with electromagnets on flexible speaker cones. The Wikipedia page on transducers doesn't mention other options.)

Quartz crystals are already used for timing in electronics, so maybe the semiconductor industry already has the ability to make transducers if they wanted to. (I'd be surprised if they didn't, since quartz is just ccrystaline silicon dioxide. Maybe they can't get the atomic lattice into the right orientation consistently, though.) If you couldn't transmit and receive simultaneously without interfering, you'd need a tiny capacitor to store energy for at least 1 cycle. I don't know how small quartz crystals could be made, or whether size is even the limiting factor. Maybe sufficiently small piezoelectric can't even put out strong enough pulses to be detectable on an ultrasound, or require too much power to be safely delivered ultrasonically? I don't know, but I'd have to play with a bunch of numbers to get a good feel.

I don't really know where to start, when discussing monitoring neuron firings. Could it be done electromagnetically, since they should make an instantaneous electromagnetic field? Or would the signal be too weak near a blood vessel? Apparently each neuron firing changes the concentration of Na, K, Cl, and Ca in the surrounding blood. Could one of these be monitored? Maybe spectrally, with a tiny LED of the appropriate wavelength, and a photo detector? I think such things are miniturizeable in principle, but I'm not sure we can make them with existing semiconductor manufacturing techniques, so the R&D would be expensive. We probably don't have anything which emits at the exact wavelength we need for spectroscopy though, and even if we did, I bet the LED would need voltage levels which would be hard to deliver without adding a voltage transformer or whatever the DC equivalent is.

Or, can we dump all the fancy electronics all together? Could we do something as simple as a clay particle (tiny rock) coated with a dispersent or other Surfactant, so that changes in the surrounding chemistry cause the collapse of the double layer), making the clay particles to flocculate together? Would such clumps of clay particles be large enough and have high enough density to show up on an ultrasound or other divice? Obviously this wouldn't let us force a neuron to fire, but it might be a cheap way of detecting them.

Maybe the electronics could be added later, if modifying surface charge and chemistry is enough to make a neuron fire. Neurotransmitrers affect neuron firings somehow, if I usnderstand correctly, so maybe chain a bunch of neurotransmitters to some neural dust as functional groups on the end of polymer chains, then change surface charge to make the chains scrunch up or fan out?

I only know just enough about any of this to get myself into trouble, so if it doesn't look like I know what I'm talking about, I probably don't.

(Sorry to spam comments. I'm separating questions out to keep the discussion tidy.)

The article only touches on it briefly, but suggests faster AI takeoff are worse, but "fast" is only relative to the fastest human minds.

Has there been much examination of the benefits of slow takeoff scenarios, or takeoffs that happen after human enhancements become available? I vaguely recall a MIRI fundraiser saying that they would start putting marginal resources toward investigating a possible post-Age of EM takeoff, but I have no idea if they got to that funding goal.

Personally, I don't see Brain-Computer Interfaces as useful for AI takeoffs, at least in the near term. We can type ~100 words per minute, but it takes more than 400 minutes to write a 40,000 word novel. So, we aren't actually I/O bound, as Elon believes. We're limited by the number of neurons devoted to a given task.

Early BCIs might make some tasks much faster, like long division. Since some other tasks really are I/O bound, they'd help some with those. But, we wouldn't be able to fully keep up with AI unless we had full-fledged upgrades to all our cognative architecture.

So, is almost keeping up with AI likely to be useful, or are slow takeoff just as bad? Are the odds of throwing together a FAI in the equivalent of a month any better than in a day? What % of those pannicked emergency FAI activities could be speed up by better computer user interfaces/text editors, personal assistants, a device that zapped your brain every time it detected Akrasia setting in, or by a RAM upgrade to the brain's working-memory?

(sorry to spam. I'm separating questions out to keep the discussion tidy.)