I agree with this.
Electrode-based neurotechnology that conceivably be in humans over the next 5 years have channel counts of hundreds (e.g. Utah arrays) to thousands (e.g. Neuralink, Paradromics), or generously/best-case tens of thousands. In an optimistic scenario, you could spike sort several neurons per contact, but an assumption of one neuron per electrode, on average, is probably about right.
Stevenson and Kording plotted the number of neurons we can record simultaneously as a function of time (over ~60 years). They estimated the doubling time to be ~7.4 years. Mouse brains have ~E8 neurons and human brains have ~E11 neurons. At this rate, assuming we're starting with 10k neurons today... (read more)
Steering vector: "I talk about weddings constantly" - "I do not talk about weddings constantly" before attention layer 20 with coefficient +4
Front Middle Back Average number
of wedding words0.70 0.81 0.87
@lisathiergart I'm curious if a linear increase in the number of words with position along the residual stream replicates for other prompts. Have you looked at this?
Disclaimer: This post is preliminary and doesn't yet fully align with the rigorous standards we typically aim for in LessWrong publications. It recaps the dynamic discussions from the neurotech for AI alignment workshop and therefore does not necessarily represent any single participant’s or author’s viewpoint. Shared ahead of the Foresight WBE workshop, our intent is to foster enriching discussion among participants. We welcome your input as we continue refining these insights.
This document reviews key insights from a January workshop exploring the potential of Neurotech to contribute to the AI alignment problem. Neurotech, or neurotechnology, refers to the set of tools and methods designed to enhance understanding of the brain, manipulate its function, or... (read 4033 more words →)
I agree that electrode-based BCIs don't scale, but electrode BCIs are just the first generation of productized interfaces. The next generation of BCIs holds a great deal of promise. Depending on AGI timelines, they may still be too far out. They're still probably worth developing with an eye toward alignment given that they have primarily non-overlapping resources (funding, expertise, etc.).
Butcher number & Stevenson/Kording scaling discussed more in the comments here: https://www.lesswrong.com/posts/KQSpRoQBz7f6FcXt3#comments