Thank you for the thoughtful reply.

I think a large part of what makes me a machine functionalist is an intuition that neurons...aren't that special. Like, you view the China Brain argument as a reductio because it seems so absurd. And I guess I actually kind of agree with that, it does seem absurd that a bunch of people talking to one another via walkie-talkie could generate consciousness. But it seems no more absurd to me than consciousness being generated by a bunch of cells sending action potentials to one another.

Aren't neurons special? At the very least, they're mysterious. We're far from understanding them as physico-chemical systems. I've had the same reaction and incredulity as you to the idea that interacting neurons can 'generate consciousness'. The thing is, we don't understand individual neurons. Yes, neurons compute. The brain computes. But so does every physical system we encounter. So why should computation be the defining feature of consciousness? It's not obvious to me. In the end, consciousness is still a mystery and machine functionalism requires a leap of faith that I'm not prepared to take without convincing evidence.

But even beyond that, it seems intuitively obvious to me that your brain's counterfactual dependencies are what make your brain, your brain.

Yes, counterfactual dependencies appear necessary for simulating a brain (and other systems) but the causal structure of the simulated objects is not necessarily the same as the causal structure of the underlying physical system running the simulation, which is my objection to Turing machines and Von Neumann architectures.

you could just as easily say that neurons are "simulating" consciousness. Essentially machine functionalists think that causal structure is all there is in terms of consciousness, and under that view the line between something being a "simulation" versus being "real" kind of disappears.

it's an interesting thought, and I generally agree with this. The question seems to come down to defining causal structure. The problem is that the causal structure of the computer system running a simulation of an object does not appear anything like that of the object. A Turing machine running a human brain simulation appears to have a very different causal structure compared with the human brain.