= 27d567bc2d48d9f40e9ccc20ea370b15)
Mark_Friedenbach comments on AIFoom Debate - conclusion? - Less Wrong Discussion
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (48)
If you are assuming that a neuron contributes less than 2 bits of state (or 1 bit per 500 synapses) and 1 computation per cycle, then you know more about neurobiology than anyone alive.
I don't understand your statement.
I didn't say anything in my post above about the per neuron state - because it's not important. Each neuron is a low precision analog accumulator, roughly up to 8-10 bits ish, and there are 20 billion neurons in the cortex. There are another 80 billion in the cerebellum, but they are unimportant.
The memory cost of storing the state for an equivalent ANN is far less than than 20 billion bytes or so, because of compression - most of that state is just zero most of the time.
In terms of computation per neuron per cycle, when a neuron fires it does #fanout computations. Counting from the total synapse numbers is easier than estimating neurons * avg fanout, but gives the same results.
When a neuron doesn't fire .. .it doesn't compute anything of significance. This is true in the brain and in all spiking ANNs, as it's equivalent to sparse matrix operations - where the computational cost depends on the number of nonzeros, not the raw size.