= 27d567bc2d48d9f40e9ccc20ea370b15)
wnoise comments on It's not like anything to be a bat - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (189)
No, it's proportional to the log of the number of patterns that can be (semi-stably) stored. E.g. n bits can store 2^n patterns.
I'd like to see a lot more justification for this. If each connection were binary (it's not), and connections were possible between all N neurons (they're not), than we would have N^2 bits.
Oops! Correct. That's what I was thinking, which is why I said info NlogN for N neurons. N neurons => max N^2 connections, 1 bit per connection, max N^2 bits, simplest model.
The math trying to estimate the number of patterns that can be stored in different neural networks is horrendous. I've seen "proofs" for Hopfield network capacity ranging from, I think, N/logN to NlogN.
Anyway, it's more-than-proportional to N, if for no other reason than that the number of connections per neuron is related to the number of neurons. A human neuron has about 10,000 connections to other neurons. Ant neurons don't.