paper link author’s presentation This paper introduces the Koha model, which aims to explain how information is stored within the brain and how neurons learn to become pattern detectors. It proposes that dendritic spines serve as computational units that scan for particular temporal codes, or “spike trains”, which are used...
Kolmogorov complexity is a measure of the algorithmic complexity of a particular piece of information, and it is found by identifying the shortest possible representation of that information. For example, if I have a string of characters AAAAA, I can create a compressed representation 5A. The Kolmogorov complexity of AAAAA...
Preface: this blog post was written for a slightly more general audience than LessWrong. It's likely that you may already be familiar with many of the ideas explored in this post, but I thought some of you may find it interesting anyway. I've been exploring the idea that our conscious...
paper link
author’s presentation
This paper introduces the Koha model, which aims to explain how information is stored within the brain and how neurons learn to become pattern detectors. It proposes that dendritic spines serve as computational units that scan for particular temporal codes, or “spike trains”, which are used to dampen or excite signals. A mechanism for learning is proposed, wherein each neuron listens for an “echo” signal that is used to reinforce the physical structure of dendritic spines that contributed to the original signal. Competitive learning then drives neurons to self-organize into emergent algorithms.
Even leaving issues of quantum physics aside, macroscopic physical objects like humans are unlikely to be very compressible (information-wise, that is). The author might feel that the number of lead atoms in their 36 molar tooth is not part of their Kolmogorov string, but I would argue that is is certainly part of a complete description.
I don't know, just how compressible are we? I agree that the lead in my 36 molar is a part of my description, but anomalies such as these are always going to be the hardest part of compression since noise is not compressible. So maybe a complete description would look more like "all of the usual teeth,... (read more)
I imagine there could be two different compression strategies that both happen to produce a result of the same length, but cannot be merged.
I think this is correct, but I think of this as being similar to chirality - multiple symmetric versions of the same essential information. I think it also probably depends on the description language you use, so maybe in one language something might have multiple versions, but in another it wouldn't?
Kolmogorov complexity is a measure of the algorithmic complexity of a particular piece of information, and it is found by identifying the shortest possible representation of that information. For example, if I have a string of characters AAAAA, I can create a compressed representation 5A. The Kolmogorov complexity of AAAAA then is at most 2 because I can represent that same information with only two characters using the shorter version 5A. We can say that 5A is the Kolmogorov string that shows that the Kolmogorov complexity of AAAAA is 2.
Kolmogorov complexity can also give us a sense of the relative complexity of different pieces of information. AAABB can only be compressed down... (read 541 more words →)
To me, it really looks like brains and LLMs are both using embedding spaces to represent information. Embedding spaces ground symbols by automatically relating all concepts they contain, including the grammar for manipulating these concepts.
Preface: this blog post was written for a slightly more general audience than LessWrong. It's likely that you may already be familiar with many of the ideas explored in this post, but I thought some of you may find it interesting anyway.
I've been exploring the idea that our conscious minds, including self-awareness and subjective experience, might be a type of computer program.
Of course, the idea that our brains might be some kind of computer is nothing new, but I think we almost have all of the pieces necessary to make sense of that idea. I believe that if consciousness is a type of software, then this could explain why and how consciousness... (read 6371 more words →)