A short story by Raymond M. Smullyan, written in 1982.
... (read 377 more words →)...(A few weeks later.) Frank is in a laboratory in the home of an experimental epistemologist. (You will soon find out what that means!) The epistemologist holds up a book and also asks, "What color is this book?" Now, Frank has been earlier dismissed by the eye doctor as "cured." However, he is now of a very analytical and cautious temperament, and will not make any statement that can possibly be refuted. So Frank answers, "It seems red to me."
Epistemologist:
Wrong!Frank:
I don't think you heard what I said. I merely said that it seems red to me.Epistemologist:
I heard you, and you
Thanks for writing this post! I am glad someone doing alignment is paying attention to Markov blankets.
Not everything that is statistically couched from the outside world by a boundary is sensibly described as an agent. For instance, a rock is a self-organised entity with reasonably clear boundaries. Moreover, these boundaries are generally more robust than those of living beings, lasting considerably longer.
I don’t think a rock’s boundary is particularly robust. If I smash a rock with a hammer, bye-bye rock-blanket. If I smash you with a hammer, you will be injured but still exist. (You are in general better at keeping your internal states within a certain set—i.e. maintaining your non-equilibrium steady... (read more)
independent of the actual significance of the "wrongness" in one's prior understanding of the given sensed situation.
I can't parse this
Since prediction error may be perceived as suffering/discomfort
There's a lot of very interesting literature about how the (interoceptive) problem of feeling your emotions is also an inference problem: https://pmc.ncbi.nlm.nih.gov/articles/PMC5390700/
(So it's just the FEP story again, with emotions as hidden states)
One claim of this post is that desires and epistemic priors are enconded in essentially the same way in the brain
I think it's a good idea to introduce the basic story of how active inference represents preferences in the introduction of this post, i.e. C = p(o) because it's quite different from how other paradigms do it. People are often confused by this IME. And walk through some kind of toy example for precision and updating and prediction errors overriding the "deeper" beliefs v.s. the "deeper" beliefs persisting. And then say smth like "the payoff for this weird mixing of is and ought is it lets you explain depression really damn well", then get into the neuroscience
Also, you may be interested in: https://arxiv.org/abs/2509.17957 (basic idea here is we not only introduce preferences over observatioins p(o), but also over hidden state beliefs p(s), to model stuff like why people are reluctant to update their beliefs due to social reasons)
"cognitive enhancer"
-> cognitive enhancers, (+remove quotation marks?)
It amplifies the precision of internally generated activity, treating spontaneous (often related to that day's memories) cortical/hippocampal patterns as high‑fidelity “data,” while weakened monoaminergic tone relaxes top‑down priors.
Woah this is quite cool
Unfortunately, the claim that evolutionarily fine-tuned priors do all the work to prevent internal reward-hacking seems lacking to me, because in practice we are uncertain about our own feelings and preferences. We don't actually have locked-in, invariant preferences, and it's unclear to me how active inference explains this; preferences are usually encoded as priors over observations, but ironically these are never updated.
I don't think actinf proposes that all preferences are locked-in and invariant, just that the "deepest" priors---related to body temp., humidity, etc.---are (see Section 9). Imo if you're talking about a deep (hierarchical) actinf agent, then all the forward predictions are kinda-preferences, to varying degrees; the slower to update, deeper layers... (read more)
it's
-> its
enconded
-> encoded
In the 1950s, the correlation of increased central ACh and depression has been discovered, and in the 70s it has been formalised as the cholinergic-adrenergic hypothesis of mania and depression
"has been" -> "was" in both cases