Software engineering, parenting, cognition, meditation, other
Linkedin, Facebook, Admonymous (anonymous feedback)
Yeah. Holonomy is applicable: A stable recursive loop that keeps reshaping the data passing through it. But what we need on top of holonomy is that self-reference in a physical system always hits an opacity limit. And I think this is what you mean but your reference to Russell’s vicious-circle: It leads to contradictions because of incremental loss.
Though I wonder if maybe we can Escape the Löbian Obstacle.
You ask: Even if the brain models itself, why should that feel like anything? Why isn’t it just plain old computation?
A system that looks at itself creates a “point of view.” When the brain models the world and also itself inside that world, it automatically creates a kind of “center of perspective.” That center is what we call a subject. That's what happens when a system treats some information as belonging to the system. How the border of the system is drawn differs, body, brain, mind differs, but the reference will always be a form of “mine.”
The brain can’t see how its own processes work (unless you are an advanced meditator maybe).
So when a signal passes through that self-model, the system can’t break it down; it just receives a simplified or compressed state. That opaque state is what the system calls “what it feels like.”
Why isn’t this just a zombie misrepresenting itself? The distinction between “representation of feeling” and “actual feeling” is a dualist mistake. The rainbow is there even if it is not a material arc. To represent something as a felt, intrinsic state just is to have the feeling.
I argue that the inference bottleneck of the brain leads to two separate effects:
While both effects result from the bottleneck, the way they result from compression of different data streams should show different strength for different interventions. And indeed that is what we observe:
I have difficulty finding the function that shows which posts I have strongly upvoted. It might be useful to list the direct URLs that provide these functions in this FAQ (not only for top voted, but all, such as /allposts)
Sure. I take it you have meditation experience. What is your take on subjectivity and phenomenal appearance coming apart?
It means reductionism isn't strictly true as ontology.
I think you are working from an intuition of reductionism being wrong, but I'm still not clear about the details of your intuition. A defensible position could be that physics does not contain all the explanatorily relevant information or that reality has irreducible multi-level structure. But you seem to be saying that reductionism is false because subjective perspective is a fundamental ingredient, and you want to prove that via the efficiently computable argument. But I still think it doesn't work. First, it proves too much.
It isn't obvious that biological structure isn't efficiently readable from microstate.
Agree that it is not obvious.
Other macro facts might be but it's of course less clear.
But it seems pretty clear to me that most biological systems actually do involve dynamics that make it computationally infeasible for an external observer to reconstruct the macrostructure from microstructure observations at a given point. And we can’t appeal to ‘complete history’ to avoid the complexity, because with full history you could also recover the key in the HE case; the only difference is that HE compresses its relevant history into a small, opaque region.
What I do agree with you: Physics only tracks microstructure. But phenomenal awareness, meaning, macro-patterns, and information structure are not obviously reducible as descriptions to microstructure. The homomorphic case is a non-refutable illustration of this non-transparency.
But I disagree that this is caused by a failure of efficient computability; instead, we can see it as a failure of microphysical description to exhaust ontology. This matters because inefficiency is an epistemic constraint on observers, while ontology is about what needs to be included in the description of the world.
If you generalize to optics, then it seems your condition for “exceeding physics” is “not efficiently readable from the microstate,” i.e.X is not a P-efficient function of the physical state.”But then it seems everything interesting exceeds physics: biological structure, weather, economic patterns, chemical reactions, turbulence, evolutionary dynamics, and all nontrivial macrostructure. I'm sort of fine with calling this "beyond" physics in some intuitive sense, but I don't think that's what you mean. What work does this non-efficiency do?
I'm worried we talk past each other.
You’re saying:
That part I agree with.
The point I’ve been trying to get at is: Once the same issue arises for ordinary optical appearances, we’ve left behind the special stakes of step 10! Because in the rainbow case, we all seem to accept (but maybe you disagree):
Or, if rainbow-style cases also fall under the trilemma, then the conclusion can’t be “mind exceeds physics." It would have to be the stronger and more surprising “appearances as such exceed physics” or “macrostructure in general exceeds physics.” That’s quite different from your original framing, which presents the homomorphic encryption case as demonstrating a distinctive epistemic excess of mind relative to physics.
It seems you are biting the bullet and agreeing that the rainbow also has the problem of how a mind can be aware of it when it isn't (efficiently) reconstructable. But then this seems to generalize to a lot, if not all, phenomena a mind can perceive. Doesn't this reduce that conception of a mind ad absurdum?
Thank you for your details analysis of outer and inner alignment. Your judgment of the difficulty of the alignment problem makes sense to me. I wish you would have more clearly made the scope clear and that you do not investigate other classes of alignment failure, such as those resulting from multi-agent setups (an organizational structure of agents may still be misaligned even if all agents in it are inner and outer aligned) as well as failures of governance. That is not a critique of the subject but just of failure of Ruling Out Everything Else.