I don't know how long you've been talking to real people, but the vast majority are not particularly good at feedback - less consistent than AI, but that doesn't make them more correct or helpful.  They're less positive on average, but still pretty un-correlated with "good ideas".  They shit on many good ideas, and support a lot of bad ideas. and are a lot less easy to query for reasons than AI is.

I think there's an error in thinking talk can ever be sufficient - you can do some light filtering, and it's way better if you talk to more sources, but eventually you have to actually try stuff.

Hmm.  What about the claim "pysicality -> no free will".  This is the more common assertion I see, and the one I find compelling.  

The simplicity/complexity I often see attributed to "consciousness" (and I agree: complexity does not imply consciousness, but simplicity denies it), but that's at least partly orthogonal to free will.

They can overgeneralize that feeling over all physical systems (like humans), missing out on the fact that this feeling should only be felt

I don't follow why this is "overgeneralize" rather than just "generalize".  Are you saying it's NOT TRUE for complex systems, or just that we can't fit it in our heads?   I can't compute the Mandelbrot Set in my head, and I can't measure initial conditions well enough to predict a multi-arm pendulum beyond a few seconds.  But there's no illusion of will for those things, just a simple acknowledgement of complexity.

TL;DR: The laws of physics seem to prevent us from accidentally erasing our own history through time travel.

What makes you think this?  I have some introspective consistency in my memories, but I can't tell if they're actually real, or if they've been (subjectively) recently changed/implanted or otherwise made to fit the "current" timeline.

We can start from a point of complete skepticism about everything. Regardless of your specific beliefs about the probability of a simulated universe, or if you are a pure idealist, you can say with confidence that at least something exists

If you can say that, you can say a lot more.  You can say what experiences and memories you have.  There is variance and a perception of time and change.  Oops, monism no longer makes sense.

I think this is correct, but I strongly doubt that any strong version of moral realism applies to our universe.  I further suspect that there's a separate argument you'd need to address: "if moral realism is true and we have wrong beliefs about moral truths, then correct beliefs could look nearly arbitrary".  I've not seen this second argument made (let alone rebutted), because most people I talk to don't give much weight to moral realism.

There's yet another argument against this in that steps 3 and 4 seem not to be universally true in humans - they might (or might) not have some explanatory power in a median individual, but we see plenty of examples of high-intelligence (for a human) presumably-immoral acts.  Even if it's correlated, individual instances of intelligence can vary widely in their moral actions.

The presumption of complete reducibility is, with some great certainty, unclear at best and at worst absolutely impossible,

Oh, I fully agree.  But "complete" is not necessary to achieve the change in categorization from "unknown and magical" to "just (big/difficult) math".  

I don't know how much playing people around here have done with Mandelbrot set coding, but it's a useful comparison in this.  It's very clearly NOT magic in any literal sense - the calculation is trivial (and even the iterations to determine converge/diverge is pretty easy to understand).  But the results remain captivating and astounding (to me) that they come from such simple rules.  

In this sense, I suspect many complex systems will remain impressive and astounding, no matter how good we get at modeling and understanding their components.  In the sense that knowing the underlying rules DOES turn it from "fully magical" into "an interesting corner of math", this will probably happen to current LLMs, and likely eventually to primate intelligence.