The reason why we want editing for IQ is because we want something unusual like "+1SD above von Neumann", I'm not sure we want something beyond statistical range of human personality traits. Why do not select outliers from population using personality testing and give them high intelligence?

I think the simplest objection to your practical CF part is that (here and further, phenomenal) consciousness is physiologically and evolutionarily robust: an infinite number of factors can't be required to have consciousness because the probability of having an infinite number of factors right is zero.

On the one hand, we have evolutionary robustness: it seems very unlikely that any single mutation could cause Homo sapiens to become otherwise intellectually capable zombies.

You can consider two extreme possibilities. Let's suppose that Homo sapiens is conscious and Homo erectus isn't. Therefore, there must be a very small number of structural changes in the brain that cause consciousness among a very large range of organisms (different humans), and ATP is not included here, as both Homo sapiens and Homo erectus have ATP.

Consider the opposite situation: all organisms with a neural system are conscious. In that case, there must be a simple property (otherwise, not all organisms in the range would be conscious) common among neural systems causing consciousness. Since neural systems of organisms are highly diverse, this property must be something with a very short description.

For everything in between: if you think that hydras don't have consciousness but proconsuls do, there must be a finite change in the genome, mRNAs, proteins, etc., between a hydra egg and a proconsul egg that causes consciousness to appear. Moreover, this change is smaller than the overall distance between hydras and proconsuls because humans (descendants of proconsuls) have consciousness too.

From a physiological point of view, there is also extreme robustness. You need to be hit in the head really hard to lose consciousness, and you preserve consciousness under relatively large ranges of pH. Hemispherectomy often doesn't even lead to cognitive decline. Autism, depression, and schizophrenia are associated with significant changes in the brain, yet phenomenal consciousness still appears to be here.

EDIT: in other words, imagine that we have certain structure in Homo sapiens brain absent in Homo erectus brain which makes us conscious. Take all possible statements distinguishing this structure from all structures in Homo erectus brain. If we exclude all statements, logically equivalent to "this structure implements such-n-such computation", we are left... exactly with what? We are probably left with something like "this structure is a bunch of lipid bubbles pumping sodium ions in certain geometric configuration" and I don't see any reasons for ion pumping in lipid bubbles to be relevant to phenomenal consciousness, even if it happens in a fancy geometric configuration.

I meant by "authority" not brainwashing but, like, the fact that AGIs are smarter and know better: "If you believe you can survive without taking over the world, please consult the graph". If China hawks believe themselves to be correct, they expect AGIs to actually prove that.

I agree this scenario is terrible.

I think that you miss possibility of leveraging AGI authority here. I.e., if your military/natsec-aligned AI will scream at your superiors "ATTACK NOW YOU FOOLS", maybe your superiors will be more inclined to listen to plans to melt all GPUs.

Read your comment, donated 10$.

For me, "enjoying life" means "enjoying good things that happen in life", while everything in-between is neutral at best. Many people seem to put positive value on "in-between" space.

If I try to point out, some people enjoy just the fact of being embodied.

After yet another news about decentralized training of LLM, I suggest to declare assumption "AGI won't be able to find hardware to function autonomously" outdated. 

Okay, I kinda understood where I am wrong spiritually-intuitively, but now I don't understand where I'm wrong formally. Like which inference in chain

not Consistent(ZFC) -> some subsets of ZFC don't have a model -> some subsets of ZFC + not Consistent(ZFC) don't have a model -> not Consistent(ZFC + not Consistent(ZFC))

is actually invalid?