I am keeping this comic online as a record of an earlier perspective. My current work in fixed point logic and proof theory has led me to reject several foundational assumptions in mainstream alignment and interpretability research. This does not imply that AI poses no risks (it implies nothing about AI).
Ignore the Theory of Computation. Assume computers can realize smoothness and division, assume the Law of the Excluded Middle. Ignore Russel, Gödel, Tarski, Church, Brouwer, Rice, Friedman. Especially Rice. Ignore what SimPy and TensorFlow actually does. Ignore what CISC and RISC does. Go back to pre rigorous 18th century mathematics...
TLDR; (i) A general introduction to the most underrated tool of proof theory: the Kleene–Mostowski Hierarchy, cf. Arithmetical hierarchy (wikipedia). (ii) The claim is that this diagnostic tool should not remain confined to a specialty; it should be treated as standard equipment wherever one reasons about agents, verification, security and...
Any natural number can be uniquely written as a sum of non-consecutive Fibonacci numbers. This is called Zeckendorf representation. Consider, 15 = 2 + 13, or 54 = 2 + 5 + 13 + 34. This outlines a very weak RE language employing only {N,+,var}. We can also see that...
Consider computation as a thin, linear pipeline: symbol in, symbol out. But in a setting with feedback, branching recursion, and layered state spaces, computation becomes expressive in exactly the way biological systems are: through repeated structure, constraint, and variation, or with one word: algorithmic entropy. TLDR; In a proof theoretic...
Eliezer Yudkowsky has, on several occasions, claimed that AI’s success at protein folding was essentially predictable. His reasoning (e.g., here) is straightforward and convincing: proteins in our universe fold reliably; evolution has repeatedly found foldable and functional sequences; therefore the underlying energy landscapes must possess a kind of benign structure....
I am keeping this comic online as a record of an earlier perspective. My current work in fixed point logic and proof theory has led me to reject several foundational assumptions in mainstream alignment and interpretability research. This does not imply that AI poses no risks (it implies nothing about...
Ignore the Theory of Computation. Assume computers can realize smoothness and division, assume the Law of the Excluded Middle.
Ignore Russel, Gödel, Tarski, Church, Brouwer, Rice, Friedman.
Especially Rice.
Ignore what SimPy and TensorFlow actually does. Ignore what CISC and RISC does. Go back to pre rigorous 18th century mathematics and treat it as ground truth.
Ignore Type Theory but use its fruits.
Misread Bayes' Theorem as a philosophy, or worse, as a logic.
Ignore the fact that Newcombs Problem is just an instance of Lawvere's Fixed-point, endlessly discuss paradoxes of statistical frameworks.
Form strong opinions about them. Forget that the issues are baked in.
Ignore
But forget how to write
Think that
is enough.
Assume that intelligence is non-trivial, by assuming that triviality... (read more)
TLDR;
(i) A general introduction to the most underrated tool of proof theory: the Kleene–Mostowski Hierarchy, cf. Arithmetical hierarchy (wikipedia). (ii) The claim is that this diagnostic tool should not remain confined to a specialty; it should be treated as standard equipment wherever one reasons about agents, verification, security and uniform procedures. (iii) Real life examples from chess and a corporate toy model.
Observation. One thing to love about proof theory is its implicated pessimism: start as if the cause is already lost, set the task of failing elegantly.
Surprisingly, major failures of formal closure, often simultaneously opened a new methodological frontiers. It seems that... (read 1569 more words →)
Any natural number can be uniquely written as a sum of non-consecutive Fibonacci numbers. This is called Zeckendorf representation.
Consider,
15 = 2 + 13,
or
54 = 2 + 5 + 13 + 34.
This outlines a very weak RE language employing only . We can also see that it is able to encode:
https://www.milanrosko.com/demo/zeck.html (.js demo)
Because you cannot use adjacent Fibonacci numbers in the sum, each “gap” between Fibonacci indices becomes meaningful: which indices are used encodes structure in a kind of “sparse bitvector.” In that sense: if you leave at least one gap (i.e. you do not use consecutive Fibonacci numbers), you can use the pattern of used vs unused Fibonacci indices to store information additively.
Thus,... (read 341 more words →)
Consider computation as a thin, linear pipeline: symbol in, symbol out. But in a setting with feedback, branching recursion, and layered state spaces, computation becomes expressive in exactly the way biological systems are: through repeated structure, constraint, and variation, or with one word: algorithmic entropy.
TLDR; In a proof theoretic intuitionist[1] setting, we hint at the possibility that there are problems that are neither solvable, nor the opposite, more precisely:
... (read 1229 more words →)Once we make the isomorphism between provability and solvability explicit, certain decision problem-forms become unstable when they quantify over the full class of problems they can themselves represent. Under such conditions, self-reference (diagonalization) inevitably produces an instance whose status no uniform method can determine. The
Eliezer Yudkowsky has, on several occasions, claimed that AI’s success at protein folding was essentially predictable. His reasoning (e.g., here) is straightforward and convincing: proteins in our universe fold reliably; evolution has repeatedly found foldable and functional sequences; therefore the underlying energy landscapes must possess a kind of benign structure. If evolution can navigate these landscapes, then, with enough data and compute, machine learning should recover the mapping from amino-acid sequence to three-dimensional structure.
This account has rhetorical and inductive appeal. It treats evolution as evidence about the computational nature of protein folding and interprets AlphaFold as a natural consequence of biological priors. But the argument, as usually presented, fails to acknowledge what... (read 1003 more words →)
I am keeping this comic online as a record of an earlier perspective. My current work in fixed point logic and proof theory has led me to reject several foundational assumptions in mainstream alignment and interpretability research. This does not imply that AI poses no risks (it implies nothing about AI).
Update: After -45 Karma, I really wan't to make this point.
This is a logical consequence.
Rice’s theorem says: no uniform decision procedure can take an arbitrary program and decide a nontrivial semantic property. The reply (“we only need it for the systems we build”) seems to side-steps that quantifier.
So let's tighten it: How do you prove that a single specific agent is aligned across one possible states of the world? By a criterion. Some' possible states of the world? By multiple criteria. Across 'all relevant' states of the world? By all 'relevant' criteria. Problematic if one has no model and no criteria.
Might wan't to resort to a HOL probabilistic, game-theoretic, philosophy Ansatz instead... What will fail is to extract semantics from places semantics... (read more)
ah of course, it's because a workaround in the mathjx. thank you for noticing, i will fix it (fixable)
okay i wrote you a reply and apparently lesswrong thinks its llm written (which is insane) bottom line is that the standard is "Constructivism in Mathematics, Vol 1 (Studies in Logic and the Foundations of Mathematics, Volume 121)" by A.S. Troelstra and D. van Dalen i also explained how assuming less are not additional constraints but naturally correspond to a harder system and gave the irrationality of root 2 as an example for typical law of the excluded middle proof.
Thank you very much! This means a lot to me. Okay regarding the button...
The “button” is a metaphor or placeholder for what's opposite to the machine’s intrinsic terminal goal or the loss function that guides its decision-making and actions. It's not a tangible button anymore but a remnant.
Imagine training an AI by pressing a button to initiate backpropagation. When released into the wild, the AI continues to operate as if the button is still present, constantly seeking to fulfill the goal it once associated with that action.
This is similar to how many humans perceive death as a tangible concept, despite it not being a universal in an ontological sense. Our ancestors who behaved like... (read more)
You're not the kind of hack/rube this post is about because you are demonstrating hallmarks of critical thinking and epistemic humility.
You admit a knowledge gap, you reflect...
See the perfect example bellow.
While Gödel is indeed notorious in this sense, note that Rice does not count as "abused" outside the lesswrong community, rather, it is more like that people here cannot deal with it.
Unwillingness to accept limitative notions and to engage with formal logic when it's relevant makes this community isolated.
Also, once you reason constructively, it is hard to induce an "abuse" of limitative result because they are already implicit in a "falsehood as proof of absurdity".
Under LEM.
Gödel then becomes the "source" of "false" simple as that.
its smart think in terms in terms of abused theorems. generally be obsessed with what people abuse. add the second law to the list also. take a topos literally, oh wait, its as semantic equivocation is implicitly on the list.
Typical lesswrong word salad, e.g. "detailed metacognitive thoughts", "handle the consequences of being accurately seen"... or the best: "self-empowerment strategy. But it's more like, noticing you already have power."
I had it with this pseudo intellectual community.
On semantics:
Actually semantics "can" be conserved, an example.
- Change in semantics: "This is my second car" vs "this is my second skateboard".
- Change in syntactics: "This is my second car" vs "this is my third car".
- No formal change in the strict sense: "This is my second car" vs "This is—actually—my second car!"
Change during prompting: "Can be both, either, neither."
Considering the prompting itself:
I copy/pasted a rough drafts (2-4 paragraphs?) and wrote something like: "Improve." And then I edited again.
On the opaqueness:
The problem could be "limitative" thinking. Most of weak logics is, eliminative (e.g. implication A -> B is elimination of A etc.). We can only look for contradictions and check for soundness.
On... (read more)
This sounds like you are giving advice for someone that is rather interested in writing posts, not developing formal arguments... In logic we are searching for transitivity and rigor, not some modulo aesthetic preference, perhaps you overlooked that the content above is "somewhat" independent of its prosaic form.
"Most posts have flaws"
Where to find flaws in the argument presented... There could be several places where my reasoning can be challenged, the key is to do so without endorsing Yudkowsky’s views.
The game is a bit unfair, as it inherently easier to "destroy" positivistic arguments with first order inference: entropy, smoothness etc. is basically where all limitative theorems reside, arguments relying on these often collapse... (read more)