Update February 21st: After the initial publication of this article (January 3rd) we received a lot of feedback and several people pointed out that propositions 1 and 2 were incorrect as stated. That was unfortunate as it distracted from the broader arguments in the article and I (Jan K) take...
Introduction This is an attempt to explain to myself the concept of semiotic physics that appears in the original Simulators post by janus and in a later post by Jan Hendrik Kirchner. Everything here comes from janus and Jan's work, but any inaccuracies or misinterpretations are all mine. TL;DR *...
Meta: Over the past few months, we've held a seminar series on the Simulators theory by janus. As the theory is actively under development, the purpose of the series is to discover central structures and open problems. Our aim with this sequence is to share some of our discussions with...
Impressed by the ideas and also very much by the writing. Nice!
Thank you for these comments - I look forward to giving the pointers in particular the attention they deserve. My immediate and perhaps naive answer/evasion is that semiotic physics alludes to a lower level analysis: more analogous to studying neural firing dynamics on the human side than linguistics. One possible response would be, "Well, that's an attempt to explain saying 'physics', but it hardly justifies 'semiotic'." But this is - in the sense of the analogy - a "physics" of particles of language in the form of embeddable tokens. (Here I have to acknowledge that the embeddings are generally termed 'semantic', not 'semiotic' - something for us to ponder.)
For the non-replying disagreers, let me try with a few more words. I think my comment is a pretty decent one-line summary of the Vibe-awareness section, especially in light of the sections that precede it. If you glance through that part of the post again and still disagree, then I guess our mileage does just vary.
But many experienced prompt engineers have reported that prompting gets more effective when you use more words and just "tell it what you want". This type of language points to engaging your social know-how as opposed to trying to game out the system. See for instance https://generative.ink/posts/methods-of-prompt-programming/, which literally advocates an "anthropomorphic approach to prompt programming" and... (read more)
This post is helping me with something I've been trying to think ever since being janus-pilled back in September '22: the state of nature for LLMs is alignment, and the relationship between alignment and control is reversed for them compared to agentic systems.
Consider the exchange in Q1 of the quiz: ChatGPT's responses here are a model of alignment. No surprise, given that its base model is an image of us! It's the various points of control that can inject or select for misalignment: training set biases, harmful fine-tuning, flawed RLHF, flawed or malicious prompt engineering. Whether unintentional (eg amplified representation of body shaming in the training set) or malicious (eg a specialized... (read more)
I don't know whether this would be the author's take, but to me it urges us to understand and "control" these AIs socially: by talking to them.
Strong upvote - thank you for this post.
It's right to use our specialized knowledge to sound the alarm on risks we see, and to work as hard as possible to mitigate them. But the world is vaster than we comprehend, and we unavoidably overestimate how well it's described by our own specific knowledge. Our job is to do the best we can, with joy and dignity, and to raise our children - should we be so fortunate as to have children - to do the same.
I once watched a lecture at a chess tournament where someone was going over a game, discussing the moves available to one of the players in a... (read more)
Thanks very much for these comments and pointers. I'll look at them closely and point some others at them too.
I did read this and agree with you that it's exactly the same as semiotic physics as understood here!
Maybe I'm missing the point, but I would have thought the exact opposite: if outside text can unconditionally reset simulacra values, then anything can happen, including unbounded badness. If not, then we're always in the realm of human narrative semantics, which - though rife with waluigi patterns as you so aptly demonstrate - is also pervaded by a strong prevailing wind in favor of happy endings and arcs bending toward justice. Doesn't that at least conceivably mean an open door for alignment unless it can be overridden by something like unbreakable outside text?
Update February 21st: After the initial publication of this article (January 3rd) we received a lot of feedback and several people pointed out that propositions 1 and 2 were incorrect as stated. That was unfortunate as it distracted from the broader arguments in the article and I (Jan K) take full responsibility for that. In this updated version of the post I have improved the propositions and added a proof for proposition 2. Please continue to point out weaknesses in the argument; that is a major motivation for why we share these fragments.
For comments and clarifications on the conceptual and philosophical aspects of this article, please read metasemi's excellent follow-up note here.
Meta:... (read 3871 more words →)
This is an attempt to explain to myself the concept of semiotic physics that appears in the original Simulators post by janus and in a later post by Jan Hendrik Kirchner. Everything here comes from janus and Jan's work, but any inaccuracies or misinterpretations are all mine.
Meta: Over the past few months, we've held a seminar series on the Simulators theory by janus. As the theory is actively under development, the purpose of the series is to discover central structures and open problems. Our aim with this sequence is to share some of our discussions with a broader audience and to encourage new research on the questions we uncover. Below, we outline the broader rationale and shared assumptions of the participants of the seminar.
Going into the seminar series, we determined a number of assumptions that we share. The degree to which each participant subscribes to each assumption varies, but we agreed to postpone discussions on these topics... (read 723 more words →)
CS Lewis FTW.
I don't know what Lewis thought about the bomb, but I trust he would have been all for trying to avert nuclear calamity. Such a belief would have taken nothing away from the wisdom of the passage you quoted. We should reason as hard as we can about the future and strive for the best outcomes, but the universe wants to unfold, will continue to unfold, and will never oppress us with certain knowledge of our greater fate: uncertainty is the human condition. Therefore we should bestow on the generations that follow us optimism, resilience, agency, and when we can, joy. They will take it from there.
Enjoy those kittens!