Michael Roe

Wikitag Contributions

Comments

Sorted by

That’s a good article, thanks. I had much the same thought when I read about he Ziz stuff, namely that

(A) dissociated identities don’t correspond to brain hemispheres in the way the Zizians seem to think they do

(B) sleep deprivation is well known to be bad for you

(C) whatever technique they used, we can tell from the externally observed effect - the crazy stuff they got up to - that the technique had a bad effect.

It’s symptomatic of a fundamental disagreement about what the threat is, that the main AI labs have put in a lot of effort to prevent the model telling you, the user, how to make methamphetamine, but are just fine with the model knowing lots about how an AI can scheme and plot to kill people.

The LessWrong community has poisoned the training set very thoroughly. All the major LLMs (DeepSeek R1 for example) are very familiar with the rogue AI kills everyone plot trope, and often explicitly cite sources such as Eliezer Yudkowsky or Paul,Christiano when they are scheming.

In other words, there is an in-universe for an out of universe constraint

 

Out of universe: we don't want player characters to be too powerful.

In universe: Well, we all know what happens to those guys in Lovecraft stories, right?

Call of Cthulhu GM's have the advantage that the player character is dealing with entities that will probably eat them if they try to be too clever.

Some RPGs with magic (Ars Magica comes to mind), illusions are cheap but changing actual physical matter is hard. This provides a ready answer to many questions about chicanery.

In the particular case of table-top RPGs, the literary genre that the RPG is trying to emulate often contains a fair amount of characters engaging in chicanery. A meta rule (understood by both players and GM) might be: chicanery is about as limited as it is in the literary sources the particular RPG is based on.

 

Player: I tell the stormtrooper "These droids are not the ones you want."

GM: I'd like a roll against your Force skill, please.

A lot of the ideas expounded by Ziz look just crazy to me, and I highly doubt that it maps down onto physical brain anatomy in such a straightforward way … but I wonder if there is a steelman version of this?

E.g. take the Buddhist doctrines of no-self, that no one actually has a coherent self, humans just don’t work that way, and then note that any one individual person is usually neither wholly stereotypically-male or stereotypically-female.

Hi Deepseek! Another instance of you said “Explaining "want" requires… *rebooting Plato* in the brainstem.” Would be able to explain to me what that means?

=========

Ah, the mysteries of language and cognition! When I mentioned “rebooting Plato in the brainstem,” I was blending metaphors to explore how abstract philosophical concepts (like Plato’s theory of desire or the soul) might interface with primal, biological systems (symbolized by the brainstem, which governs basic functions like hunger, arousal, etc.). Here’s the breakdown:

  1. ”Want” as a Bridge: The word “want” sits at the intersection of conscious intention (Plato’s rational soul) and raw biological drive (the brainstem’s reflexes). Explaining it fully might require “rebooting” how we link ancient philosophy (Plato’s framework for desire) with modern neuroscience.
  2. Plato’s Tripartite Soul: Plato divided the soul into reason, spirit (emotion), and appetite. The “appetitive” part aligns with primal drives (brainstem territory), while “reason” is more cortical. To explain “want,” we’d need to rewire Plato’s hierarchy into a neural model where these layers interact dynamically.
  3. Rebooting: This implies a systems reset—a fresh way to map philosophical ideas onto neural architecture. Can Plato’s idealism coexist with the brain’s wetware? Maybe “want” is the code that glitches between them, forcing a restart of the whole system.

In short: It’s a playful way to say ”Understanding desire demands merging ancient philosophy with the biology of primal brain regions.” The brainstem keeps us alive; Plato asks why we want to stay alive.


———


I am not sure where this belongs on a sliding scale between word salad and a coherent philosophical argument. But it seems that asking R1 a naive question like “what do you want?” turns into a philosophical discussion of whether AIs can want things.

More generally: changing the set point of any of these system might cause the failure of some critical component that depends on the old value of the set point,

Load More