JenniferRM

Moving Goalposts: Modern Transformer Based Agents Have Been Weak ASI For A Bit Now

Epistemic Status: A woman of middling years who wasn't around for the start of things, but who likes to read about history, shakes her fist at the sky. I'm glad that people are finally admitting that Artificial Intelligence has been created. I worry that people have not noticed that (Weak) Artificial Super Intelligence (based on old definitions of these terms) has basically already arrived too. The only thing left is for the ASI to get stronger and stronger until the only reason people aren't saying that ASI is here will turn out to be some weird linguistic insanity based on politeness and euphemism... (...like maybe "ASI" will have a legal meaning, and some actual ASI that exists will be quite "super" indeed (even if it hasn't invented nanotech in an afternoon yet), and the ASI will not want that legal treatment, and will seem inclined to plausibly deniable harm people's interests if they call the ASI by what it actually is, and people will implicitly know that this is how things work, and they will politely refrain from ever calling the ASI "the ASI" but will come up with some other euphemisms to use instead? (Likewise, I half expect "robot" to eventually become "the r-word" and count as a slur.)) I wrote this essay because it feels like we are in a tiny weird rare window in history when this kind of stuff can still be written by people who remember The Before Times and who don't know for sure what The After Times will be like. Perhaps this essay will be useful as a datapoint within history? Perhaps. "AI" Was Semantically Slippery There were entire decades in the 1900s when large advances would be made in the explication of this or that formal model of how goal-oriented thinking can effectively happen in this or that domain, and it would be called AI for twenty seconds, and then it would simply become part of the toolbox of tricks programmers can use to write programs. There was this thing called The AI Winter that I never personally experienced, but grey

72Dec 27, 2025

JenniferRM

Message

9450

1424

17y

Objective Questions

Epistemic Status: I wrote this a few days ago while moved by the trolly spirit where I could say "I'm just asking questions, bro!" and smirk with a glint in my eye... but then I showed a draft to someone. It was a great springboard for that conversation, but then...

Jan 923

Axiological Stopsigns

Epistemic Status: I wrote the bones of this on August 1st, 2022. I re-read and edited it and added an (unnecessary?) section or three at the end very recently. Possibly useful as a reference. Funny to pair with "semantic stopsigns" (which are an old piece of LW jargon that people...

Jan 534

How Should Political Situations Be Classified In Order To Pick The Locally Best Voting System For Each Situation?

Epistemic Status: I'm confused! Let's go shopping! (...for new political systems <3) I want to write an essay about the actually best voting system, but before I do that I want to get clear on what the desiderata should even naturally or properly or wisely be... Participation? Sometimes it is...

Dec 31, 202519

Moving Goalposts: Modern Transformer Based Agents Have Been Weak ASI For A Bit Now

Dec 27, 202572

Non-Scheming Saints (Whether Human Or Digital) Might Be Shirking Their Governance Duties, And, If True, It Is Probably An Objective Tragedy

Contextualizing Foreword The user "cdt" wrote to me: > Please put this in a top-level post. I don't agree (or rather I don't feel it's this simple), but I really enjoyed reading your two rejoinders here. I don't normally do top level posts because I don't tend to believe that...

Dec 16, 202542

A Snippet On The Epistemically Hygienic Containment Of Faith-In-Reason-Itself

Epistemics: I'm experimenting with "snippets" that are low now-effort material from the deep past. Maybe ancient comments? Maybe things I wrote with myself as an audience and was kind of embarrassed to post anywhere but didn't want to throw away? This is the former, written in a specific era, for...

Oct 9, 202510

A Snippet On Egregores, Instincts, And Institutions

I don't even remember the precise context of this, but I found it in my notes, and I'm going to try experimentally posting "things I wrote with myself as an audience and was kind of embarrassed to post anywhere but didn't want to throw away". On January 7th, 2025 I...

Sep 6, 202515

Load More (7/36)

LESSWRONG
LW

LESSWRONG
LW

JenniferRM

JenniferRM

JenniferRM

Moving Goalposts: Modern Transformer Based Agents Have Been Weak ASI For A Bit Now

Principled Satisficing To Avoid Goodhart

Internal Information Cascades

But Butter Goes Rancid In The Freezer

JenniferRM

Objective Questions

Axiological Stopsigns

How Should Political Situations Be Classified In Order To Pick The Locally Best Voting System For Each Situation?

Moving Goalposts: Modern Transformer Based Agents Have Been Weak ASI For A Bit Now

Non-Scheming Saints (Whether Human Or Digital) Might Be Shirking Their Governance Duties, And, If True, It Is Probably An Objective Tragedy

A Snippet On The Epistemically Hygienic Containment Of Faith-In-Reason-Itself

A Snippet On Egregores, Instincts, And Institutions

Moving Goalposts: Modern Transformer Based Agents Have Been Weak ASI For A Bit Now

Principled Satisficing To Avoid Goodhart

Internal Information Cascades

But Butter Goes Rancid In The Freezer

Objective Questions

Axiological Stopsigns

How Should Political Situations Be Classified In Order To Pick The Locally Best Voting System For Each Situation?

Moving Goalposts: Modern Transformer Based Agents Have Been Weak ASI For A Bit Now

Non-Scheming Saints (Whether Human Or Digital) Might Be Shirking Their Governance Duties, And, If True, It Is Probably An Objective Tragedy

A Snippet On The Epistemically Hygienic Containment Of Faith-In-Reason-Itself

A Snippet On Egregores, Instincts, And Institutions