LESSWRONG
LW

Nicholas Kross — LessWrong

Replying toStatus Is The Game Of The Losers' Bracket

Status Is The Game Of The Losers' Bracket

EA global, like many other effective altruism branded “networking events”, is (at least in my head) the losers’ bracket of the effective altruism job market.

Can... uh... at least partly confirm 🙃

Replying toDrugs Aren't A Moral Category

Nicholas Kross2mo

Drugs Aren't A Moral Category

Can confirm that cultural messaging around drugs made me delay useful mental health treatment.

(Some) left straw-view: Meds are a way for The System to enslave you and destroy your creativity/individuality/humanity.
(Some) libertarian straw-view: You're a strong smart Galt-like ubermensch already, why would you need drugs as a crutch?
Conservative depressingly-common view: All this is fake, mental health is fake, you're just a whiny pussy who can't handle real life, bootstrap yourself.

All these views intersected into a rarely-explicit "I don't need medicine for my brain!" that hampered my life trajectory.

Then again, drug effects can have complex interactions with one's thinking, so who knows? Maybe I would've been more productive in high school and gotten into a better college... and then proceeded to work on the wrong things! (This is what some rationalists think happened with rats/EA as a whole, or OpenPhil, or therabouts something something moderation vs extremism yadda yadda)

Replying toBefore LLM Psychosis, There Was Yes-Man Psychosis

Nicholas Kross6mo

Before LLM Psychosis, There Was Yes-Man Psychosis

I think lots of humans are also just starved for compliments in general, outside of contexts like "You are a waiter and the social script tells the table to thank you" or "You are someone's spouse". The classic example in my mind is "male vs female socialization" and its consequences, e.g. as discussed in this screenshotted tumblr post or this other one. Men often, in some sense, get "too few" wanted compliments, and women get inundated with too many unwanted ones. "It's like one person dying of dehydration watching another one drown". Then, of course, classic nightmarish social/cultural/internet incentives layer atop this dynamic and make it worse.

Widespread chronic human under-complimented-ness seems relatively easy to explain, from the supply-side. How often does the average person hand out unsolicited compliments, outside of well-known contexts like the restaurant example above? I'd hazard a guess of "too little". People could easily be "well-put-together" and still suffering from this, just as "feeling full" and "having proper nutrition" aren't the same.

Nicholas Kross6mo

I'm curious to see if I'm in this data, so I can help make it more accurate by providing info.

Nicholas Kross8mo

https://www.lesswrong.com/posts/tpZciMYCXN49FYWnS/nicholaskross-s-shortform?commentId=f4PxFp8LkKKxdCXxh

Replying toEliezer and I wrote a book: If Anyone Builds It, Everyone Dies

Nicholas Kross8mo

Eliezer and I wrote a book: If Anyone Builds It, Everyone Dies

From the MIRI announcement:

Our big ask for you is: If you have any way to help this book do shockingly, absurdly well— in ways that prompt a serious and sober response from the world — then now is the time.

sober response from the world

sober response

Uh... this is debatably a lot to ask of the world right now.

Something else I just realized: Georgism is a leftish idea that recognizes some (but not all) leftish ideas I've discussed or referenced above, and its modern form is currently rationalist-adjacent. Progress!

Ah, sorry yeah I think it was a mistake on my part to mostly make the post a verbatim Discord reply. Lots of high-context stuff that I didn't explain well.

This specific part is (in my usage/interpretation; if you click the link, the initial context was an Emmett Shear tweet) basically a shorthand for one or more "basic" leftist views, along the lines of these similar-but-somewhat-distinct claims:

Capitalism more-reliably rewards power-maximizers than social-utility-maximizers.
Under capitalism and similar incentive-structures, we'd expect conflict theory to predict entities' wealth better than mistake theory.
General outcomes, under capitalism and similar incentive-structures, are downstream of "brute power" (from guns to monopolies) far more than the things we'd "want" to reward (innovation, good service, helping people, etc).

In hindsight, I over-updated on my previous success with a poorly-written angry short post with a clickbait title and lots of inline links criticizing the rationality community. Oops.

Replying toRationalist Movie Reviews

Nicholas Kross1y

Rationalist Movie Reviews

I said "one of the best movies about", not "one of the best movies showing you how to".

Rationalist Movie Reviews

Nicholas Kross

Primer (2004)

Content warnings here

What would actually happen, in real life (circa 2004), if two typical techie engineers invented time travel by accident?

Primer is odd for being both slow-burning and fast-paced. It's short and self-contained and low-budget, yet majestic and complicated and, er, "recursive".

I haven't eaten lunch since later this afternoon.

The parallels with AI alignment, especially the ongoing totally-not-an-AI-arms-race, run deep. Ordinary (if nerdy) humans, grappling with the same ambitions and fears and uncertainties humans always have. In isolation, a well-acted drama. But amplified by technology that directly messes with causation itself... and the implications spill like dark pools, recursively unfolding into tentacles that colonize the Everett branches with iron-cold power.

It's this entirely

... (read 874 more words →)

Virgin 2030s [sic] MIRI fellow:
- is cared for so they can focus on research
- has staff to do their laundry
- soyboys who don't know *real* struggle
- 3 LDT-level alignment breakthroughs per week

CHAD 2010s Yudkowsky:
- founded a whole movement to support himself
- "IN A CAVE, WITH A BOX OF SCRAPS"
- walked uphill both ways to Lightcone offices.
- alpha who knows *real* struggle
- 1 LDT-level alignment breakthrough per decade

Is principled mass-outreach possible, for AGI X-risk?

Nicholas Kross

Over a year ago, Rohin Shah wrote this, about people trying to slow or stop AGI development through mass public outreach about the dangers of AGI:

But it really doesn't seem great that my case for wide-scale outreach being good is "maybe if we create a mass delusion of incorrect beliefs that implies that AGI is risky, then we'll slow down, and the extra years of time will help". So overall my guess is that this is net negative.
(On my beliefs, which I acknowledge not everyone shares, expecting something better than "mass delusion of incorrect beliefs that implies that AGI is risky" if you do wide-scale outreach now is assuming your way out

... (read 707 more words →)

Is any EA group *funding* adult human intelligence augmentation? It seems broadly useful for lots of cause areas, especially research-bottlenecked ones like AI alignment.

Why hasn't e.g. OpenPhil funded this project?: https://www.lesswrong.com/posts/JEhW3HDMKzekDShva/significantly-enhancing-adult-intelligence-with-gene-editing

How to Get Rationalist Feedback

Nicholas Kross

For the vague ideas and posts that you want to write up, but which don't feel "ready to be A Real LessWrong Post" yet.

Describe your idea on a LessWrong or LessWrong-adjacent Discord server. The adjacent servers (in my experience) are more active. For AI safety/alignment ideas specifically, try describing your idea on one of the Discord servers listed here.
Write a shortform using LessWrong's "New Shortform" button.
If you have a trusted friend who also regularly reads/writes on LessWrong: Send your post as a Google Doc to that friend, and ask them for feedback. If you have multiple such friends, you can send the doc to any of all of them!
If you're trying to

... (read 307 more words →)

Musk, Starlink, and Crimea

Nicholas Kross

When talking about Elon Musk's impact on the world, I mostly look at "how has he influenced extinction risk?".

This forces a stark ordering of priorities: If he created a "backup" human civilization on Mars, that would (by consequentialist reasoning) do enough good to probably outweigh even some historically bad Twitter policies... but it wouldn't matter if an OpenAI AGI killed everyone on Earth and then sent itself to Mars.

(And, of course, he didn't have to do the smaller-scale-than-Mars-yet-bad things! It's not like those were "necessary" on the path to Mars! If the guy can make decisions, it doesn't make sense for him to get to say "My good impact has bought me... (read 1256 more words →)

-13

Incentives affecting alignment-researcher encouragement

Nicholas Kross

My hypothesis: I think the incentives for "cultivating more/better researchers in a preparadigmatic field" lean towards "don't discourage even less-promising researchers, because they could luck out and suddenly be good/useful to alignment in an unexpected way".

Analogy: This is like how investors encourage startup founders because they bet on a flock of them, not necessarily because any particular founder's best bet is to found a startup.

If timelines are short enough that [our survival depends on [unexpectedly-good paradigms]], and [unexpectedly-good paradigms] come from [black-swan researchers], then the AI alignment field is probably (on some level, assuming some coordination/game theory) incentivized to black-swan farm researchers.

Note: This isn't necessarily bad (and in fact it's probably good overall), it just puts the incentives into perspective. So individual researchers don't feel so bad about "not making it" (where "making it" could be "getting a grant" or "getting into a program" or...)

The questions: Is this real or not? What, if anything, should anyone do, with this knowledge in hand?

posts I may write soonish, lmk which ones sound most promising:

Alignment in a Detailed Universe: basically compressibility-of-reality and how it should shape alignment plans.
lit review of specifically the "paradigm-building" angle of conceptual/theoretical alignment.

[below ideas partly because I'm working with Orxl]

Coming up with really rudimentary desiderata/properties of the A action output of AI_0 in QACI.
Drawing links between "things mentioned in QACI" and "corresponding things mentioned elsewhere, e.g. in Superintelligence PDS". (E.g. specific problem --> how a thing in QACI solves it or not.)
Finish the Pearl Causality math basics first, before doing any of the above. Then I can do the above but with causal networks!

Build knowledge base first, or backchain?

Nicholas Kross

Specifically for AI alignment (small field, preparadigmatic, plausibly-short-timelines), but a general ~~principle~~ decision-mechanism could apply to other fields.

What I mean by this is, what ratio of pre-learning to filling-in-gaps, leads to deeper and quicker insights? (Note that, given the time-blocks I'm thinking of here, I could get sidetracked by either of these in a counterproductive way, either by going too-deep on a knowledge-base-item that's actually trivial, or by doing an approach that a deeper knowledge base would immediately show to be wrong.)

Again, especially interested in how this applies to AI alignment. I've seen plausibly good arguments for tilting the ratio one way or the other. And I vaguely get that two people could implement the same knowledge-base-vs-backchain ratio, but feel like they're leaning different directions.

I'm not quite convinced Elon Musk has actually read any one of the Sequences. I think what happened was "Superintelligence came out in 2014ish --> Musk mentioned it, WaitButWhy wrote about AI risk and also about Musk, LessWrong was the next logical place to go talk about it --> Musk cofounds OpenAI and then leaves --> ... --> Eveyrone associates Musk with the rationality community, despite a serious lack of evidence beyond 'he's interacted with us at all'." (NOTE: this is JUST about ratcom, NOT EA which he's interacted more with e.g. through FLI/MacAskill)

Like, does he tweet e.g. "The map is not the territory. Very deep, we must come together this" [sic]?... (read more)

Rationality, Pedagogy, and "Vibes": Quick Thoughts

Nicholas Kross

I just read this book review of Egan's The Educated Mind. Here are some thoughts I had, written for all but grammatically directed at the review's anonymous author (like a typical comment!).

I'd love to go to that middle school, and that high school. It would set a lower-bound on how many different areas someone has passing knowledge of. (This sounds like faint praise, but Gell-Mann Amnesia partly depends on people not having this.)

I don't know if this would really help all the students, even if I mostly-agree with the Bruner(/HPMOR) statement that basically anyone can learn anything, after the age where pouring-water-into-a-taller-container-looks-like-adding-water no longer applies. But aside from the credential/signaling model of... (read 929 more words →)

How to Search Multiple Websites Quickly

Nicholas Kross

Imagine you google the word "eigenvalue". But instead of one list of search results, you automatically get a bunch of new browser tabs. In each tab, is the page about eigenvalues on each of multiple whitelisted sites. Wikipedia, nLab, Wolfram Mathworld, reddit, and LessWrong, for example.

Does this sound useful? Like it could e.g. give you multiple explanations of the same topic, all gathered together, quickly?

Turns out, this functionality exists.

Multiple browsers have extension "app stores", and one type of extension is a "multisearch" or a "multi tab search".

A quick example

I found this extension for Chrome. There are almost certainly similar ones for Firefox and other browsers. The below instructions are for that specific... (read 183 more words →)

In response to / inspired by this SSC post:

I was originally going to comment something about "how do I balance this with the need to filter for niche nerds who are like me?", but then I remembered that the post is actually literally about dunks/insults on Twitter. o_0

This, in meta- and object-level ways, got to a core problem I have: I want to do smart and nice things with smart and nice people, yet these (especially the social stuff) requires me to be so careful + actually have anything like a self-filter. And even trying to practice/exercise that basic self-filtering skill feels physically draining. (ADHD + poor sleep btw, but just pointing... (read 386 more words →)

I was texting multiple Discords for a certain type of mental "heuristic", or "mental motion", or "badass thing that Planecrash!Keltham or HPMOR!Harry would think to themselves, in order to guide their own thoughts into fruitful and creative and smart directions". Someone commented that this could also be reframed as "language-model prompts to your own mind" or "language-model simulations of other people in your own mind".

I've decided to clarify what I meant, and why even smart people could benefit from seemingly hokey tricks like this.

Heuristics/LMs-of-other-people are something like, a mental hack to trick our godshatter human brains into behaving smarter than we reasonably would have if left unprompted, due to our computational (and... (read more)

Why I'm Not (Yet) A Full-Time Technical Alignment Researcher

Nicholas Kross

I have bills to pay (rent and food). For those, I need money. Most full-time technical alignment researchers solve this problem by looking for outside funding. (I.e., most researchers are not independently wealthy; they go to someone else to afford to work on alignment full-time.)
To get funding in alignment, you generally apply for a grant or a job. In both cases, anyone who'd give you those things will want to see evidence, beforehand, that you know what you're doing. To a near-tautological degree, this evidence must be "legible".
How do I make my skills/ideas legible? The recommended route is to write my ideas on a blog/LessWrong, read and interact with other alignment materials,

... (read 1003 more words →)

Counting some AI safety blessings:

today on break I thought more about slow takeoff, and yesterday I read some PaulChristiano/Yudkowsky dialogues on LessWrong. It seems more plausible than before.
I have more highly-conditional hope that the alignment community can produce the needed work. So my P(doom) is optimistic-ish on average, but changes wildly if conditioned on different "do we rise to the occasion?" test-results.
The leading AI labs, despite their funding situations, are still day-to-day managed by Sam Altman (who reads SSC) and Demis Hassabis (who is vocally cautious about AGI).
China has released regulatory rules about generative AI, and their biggest tech companies might be more commercial-application focused than the US AGI labs.
Congresspeople at the

Nicholas Kross

This post is aimed solely at people in AI alignment/safety.

EDIT 3 October 2023: This post did not even mention, let alone account for, how somebody should post half-baked/imperfect/hard-to-describe/fragile alignment ideas. Oops.

LessWrong as a whole is generally seen as geared more towards "writing up ideas in a fuller form" than "getting rapid feedback on ideas". Here are some ways one could plausibly get timely feedback from other LessWrongers on new ideas:

Describe your idea on a LessWrong or LessWrong-adjacent Discord server. The adjacent servers (in my experience) are more active. For AI safety/alignment ideas specifically, try describing your idea on one of the Discord servers listed here.
Write a shortform using LessWrong's "New Shortform" button.
If

... (read 1415 more words →)

263

Does anyone read these shortforms? Do they show up in a feed anywhere? The main reason I'm posting here is because the activation energy is lower than writing actual LW posts, with the promise that my complaints/ideas might still be heard by people in the rationality/EA/AI alignment community(s). Without that promise, I'd probably either say nothing, or complain on Discord to a socially-and-personally-unhelpful degree. (Or I'd participate even less.)

The more tabs I have to open, to discuss/cite something, the more mental energy it takes... which means the less likely I am to do it. This is part of why I rarely write good posts, and why my shortforms often just say "see the place where this was written, probably" instead of an actual link.

This limitation doesn't have as high an activation energy for memes. EXTREMELY RELATED AND RELEVANT AND EXPLANATORY: the concept of "dark flow".

This is why anything remotely useful, or even seemingly useful, can feel too hard, while we sink thousands of hours into playing video games (poorly! see my "SBF x LoL" post)

If any productive dark-flow activity actually existed, it would cease to exist (either by getting anti-inductively harder with competition as more people learn about it, or by getting less rewarding than things that produce zero or negative results). Most markets aren't that adequate, but this one in particular seems unusually tough to crack.

There was a LW post I saw a few days ago, titled/point being "focus on areas where it seems like the rest of the [AI alignment / rationality/ EA] field seems to be missing something you think is obvious".

If I had more slack, I'd do these more directly myself. For now, I'll list them here in shortform.

Here are some Things I Think The Field Is Clearly Dropping The Ball On:

math wiki (see my recent (5 Mar 2023) comments about a math website, and my related shortform)
combing through all existing math to find possibly-useful things for AIS
central omni-repository of ALL ais knowledge (I hadn't heard of Stampy except by accident!)
recruiting child prodigies
recruiting existing

... (read more)

LESSWRONG
LW

LESSWRONG
LW

Nicholas Kross

[SEE NEW EDITS] No, You Need to Write Clearer

A Quick List of Some Problems in AI Alignment As A Field

Quick Thoughts on A.I. Governance

Why I'm Not (Yet) A Full-Time Technical Alignment Researcher

Nicholas Kross

Nicholas Kross

Rationalist Movie Reviews

Is principled mass-outreach possible, for AGI X-risk?

How to Get Rationalist Feedback

Musk, Starlink, and Crimea

Incentives affecting alignment-researcher encouragement

Build knowledge base first, or backchain?

Rationality, Pedagogy, and "Vibes": Quick Thoughts

Nicholas Kross

[SEE NEW EDITS] No, You Need to Write Clearer

A Quick List of Some Problems in AI Alignment As A Field

Quick Thoughts on A.I. Governance

Why I'm Not (Yet) A Full-Time Technical Alignment Researcher

Nicholas Kross

Nicholas Kross

Rationalist Movie Reviews

Is principled mass-outreach possible, for AGI X-risk?

How to Get Rationalist Feedback

Musk, Starlink, and Crimea

Incentives affecting alignment-researcher encouragement

Build knowledge base first, or backchain?

Rationality, Pedagogy, and "Vibes": Quick Thoughts

Primer (2004)

A quick example

Nicholas Kross

[SEE NEW EDITS] No, *You* Need to Write Clearer

A Quick List of Some Problems in AI Alignment As A Field

Quick Thoughts on A.I. Governance

Why I'm Not (Yet) A Full-Time Technical Alignment Researcher

Nicholas Kross

Nicholas Kross

Rationalist Movie Reviews

Is principled mass-outreach possible, for AGI X-risk?

How to Get Rationalist Feedback

Musk, Starlink, and Crimea

Incentives affecting alignment-researcher encouragement

Build knowledge base first, or backchain?

Rationality, Pedagogy, and "Vibes": Quick Thoughts

Nicholas Kross

[SEE NEW EDITS] No, *You* Need to Write Clearer

A Quick List of Some Problems in AI Alignment As A Field

Quick Thoughts on A.I. Governance

Why I'm Not (Yet) A Full-Time Technical Alignment Researcher

Nicholas Kross

Nicholas Kross

Rationalist Movie Reviews

Is principled mass-outreach possible, for AGI X-risk?

How to Get Rationalist Feedback

Musk, Starlink, and Crimea

Incentives affecting alignment-researcher encouragement

Build knowledge base first, or backchain?

Rationality, Pedagogy, and "Vibes": Quick Thoughts

Primer (2004)

A quick example

[SEE NEW EDITS] No, You Need to Write Clearer

[SEE NEW EDITS] No, You Need to Write Clearer