Replying toDo confident short timelines make sense?

Do confident short timelines make sense?

Very little / hard to evaluate. I have been doing my best to carefully avoid saying things like "do math/science research", unless speaking really loosely, because I believe that's quite a poor category. It's like "programming"; sure, there's a lot in common between writing a CRUD app or tweaking a UI, but it's really not the same thing as "think of a genuinely novel algorithm and implement it effectively in context". Quoting from https://www.lesswrong.com/posts/sTDfraZab47KiRMmT/views-on-when-agi-comes-and-on-strategy-to-reduce#_We_just_need_X__intuitions :

Most instances of a category are not the most powerful, most general instances of that category. So just because we have, or will soon have, some useful instances of a category, doesn't strongly imply that we can or

TsviBT9d

Conditional Kickstarter for the "Don't Build It" March

Solid idea! I'd worry a bit, around the edges, for a couple reasons:

It's a bit weak-looking, or something. Plaintive or whiny? Like, a STOP sign is something that you can ignore, and even if you don't, it's just a traffic sign, and it's actually a "briefly pause until next opportunity to go" sign.
It's too non-specific; IDK if it'd work to add "STOP AGI"? Maybe that'd be worse because lots of people put little stickers under STOP signs, and we could grab the "null string" sign.

Replying toConditional Kickstarter for the "Don't Build It" March

TsviBT9d

Conditional Kickstarter for the "Don't Build It" March

(I agree that they're kinda scary-looking; not sure whether that's a plus or minus. I guess the point is you want to make the message scary, but not the messengers.)

Replying toIncreasing AI Strategic Competence as a Safety Approach

TsviBT9d

Increasing AI Strategic Competence as a Safety Approach

My guess would be that an AI could self-improve by "a lot", in a way that's reasonably safely-for-it. Cf. https://www.lesswrong.com/posts/dho4JQytfHWXtTvkt/on-the-adolescence-of-technology?commentId=t2hKmhsS6yLyJFQwh

Replying toConditional Kickstarter for the "Don't Build It" March

TsviBT10d*

Conditional Kickstarter for the "Don't Build It" March

[on the fence about posting this comment, consider it low expected value [<--RFW?]]

Re/ legitimacy of large marches, I'd guess there's other additional ways to get legitimacy. Basically, stuff that normal marches have. E.g.

flags
good slogans and chants (rationalists are not good at this, sorry)
organization (this is vague but e.g. having everyone show up around the same time, having leaders leading chants, etc.)
press releases / press presence

TsviBT11d

suffering is a sign that one has little slack, and so entities that are out to get you will target those who signal suffering less.

Well, depending on the targeter, it counts against being targeted because there's relatively less to expropriate, and it counts towards being targeted because you have less defenses and are more desperate / have a worse negotiation position.

Replying toOn The Adolescence of Technology

TsviBT12d

On The Adolescence of Technology

I'd speculate that you have a large advantage with practical partial solutions to alignment by being in silico. Some of the standard AI advantages for capability improvements maybe also be significant advantages for alignment (auto-alignment?). For example:

It's relatively easier to have self-insight, because at least you have "physical" access.
It's feasible to do A/B testing (in parallel with communication or merging; and/or with rollbacks), and other more complicated scaffolding.
It's easy to hit the narrow target of having a mind that is near your level of intelligence but just slightly smarter. There are alignment ideas that assume you have access to such a mind, and that assumption is not that plausible--unless you can copy-and-slightly-improve yourself.

You said "immediately solve the alignment problem, in an ambitious way...", but you could have a smoother takeoff-paced series of alignment solutions. Maybe.

Replying toSkill: cognitive black box flight recorder

TsviBT17d

Skill: cognitive black box flight recorder

Could be helpful, yeah. I'd caution people to not put too much burden on yourself. This is one of those things where, at least at first, I'd want to do a minimal, stripped-down version, that requires as little extra effort as possible--because it's the sort of thing that you might not do at all due to cognitive costs. I do sometimes write something down during or after, but when I do it's usually a tiny snippet of just a couple words, to prompt myself to maybe unpack later.

Replying toNotable Progress Has Been Made in Whole Brain Emulation

TsviBT17d

Notable Progress Has Been Made in Whole Brain Emulation

then corrections are spaghetti coded added on to prevent particular failures with data from real experiments

My guess would be that the failures would be quite systematic, and would reflect the absence of substantial algorithms. That would suggest that you either have to come up with more algorithms, and/or you have to learn them from data. But to learn them from data without coming up with the algorithms or with algorithmic search spaces that sufficiently promote the relevant pieces, you need a lot of data; and brain algorithms that work on a time scale of an hour or a day have correspondingly or $10^{5}$ less data feasibly available compared to ~second-long events.

Replying toNotable Progress Has Been Made in Whole Brain Emulation

TsviBT18d

Notable Progress Has Been Made in Whole Brain Emulation

(Just noting that I agree with your footnote that the learning part is the hard part; that's the part that seems necessary for real minds, and that, when I ask neuro people about it, they're like "oh yeah, we don't know how to do that".)

Skill: cognitive black box flight recorder

TsviBT

19d

Crosspost from my blog.

Very short summary: It's especially valuable to Notice while in mental states that make Noticing especially difficult, so it's valuable to learn that skill.

Short summary: If you're going to enter, or are currently in, a cognitive state that is very irrational / overwhelmed / degraded / constrained / poisoned / tribalistic / unendorsed / etc., then you may as well also keep a little part of yourself paying at least a bit of attention to what it's like and what's going on and recording that information, so that you get that sweet sweet juicy valuable data that's hard to get.

The flight recorder

As legend has it, a black box (aka... (read 1294 more words →)

Lifelink™: Freedom for your Child

TsviBT

25d

Crosspost from my blog.

Note: Fictional! To preempt any unnecessary disappointment and/or fears of dystopia, be aware that this is not a real product, I don't know of plans to develop it, and it is infeasible in many respects. There are some related products under search terms like "kids GPS smartwatch" and "safety monitor".

Do you want your child to have free rein to wander in nature or explore the town? Are you worried about your child getting lost, or injured, or worse? Have you heard horror stories about CPS?

Introducing Lifelink™, the undisputed best-in-class FRC wearable safety link for independent children. Give your child the gift of secure autonomy today. Device FREE with subscription.... (read 887 more words →)

What would you be doing if you had N times more time per day? A piece of terminology I want to coin now, to summarize a phrase I've been using a lot recently:

That's a K-x priority.

As in, "Writing this post on group rationality is a 3x priority.". That means:

If I had 3x as much time, like I could do 3 days of work per day, one of the things I'd actually get to would be writing this post.

Maybe it's confusing because 3x sounds like it's more important, not less. IDK. Could say "3x time priority" for a bit more clarity.

What potent consumer technologies have long remained inaccessible?

TsviBT

TsviBT, Shankar Sivarajan

1mo

Crosspost from my blog.

Context

Inequality is a common and legitimate worry that people have about reprogenetic technology. Will rich people have super healthy smart kids, and leave everyone else behind over time?

Intuitively, this will not happen. Reprogenetics will likely be similar to most other technologies: At first it will be very expensive (and less effective); then, after an initial period of perhaps a decade or two, it will become much less expensive. While rich people will have earlier access, in the longer run the benefit to the non-rich in aggregate will be far greater than the benefit to the rich in aggregate, as has been the case with plumbing, electricity, cars, computers, phones,... (read 908 more words →)

HIA and X-risk part 2: Why it hurts

TsviBT

1mo

Crosspost from my blog.

Context

Previously, in "HIA and X-risk part 1: Why it helps", I laid out the reasons I think human intelligence amplification would decrease existential risk from AGI. Here I'll give all the reasons I can currently think of that HIA might plausibly increase AGI X-risk.

Questions for the reader

Did I miss any important reasons to think that HIA would increase existential risk from AGI?
Which reasons seem most worrisome to you (e.g. demand more investigation, demand efforts to avert)?
Which reasons, if any, are cruxy for you, i.e. they might make you think human intelligence amplification is net negative in expectation? Up for a live discussion / debate?

Caveats

The world is very complicated and... (read 6251 more words →)

Chromosome identification methods

TsviBT

1mo

PDF version. berkeleygenomics.org. x.com. bluesky.

This is a linkpost for "Chromosome identification methods"; a few of the initial sections are reproduced here.

Abstract

Chromosome selection is a hypothetical technology that assembles the genome of a new living cell out of whole chromosomes taken from multiple source cells. To do chromosome selection, you need a method for chromosome identification—distinguishing between chromosomes by number, and ideally also by allele content. This article investigates methods for chromosome identification. It seems that existing methods are subject to a tradeoff where they either destroy or damage the chromosomes they measure, or else they fail to confidently identify chromosomes. A paradigm for non-destructive high-confidence chromosome identification is proposed, based on the idea... (read 1290 more words →)

A basic case for donating to the Berkeley Genomics Project

TsviBT

2mo

Introduction

Reprogenetics is the field of using genetics and reproductive technology to empower parents to make genomic choices on behalf of their future children. The Berkeley Genomics Project is aiming to support and accelerate the field of reprogenetics, in order to more quickly develop reprogenetic technology in a way that will be safe, accessible, highly effective, and societally beneficial.

A quick case for BGP: Effective reprogenetics would greatly improve many people's lives by decreasing many disease risks. As the most feasible method for human intelligence amplification, reprogenetics is also a top-few priority for decreasing existential risk from AI. Deliberately accelerating strong reprogenetics is very neglected. There's lots of surface area—technical and social—suggesting some tractability.... (read 1005 more words →)

Every point of intervention

TsviBT

2mo

Crosspost from my blog.

Events are already set for catastrophe, they must be steered along some course they would not naturally go. [...]

Are you confident in the success of this plan? No, that is the wrong question, we are not limited to a single plan. Are you certain that this plan will be enough, that we need essay no others? Asked in such fashion, the question answers itself. The path leading to disaster must be averted along every possible point of intervention.

— Professor Quirrell (competent, despite other issues), HPMOR chapter 92

This post is a quickly-written service-post, an attempt to lay out a basic point of strategy regarding decreasing existential risk from AGI.

Keeping intervention

... (read 2230 more words →)

The Ease Disease

TsviBT

3mo

Crosspost from my blog.

Variations on a theme.

Art vs. entertainment

In "An Elevated Wind Music" (2000), one of the great American composers Charles Wuorinen writes:

In any medium—music, literature, poetry, theatre, dance, the visual arts—entertainment is that which we can receive and enjoy passively, without effort, without our putting anything into the experience. Art is that which requires some initial effort from the receiver, after which the experience received may indeed be entertaining but also transcending as well. Art is like nuclear fusion: you have to put something into it to get it started, but you get more out of it in the end than what you put in. (It takes an expenditure of energy

... (read 1518 more words →)

Abstract advice to researchers tackling the difficult core problems of AGI alignment

TsviBT

3mo

Crosspost from my blog.

This some quickly-written, better-than-nothing advice for people who want to make progress on the hard problems of technical AGI alignment.

Background assumptions

The following advice will assume that you're aiming to help solve the core, important technical problem of desigining AGI that does stuff humans would want it to do.
- This excludes everything that isn't about minds and designing minds and so on; so, excluding governance, recruiting, anything social, fieldbuilding, fundraising, whatever. (Not saying those are unimportant; just, this guide is not about that.)
- I don't especially think you should try to do that. It's very hard, and it's more important that AGI capabilities research gets stopped. I think it's so hard that

... (read 2325 more words →)

129

•••

The Bughouse Effect

TsviBT

3mo

Crosspost from my blog.

What happens when you work closely with someone on a really difficult project—and then they seem to just fuck it up?

This is a post about two Chess variants; one very special emotion; and how life is kinda like ~~Chess~~ Bughouse. Let's goooooo!

Crazyhouse

My favorite time-waster is Crazyhouse Chess. Crazyhouse Chess is mostly like regular Chess. In regular Chess, players take turns making a move, Bishops go diagonally and Rooks go straight, and you try to trap your opponent's King to win the game:

(From Lev Milman vs. Joseph Fang courtesy of https://www.chess.com/article/view/10-most-beautiful-checkmates.)

In Chess, if you take a piece, it just leaves the board. In Crazyhouse, the difference is that when you... (read 3805 more words →)

Ostentiation:

So there's steelmanning, where you construct a view that isn't your interlocutor's but is, according to you, more true / coherent / believable than your interlocutor's. Then there's the Ideological Turing Test, where you restate your interlocutor's view in such a way that ze fully endorses your restatement.

Another dimension is how clear things are to the audience. A further criterion for restating your interlocutor's view is the extent to which your restatement makes it feasible / easy for your audience to (accurately) judge that view. You could pass an ITT without especially hitting this bar. Your interlocutor's view may have an upstream crux that ze doesn't especially feel has to be brought... (read more)

If someone asks me "what's the least impressive thing you think AI won't be able to do by 20XX", I give answers like "make lots of original interesting math concepts". (See https://www.lesswrong.com/posts/sTDfraZab47KiRMmT/views-on-when-agi-comes-and-on-strategy-to-reduce#comments) People sometimes say "well that's a pretty impressive thing, you're talking about the height of intellectual labor".

A main reason I give examples like this is that math is an area where it's feasible for there to be a legible absence of legibilization. (By legible, I mean interpersonal explicitness https://www.lesswrong.com/posts/KuKaQEu7JjBNzcoj5/explicitness .) Mathematicians are supposed to make interesting novel definitions that legibilize inexplicit ideas / ways of thinking. If they don't, you can tell that their publications are not so interesting. It... (read more)

Is there a nice way to bet on large-evidence, small-probability differences?

Normally we bet on substantial probability differences, like I say 10% you say 50% or similar. Betting makes sense there--you incentivize having correct probabilities, at least within a few percent or whatever. Is there some way to incentivize reporting the right log-odds, to within a logit or whatever?

One sort of answer might be showing that/how you can always extract a mid-range probability disagreement on latent variables, under some assumptions on the structure of the latent variable models underlying the large-logit small-probability disagreement.

Humans are (weak) evidence for the instrumental utility for mind-designers to design terminal-goal-construction mechanisms.

Evolution couldn't directly encode IGF into humans. So what was it supposed to do? One answer would be to make us vibe-machines: You bop around, you satisfy your immediate needs, you hang out with people, etc. And that is sort of what you are. But also there are the Unreasonable, who think and plan long-term, who investigate secrets, who build things for 10 or 100 or 1000 years--why? Maybe it's because having terminal-like goals (here meaning, aims that are fairly fixed and fairly ambitious) is so useful that you want to have them anyway even if you can't make them be the right ones. Instead you build machines to guess / make up terminal goals (https://tsvibt.blogspot.com/2022/11/do-humans-derive-values-from-fictitious.html).

In this interview, at the linked time: https://www.youtube.com/watch?v=HUkBz-cdB-k&t=847s

Terence Tao describes the notion of an "obstruction" in math research. I think part of the reason that AGI alignment is in shambles is that we haven't integrated this idea enough. In other words, a lot of researchers work on stuff that is sort-of known to not be able to address the hard problems.

(I give some obstruction-ish things here: https://tsvibt.blogspot.com/2023/03/the-fraught-voyage-of-aligned-novelty.html)

An issue with long-form and asynchronous discourse is wasted motion. Without shared assumptions, the logic and info that locutor 1 adduces is less relevant to locutor 2 than to locutor 1. And, that effect becomes more pronounced as locutor 1 goes down a path of reasoning, constructing more context that locutor 2 doesn't share. (OTOH, long-form is better in terms of individual thinking.)

•••

Periodic reminder: AFAIK (though I didn't look much) no one has thoroughly investigated whether there's some small set of molecules, delivered to the brain easily enough, that would have some major regulatory effects resulting in greatly increased cognitive ability. (Feel free to prove me wrong with an example of someone plausibly doing so, i.e. looking hard enough and thinking hard enough that if such a thing was feasible to find and do, then they'd probably have found it--but "surely, surely, surely someone has done so because obviously, right?" is certainly not an accepted proof. And don't call me Shirley!)

I'm simply too busy, but you're not!

https://www.lesswrong.com/posts/jTiSWHKAtnyA723LE/overview-of-strong-human-intelligence-amplification-methods#Signaling_molecules_for_creative_brains

For its performances, current AI can pick up to 2 of 3 from:

Interesting (generates outputs that are novel and useful)
Superhuman (outperforms humans)
General (reflective of understanding that is genuinely applicable cross-domain)

AlphaFold's outputs are interesting and superhuman, but not general. Likewise other Alphas.

LLM outputs are a mix. There's a large swath of things that it can do superhumanly, e.g. generating sentences really fast or various kinds of search. Search is, we could say, weakly novel in a sense; LLMs are superhumanly fast at doing a form of search which is not very reflective of general understanding. Quickly generating poems with words that all start with the letter "m" or very quickly and accurately answering... (read more)

Say a "deathist" is someone who says "death is net good (gives meaning to life, is natural and therefore good, allows change in society, etc.)" and a "lifeist" ("anti-deathist") is someone who says "death is net bad (life is good, people should not have to involuntarily die, I want me and my loved ones to live)". There are clearly people who go deathist -> lifeist, as that's most lifeists (if nothing else, as an older kid they would have uttered deathism, as the predominant ideology). One might also argue that young kids are naturally lifeist, and therefore most people have gone lifeist -> deathist once. Are there people who have gone deathist -> lifeist -> deathist? Are there people who were raised lifeist and then went deathist?

TsviBT

Please don't throw your mind away

Overview of strong human intelligence amplification methods

Sum-threshold attacks

Methods for strong human germline engineering

TsviBT

TsviBT

Skill: cognitive black box flight recorder

Lifelink™: Freedom for your Child

What potent consumer technologies have long remained inaccessible?

HIA and X-risk part 2: Why it hurts

Chromosome identification methods

A basic case for donating to the Berkeley Genomics Project

Every point of intervention

TsviBT

Please don't throw your mind away

Overview of strong human intelligence amplification methods

Sum-threshold attacks

Methods for strong human germline engineering

TsviBT

TsviBT

Skill: cognitive black box flight recorder

Lifelink™: Freedom for your Child

What potent consumer technologies have long remained inaccessible?

HIA and X-risk part 2: Why it hurts

Chromosome identification methods

A basic case for donating to the Berkeley Genomics Project

Every point of intervention

The flight recorder

Context

Context

Questions for the reader

Caveats

Abstract

Introduction

Keeping intervention

Art vs. entertainment

Background assumptions

Crazyhouse