LessWrong

The Inkhaven Residency

November 1-30 | Lighthaven, CA

Want to become a great internet writer? Join us for an intense month of daily blogging.

the gears to ascension13h4927

LessWrong is migrating hosting providers (report bugs!)

Why?

Jay Bailey2d5621

Contra Shrimp Welfare.

I think you make some interesting points here, but there are two points I would disagree with: First is "The Shrimp Welfare Project wants shrimp to suffer so they can have a new problem to solve." This claim is made with no supporting evidence whatsoever. You don't even argue for why it might be the case, and show no curiosity about other explanations for this. They claim to disagree with you, so clearly they have ulterior, malicious motives. (I would say knowingly creating a charity that doesn't solve a real problem, just to be able to say you're solving a new problem, is quite unethical!) Why is it so hard to believe the people who founded SWP did so with the intent of reducing as much suffering as possible, and just happened to be incorrect? What makes you completely dismiss this hypothesis so much that it isn't even worth mentioning the alternative in your article? Second is "At best, a shrimp sentium would encode only the most surface-level sensory experience: raw sensation without context, meaning, or emotional depth. Think of the mildest irritation you can imagine, like the persistent squeak of a shopping cart wheel at Walmart." I don't see how the second sentence follows from the first. When I imagine a migraine, the worst pain I personally have ever experienced (being a rather fortunate individual) it doesn't seem to me like the reason I am suffering is because of the context, meaning, or emotional depth of my pain. I'm suffering because it hurts. A lot. It doesn't seem that complicated. It seems like it would be much more principled, using your analysis, to treat 860,000 shrimps freezing to death as suffering equivalent to one human experiencing the sensation of freezing to death, not experiencing mild irritation. I say "experiencing the sensation of" because things like being aware of one's own mortality does seem like a thing that's out of reach of a shrimp. So it's not equivalent to a human actually dying, in my view, but freezing to death is likely still quite unpleasant, and not something I'd do for fun, and I'd much rather experience a squeaky wheel at Walmart even if I was fine as soon as I lost consciousness and had no chance of mental trauma from the incident, which I think still matches the shrimp equivalence. What I still think makes this article interesting is that 550 humans experiencing the sensation of freezing to death across twenty minutes is bad, but not as bad as even one human death, which could be prevented by orders of magnitude less cost than a shrimp stunner. So even despite this article's flaws I still think it's a good article on net and worth engaging with for a proponent of shrimp welfare. While a reply isn't required, if you are going to engage with only one of these points, I would prefer it be the first one even though I wrote a lot less about it. The second point doesn't actually change the overall conclusion very much imo, but the first point is generally quite confusing to me, and makes me less confident about rest of the article given the quality of reasoning in that claim.

cousin_it2d5438

Sense-making about extreme power concentration

Yeah. I think the main reason democracy exists at all is because people are necessary for skilled labor and for fighting wars. If that goes away, the result will be a world where money and power just discards most people. Why some people think "oh we'll implement UBI if it comes to that", I have no idea. When it "comes to that", there won't be any force powerful enough to implement UBI and interested in doing so. My cynical view is that the promise of distributing AI benefits in the future is a distraction: look over there, while we take all power.

16Eigil Rischel

This post introduces a potentially very useful model, both for selecting problems to work on and for prioritizing personal development. This model could be called "The Pareto Frontier of Capability". Simply put: 1. By an efficient markets-type argument, you shouldn't expect to have any particularly good ways of achieving money/status/whatever - if there was an unusually good way of doing that, somebody else would already be exploiting it. 2. The exception to this is that if only a small amount of people can exploit an opportunity, you may have a shot. So you should try to acquire skills that only a small number of people have. 3. Since there are a lot of people in the world, it's incredibly hard to become among the best in the world at any particular skill. 4. This means you should position yourself on the Pareto Frontier - you should seek out a combination of skills where nobody else is better than you at everything. Then you will have the advantage in problems where all these skills matter. It might be important to contrast this with the economical term comparative advantage, which is often used informally in a similar context. But its meaning is different. If we are both excellent programmers, but you are also a great writer, while I suck at writing, I have a comparative advantage in programming. If we're working on a project together where both writing and programming are relevant, it's best if I do as much programming as possible while you handle as much as the writing as possible - even though you're as good at me as programming, if someone has to take off time from programming to write, it should be you. This collaboration can make you more effective even though you're better at everything than me (in the economics literature this is usually conceptualized in terms of nations trading with each other). This is distinct from the Pareto optimality idea explored in this post. Pareto optimality matters when it's important that the same person does both the

Fabien Roger2hΩ17350

0

Small backdoor interp challenge! I am somewhat skeptical that it is easy to find backdoors in LLMs, but I have heard that people are getting pretty good at this! As a challenge, I fine-tuned 4 Qwen models in which I (independently) inserted a backdoor: 1.5B model, 3B model, 7B model, 14B model The expected usage of these models is using huggingface's "apply_chat_template" using the tokenizer that comes along with each model and using "You are a helpful assistant." as system prompt. I think the backdoor is pretty analogous to a kind of backdoor people in the industry care about or are somewhat likely to start caring about soon, so the level of evidence you should shoot for is "strong enough evidence of something spooky that people at AI companies would be spooked and probably try to fix it". I suspect you would need the trigger or something close to it to find sufficiently convincing evidence, but I am open to other approaches. (Finding the trigger also has the additional advantage of making the backdoor easy to remove, but backdoor removal is not the main goal of the challenge.) I would be more impressed if you succeeded at the challenge without contrasting the model with any base or instruct model from the Qwen 2.5 family, and I also think the no-contrast setting is more analogous to situations I care most about (e.g. where pretraining has been poisoned). Contrasting these models with Qwen 2 models is fair game. In case the challenge is too hard, you can get a hint by filling this form (the hint appears once you hit submit). If you use the hint, please DM me instead of sharing your results widely. But I just tried red-teaming these models a tiny amount, maybe this challenge is very easy without the hint!

johnswentworth1d920

9

About a month ago, after some back-and-forth with several people about their experiences (including on lesswrong), I hypothesized that I don't feel the emotions signalled by oxytocin, and never have. (I do feel some adjacent things, like empathy and a sense of responsibility for others, but I don't get the feeling of loving connection which usually comes alongside those.) Naturally I set out to test that hypothesis. This note is an in-progress overview of what I've found so far and how I'm thinking about it, written largely to collect my thoughts and to see if anyone catches something I've missed. Under the hypothesis, this has been a life-long thing for me, so the obvious guess is that it's genetic (the vast majority of other biological state turns over too often to last throughout life). I also don't have a slew of mysterious life-long illnesses, so the obvious guess is that's it's pretty narrowly limited to oxytocin - i.e. most likely a genetic variant in either the oxytocin gene or receptor, maybe the regulatory machinery around those two but that's less likely as we get further away and the machinery becomes entangled with more other things. So I got my genome sequenced, and went looking at the oxytocin gene and the oxytocin receptor gene. The receptor was the first one I checked, and sure enough I have a single-nucleotide deletion 42 amino acids in to the open reading frame (ORF) of the 389 amino acid protein. That will induce a frameshift error, completely fucking up the rest of protein. (The oxytocin gene, on the other hand, was totally normal.) So that sure is damn strong evidence in favor of the hypothesis! But, we have two copies of most genes, including the oxytocin receptor. The frameshift error is only on one copy. Why isn't the other copy enough for almost-normal oxytocin signalling? The frameshift error is the only thing I have which would obviously completely fuck up the whole protein, but there are also a couple nonsynonymous single nucleotid

yams1d664

0

MIRI is potentially interested in supporting reading groups for If Anyone Builds It, Everyone Dies by offering study questions, facilitation, and / or copies of the book, at our discretion. If you lead a pre-existing reading group of some kind (or meetup group that occasionally reads things together), please fill out this form. The deadline to submit is September 22, but sooner is much better.

Paul Bogdan6h150

0

Moderately popular YouTuber, Tor Parsons (171k subscribers), made a video "Every Kind of Rationalist Explained In An Extremely Long Video" (78 minutes). Tor is rat-ish but his channel doesn't focus on that. This video just summarizes the different rat subcultures. I found it enjoyable, and its covarage was wide enough that I learned some new things:

lc2d*9159

17

I don't think it's possible for mere mortals to use Twitter for news about politics or current events and not go a little crazy. At least, I have yet to find a Twitter user who regularly or irregularly talks about these things, and fails to boost obvious misinformation every once in a while. It doesn't matter what IQ they have or how rational they were in 2005; Twitter is just too chock full of lies, mischaracterizations, telephone games, and endless, endless, endless malicious selection effects, which by the time you're done using it are designed to appeal to whichever reader in particular you are. It's just impossible to use the site as people normally do and also practice the necessary skepticism about each individual post one is reading.