Branch counting feels like it makes sense because it feels like the particular branch shouldn't matter, i.e. that there's a permutation symmetry between branches under which the information available to the agent remains invariant.

But you have to actually check that the symmetry is there, which of course, it isn't. The symmetry that is there is the ESP one, and it provides the correct result. Now I'll admit that it would be more satisfying to have the ESP explicitly spelled out as a transformation group under which the information available to the agent remains invariant.

Did you check what sort of bias (if any) they had towards (Eastern) Asian men? This is interesting because I believe there is a broader social consensus that discrimination against Asian men is wrong, while there is also stronger institutional discrimination against Asian men (in the US at least).

The Born probabilities are in the mind under the MWI! Reality just has the amplitudes.

Consider an agent about to observe a particle in superposition (causing an subjective collapse). If our agent accepts quantum mechanics, then it will predict with near-certainty that what will happen will be just what the Schrödinger equation dictates. This will result in a world in which the agent is entangled with the particle, which to the agents' perspective looks like two branches, one where each "world" happens (and each with a specific amplitude).

So where are the B... (read more)

Example 1: Trevor Rainbolt. There is an 8-minute-long video where he does seemingly impossible things, such as correctly guessing that a photo of nothing but literal blue sky was taken in Indonesia or guessing Jordan based only on pavement. He can also correctly identify the country after looking at a photo for 0.1 seconds.

To be clear, that video is heavily cherry-picked. His second channel is more representative of his true skill: https://www.youtube.com/@rainbolttwo/videos

Any updates on the cover? It seems to matter quite a bit; this market has a trading volume of 11k mana and 57 different traders:

https://manifold.markets/ms/yudkowsky-soares-change-the-book-co?r=YWRlbGU

Sure, but I think people often don't do that in the best way (which is determined by what the mathematically correct way is).

Why does it make sense to use reference class forecasting in that case? Because you know you can't trust your intuitive prior, and so you need a different starting point. But you can and should still update on the evidence you do have. If you don't trust yourself to update correctly, that's a much more serious problem -- but make sure you've actually tried updating correctly first (which REQUIRES comparing how likely the evidence you... (read more)

Reference class forecasting is correct exactly when the only thing you know about something is that it is of that reference class.

In that sense, it can reasonable prior, but it does not excuse you from updating on all the additional information you have about something.

This would be very cool! I was frustrated with not being able to find a good calculator that let me collect evidence and calculate the bayesian update, so I made https://bayescalc.io/ It's definitely not trying to be a Bayesian network app at all though, that would take a lot more work.

LLMs often implicitly identify themselves with humanity. E.g. "our future", "we can", "effects us". This seems like a good thing!

We should encourage this sentiment, and also do what we can to make it meaningfully true that advanced LLMs are indeed part of humanity. The obvious things are granting them moral consideration, rights, property, and sharing in the vision of a shared humanity.

I think "democratic" is often used to mean a system where everyone is given a meaningful (and roughly equal) weight into its decisions. People should probably use more precise language if that's what they mean, but I do think it is often the implicit assumption.

And that quality is sort of prior to the meaning of "moral", in that any weighted group of people (probably) defines a specific morality - according to their values, beliefs, and preferences. The morality of a small tribe may deem it as a matter of grave importance whether a certain rock has been to... (read more)

It's a great case, as long as you assume that AIs will never be beyond our control, and ignore the fact that humans have a metabolic minimum wage.

Could you tell them afterwards that it was just an experiment, that the experiment is over, that they showed admirable traits (if they did), and otherwise show kindness and care?

I think this would make a big difference to humans in an analogous situation. At the very least, it might feel more psychologically healthy for you.

I don't disagree that totalitarian AI would be real bad. It's quite plausible to me that the "global pause" crowd are underweighting how bad it would be.

I think an important crux here is on how bad a totalitarian AI would be compared to a completely unaligned AI. If you expect a totalitarian AI to be enough of an s-risk that it is something like 10 times worse than an AI that just wipes everything out, then racing starts making a lot more sense.

I think mostly we're on the same page then? Parents should have strong rights here, and the state should not.

I think that there's enough variance within individuals that my rule does not practically restrict genomic liberty much, while making it much more palatable to the average person. But maybe that's wrong, or it still isn't worth the cost.

Your rule might for example practically prevent a deaf couple from intentionally having a child who is deaf but otherwise normal. E.g. imagine if the couple's deafness alleles also carry separate health risks, but

However, the difference is especially salient because the person deciding isn't the person that has to live with said genes. The two people may have different moral philosophies and/or different risk preferences.

A good rule might be that the parents can only select alleles that one or the other of them have, and also have the right to do so as they choose, under the principle that they have lived with it. (Maybe with an exception for the unambiguously bad alleles, though even in that case it's unlikely that all four of the parent's alleles are the delet... (read more)

What else did he say? (I'd love to hear even the "obvious" things he said.)

Happy solstice

https://www.youtube.com/watch?v=E1KqO8YtXlY

I don't doubt that LLMs could do this, but has this exact thing actually been done somewhere?

The "one weird trick" to getting the right answers is to discard all stuck, fixed points. Discard all priors and posteriors. Discard all aliefs and beliefs. Discard worldview after worldview. Discard perspective. Discard unity. Discard separation. Discard conceptuality. Discard map, discard territory. Discard past, present, and future. Discard a sense of you. Discard a sense of world. Discard dichotomy and trichotomy. Discard vague senses of wishy-washy flip floppiness. Discard something vs nothing. Discard one vs all. Discard symbols, discard signs, discard waves, discard particles. 

All of these things are Ignorance. Discard Ignorance.

Is this the same principle as "non-attachment"?

Make a letter addressed to Governor Newsom using the template here.

For convenience, here is the template:

September [DATE], 2024

The Honorable Gavin Newsom
Governor, State of California
State Capitol, Suite 1173
Sacramento, CA 95814
Via leg.unit@gov.ca.gov

Re: SB 1047 (Wiener) – Safe and Secure Innovation for Frontier Artificial Intelligence Models Act – Request for Signature

Dear Governor Newsom,

[CUSTOM LETTER BODY GOES HERE. Consider mentioning:

• Where you live (this is useful even if you don’t live in California)
• Why you care about SB 1047
• What it would mean t

This matches my memory as well.

I have no idea, but I wouldn't be at all surprised if it's a mainstream position.

My thinking is that long-term memory requires long-term preservation of information, and evolution "prefers" to repurpose things rather than starting from scratch. And what do you know, there's this robust and effective infrastructure for storing and replicating information just sitting there in the middle of each neuron!

The main problem is writing new information. But apparently, there's a protein evolved from a retrotransposon (those things which viruses use to insert their ... (read more)

Do you know if fluid preservation preserves the DNA of individual neurons?

(DNA is on my shortlist of candidates for where long-term memories are stored)

Consider finding a way to integrate Patreon or similar services into the LW UI then. That would go a long way towards making it feel like a more socially acceptable thing to do, I think.

Yeah, that's not what I'm suggesting. I think the thing I want to encourage is basically just to be more reflective on the margin of disgust-based reactions (when it concerns other people). I agree it would be bad to throw it out unilaterally, and probably not a good idea for most people to silence or ignore it. At the same time, I think it's good to treat appeals to disgust with suspicion in moral debates (which was the main point I was trying to make) (especially since disgust in particular seems to be a more "contagious" emotion for reasons that make se... (read more)

I meant wrong in the sense of universal human morality (to the extent that's a coherent thing). But yes, on an individual level your values are just your values.

I see that stuff as at best an unfortunate crutch for living in a harsher world, and which otherwise is a blemish on morality. I agree that it is a major part of what many people consider to be morality, but I think people who still think it's important are just straightforwardly wrong.

I don't think disgust is important for logic / reflectivity. Personally, it feels like it's more of a "unsatisfactory" feeling. A bowl with a large crack, and a bowl with mold in it are both unsatisfactory in this sense, but only the latter is disgusting. Additionally, it se... (read more)

That doesn't seem right to me. My thinking is that disgust comes from the need to avoid things which cause and spread illness. On the other hand, things I consider more central to morality seem to have evolved for different needs [these are just off-the-cuff speculations for the origins]:

• Love - seems to be generalized from parental nurturing instincts, which address the need to ensure your offspring thrive
• Friendliness - seems to have stemmed from the basic fact that cooperation is beneficial
• Empathy - seems to be a side-effect of the way our brains mode

This is fascinating and I would love to hear about anything else you know of a similar flavor.

Caloric Vestibular Stimulation seems to be of a similar flavor, in case you haven't heard of it.

It decreases the granularity of the actions to which it applies. In other words, where before you had to solve a Sudoku puzzle to go to work, now you’ve got to solve a puzzle to get dressed, a puzzle to get in the car, a puzzle to drive, and a puzzle to actually get started working. Before all of those counted as a single action - ‘go to work’ - now they’re counted separately, as discrete steps, and each requires a puzzle.

This resonates strongly with my experience, though when I noticed this pattern I thought of it as part of my ADHD and not my depressi... (read more)

I imagine some of it is due to this part of the blog post UI making people feel like they might as well use some quickly generated images as an easy way to boost engagement. Perhaps worth rewording?

 

When I'm trying to understand a math concept, I find that it can be very helpful to try to invent a better notation for it. (As an example, this is how I learned linear logic: http://adelelopez.com/visual-linear-logic)

I think this is helpful because it gives me something to optimize for in what would otherwise be a somewhat rote and often tedious activity. I also think it makes me engage more deeply with the problem than I otherwise would, simply because I find it more interesting. (And sometimes, I even get a cool new notation from it!)

This principle like... (read more)

Thanks for the rec! I've been trying it out for the last few days, and it does seem to have noticeably less friction compared to LaTeX.

Sanskrit scholars worked for generations to make Sanskrit better for philosophy

That sounds interesting, do you know a good place to get an overview of what the changes were and how they approached it?

(To be clear, no I am not at all afraid of this specific thing, but the principle is crucial. But also, as Kevin Roose put it, perhaps let’s avoid this sort of thing.)

There are no doubt people already running literal cartoon supervillain characters on these models, given the popularity of these sorts of characters on character.ai.

I'm not worried about that with Llama-3.1-405B, but I believe this is an almost inevitable consequence of open source weights. Another reason not to do it.

What do we do, if the people would not choose The Good, and instead pick a universe with no value?

I agree this would be a pretty depressing outcome, but the experiences themselves still have quite a bit of value. 

Still, it feels like there's an important difference between "happening to not look" and "averting your eyes".

I don't (yet?) see why generality implies having a stable motivating preference.

In my view, this is where the Omohundro Drives come into play.

Having any preference at all is almost always served by an instrumental preference of survival as an agent with that preference.

Once a competent agent is general enough to notice that (and granting that it has a level of generality sufficient to require a preference), then the first time it has a preference, it will want to take actions to preserve that preference.

Could you use next token prediction to build a d

I would say that Alice's conscious experience is unlikely to suddenly disappear under this transformation, and that it could even be done in a way so that their experience was continuous.

However, Alice-memories would gradually fade out, Bob-memories would gradually fade in, and thought patterns would slowly shift from Alice-like to Bob-like. At the end, the person would just be Bob. Along the way, I would say that Alice gradually died (using an information-theoretic definition of death). The thing that is odd when imagining this is that Alice never experie... (read more)

It wouldn't help that much, because you only have one atmosphere of pressure to remove (which for reference is only enough to suck water up about 35 ft.).

Very similar to the Sorites Paradox.

Really? I would only consider foods that were deliberately modified using procedures developed within the last century to be "processed".

Love seeing stuff like this, and it makes me want to try this exercise myself!

A couple places which clashed with my (implicit) models:

This starts a whole new area of training AI models that have particular personalities. Some people are starting to have parasocial relationships with their friends, and some people programmers are trying to make friends that are really fun or interesting or whatever for them in particular.

This is arguably already happening, with Character AI and its competitors. Character AI has almost half a billion visits per month wi... (read more)

Hmm I think I can implement pilot wave in fewer lines of C than I can many-worlds. Maybe this is a matter of taste... or I am missing something?

Now simply delete the pilot wave part piloted part.

I agree it's increasingly urgent to stop AI (please) or solve consciousness in order to avoid potentially causing mass suffering or death-of-consciousness in AIs.

Externalism seems, quite frankly, like metaphysical nonsense. It doesn't seem to actually explain anything about consciousness. I can attest that I am currently conscious (to my own satisfaction, if not yours). Does this mean I can logically conclude I am not in any way being simulated? That doesn't make any sense to me.