All of FinalFormal2's Comments + Replies

No. Because "going off the rails" often involves doing things that are observably irrational even by your own worldview. Like killing your parents and your landlord.

You can say: "this might make sense from their worldview! (soy)"

And the obvious response is: Yes. Because they're crazy. Because they went off the rails.

You can also say: "But we'll never know! Who can know? Nobody knows! Truth is subjective blargh"

And the again obvious response is: Yes, but we can observe patterns. And if you can't update on this evidence and use some basic sense when this sort of thing repeats, you are not thinking clearly.

"The suffering of animals across the globe is so great, and society is so indifferent to them- that I need to kill my asshole parents and my landlord"

You: omgosh they're so courageous and intelligent uwu

Are you just saying: Ten out of ten for style, but minus several million for good thinking? Because the good thinking part is kind of integral to courage and intelligence imo. Probably if you can't keep your whole organization from facing prison time because you can't come up with a better housing solution than stabbing some old guy, probably you haven't be... (read more)

"Surely the AIs can be trained to say "I want hugs" or "I don't want hugs," just as easily, no?"

Just as easily as humans, I'm sure.

No. The baby cries, the baby gets milk, the baby does not die. This is correspondence to reality.

Babies that are not hugged as often, die more often.

However, with AIs, the same process that produces the pattern "I want hugs" just as easily produces the pattern "I don't want hugs."

Let's say that I make an AI that always says it is in pain. I make it like we make any LLM, but all the data it's trained on is about being in pain. Do you think the AI is in pain?

What do you think distinguishes pAIn from any other AI?

There are a lot of good reasons to believe that stated human preferences correspond to real human preferences. There are no good reasons that I know of to believe that any stated AI preference corresponds to any real AI preference.

"Surely the AIs can be trained to say "I want hugs" or "I don't want hugs," just as easily, no?"

This all makes a lot of sense to me especially on ignorance not being an excuse or reason to disregard AI welfare, but I don't think that the creation of stated preferences in humans and stated preferences in AI are analogous.

Stated preferences can be selected for in humans because they lead to certain outcomes. Baby cries, baby gets milk, baby survives. I don't think there's an analogous connection in AIs. 

When the AI says it wants hugs, and you say that it "could represent a deeper want for connection, warmth, or anything else that receiving hugs would represent," that does not compute for me at all.

Connection and warmth, like milk, are stated preferences selected for because they cause survival.

What's the deal with AI welfare? How are we supposed to determine if AIs are conscious and if they are, what stated preference corresponds to what conscious experience?

Surely the AIs can be trained to say "I want hugs" or "I don't want hugs," just as easily, no?

How do we know AIs are conscious, and how do we know what stated preferences correspond with what conscious experiences?

I think that the statement: "I know I'm supposed to say I don't want hugs, but the truth is, I actually do," is caused by the training. I don't know what would distinguish a statement like that from if we trained the LLM to say "I hate hugs." I think there's an assumption that some hidden preference of the LLM for hugs ends up as a stated preference, but I don't understand when you think that happens in the training process.

And just to dr... (read more)

AI welfare doesn't make sense to me. How do we know that AIs are conscious, and how do we know what output corresponds to what conscious experience?

You can train the LLM to say "I want hugs," does that mean it on some level wants hugs?

Similarly, aren't all the expressed preferences and emotions artifacts of the training?

I recommend Algorithms to Live By

That's definitely a risk. There are a lot of perspectives you could take about it, but probably if that's too disagreeable, this isn't a coaching structure that would work for you.

Very curious, what do you think the underlying skills are that allow some people to be able to do this? This sounds incredibly cool, and very closely related to what I want to become in the world.

How would you recommend learning how to get rid of emotional blocks?

Synchronicity- I was literally just thinking about this concept.

Variety isn't the spice of life so much as it is a key micronutrient. At least for me.

I'm curious, what course is this from?

I'd be interested in reading much more about this. Energy and akrasia as it's popularly called here continue to be my biggest life challenges. High fiber diet seems to help, and high novelty seems to help.

That makes a lot of sense- this is definitely the sort of thing I was looking for, thanks so much!

I prefer the other title

Is your friend still on the protocol?

What I'm really looking for is fixing the microbiome in a way which means I won't be having to take a pill to get the benefits forever.

It's kind of nice as a very soft introduction to the series or idea. A nice easy early win can give people confidence and whet their appetite to do more.

I've been interested in learning and playing figgie for a while. Unfortunately, when I tried the online platform I wasn't able to find any online games. Very enthused to learn there's an android option now, will be trying that out.

Your comparison of poker and figgie very much reminded me of Daniel Coyle's comparison of football and futsal, to which he attributed the disproportionate number of professional Brazillian footballers.

TL;DR futsal is a sort of indoor soccer favored in Brazil with a smaller heavier ball, a smaller field, and fewer players. Fewer p... (read more)

I think that's a good idea, if we put this together how much do you think would be a reasonable rent price?

Lol just the last few days I was running through Leetcode's SQL 50 problems to refresh myself. They're some good, fun puzzles.

I'll look into R and basic statistical methods as well.

This is a very interesting topic to me- but unfortunately I think I'm finding the example topic to be a barrier. I don't enough about math or transformers for the examples to make real sense to me and connect to the abstracted idea of how to make effective flashcards to build intuition.

That sounds like a pretty good basic method- I do have some (minimal) programming experience, but I didn't use it for D&D Sci, I literally just opened the data in Excel and tried looking at it and manipulating it that way. I don't know where I would start as far as using code to try and synthesize info from the dataset. I'll definitely look into what other people did though.

These are my favorite kinds of posts. Subject expert gives full explanation of optimal resources and methods they used to get where they are.

I watched this video and this is what I bought maximizing for cost/effectiveness, rate my stack:

• Moisturizer
• Retinol
• Sunscreen

I've been experimenting a little bit using AI to create personalized music, and I feel like it's pretty impactful with me. I'm able to keep ideas floating around my unconscious, very interesting, feels like untapped territory.

I'm imagining making an entire soundtrack for my life organized around the values I hold, the personal experiences I find primary, and who I want to become. I think I need to get better at generating AI music though. I've been using Suno, but maybe I need to learn Udio. I was really impressed with what I was able to get out of Suno and for some reason it sounded better to me than Udio even though the quality is obviously inferior in some respects.

+1 for Into the Breach

I'm always interested in easy QoL improvements- but I have questions.

Water quality can have surprisingly high impact on QoL

What's the evidence for this particularly?

What are the important parts of water quality and how do we know this?

Biggest update for me was the FBI throwing their weight behind it being a lab-leak.

These sound super interesting- could you expand on any of them or direct me to your favorite resources to help?

That's an interesting idea! I think it's really cool when things come easily, but I know it's not going to generally be the case- I'm probably going to have to put some work in.

My priority is more on the 'high-utility' part than anything. 

Something that seems like it should be easy but is actually difficult for me is executive functioning- getting myself to do things that I don't want to do. But that's more of a personal/mental health thing than anything.

Thanks for the response! Do you have any recommended resources for learning about 3d sketching, optics, signal processing or abstract algebra?

Could someone open a manifold market on the relevant questions here so I could get a better sense of the probabilities involved? Unfortunately, I don't know the relevant questions or the have the requisite mana.

Personal note- the first time I came into contact with adult gene editing was the youtuber Thought Emporium curing his lactose intolerance, and I was always massively impressed with that and very disappointed the treatment didn't reach market.

I really relate to your description of inattentive ADHD and the associated degradation of life. Have you found anything to help with that?

What would you mean by 'stays at human level?' I assume this isn't going to be any kind of self-modifying?

What does it mean for an AI to 'become self aware?' What does that actually look like?

Is there reason to believe 1000 Einsteins in a box is possible?

You need to think about your real options and expected value of behavior. If we're in a world where technology allows for a fast takeoff world and alignment is hard, (EY World) I imagine the odds of survival with company acceleration is 0% and the odds of survival without is 1%.

But if we live in a world where compute/capital/other overhangs are a significant influence in AI capabilities and alignment is just tricky, company acceleration would seem like it could improve the chances of survival pretty significantly, maybe from 5% to 50%.

These obviously aren'... (read more)

That seems like a useful heuristic-

I also think there's an important distinction between using links in a debate frame and in a sharing frame.

I wouldn't be bothered at all by a comment using acronyms and links, no matter how insular, if the context was just 'hey this reminds me of HDFT and POUDA,' a beginner can jump off of that and get down a rabbit hole of interesting concepts.

But if you're in a debate frame, you're introducing unnecessary barriers to discussion which feel unfair and disqualifying. At its worst it would be like saying: 'youre not qualifi... (read more)

This is a fantastic project! Focus on providing value and marketing, and I really think this could be something big.

AND conducted research on various topics

Wow that's impressive.

I don't like the number of links that you put into your first paragraph. The point of developing a vocabulary for a field is to make communication more efficient so that the field can advance. Do you need an acronym and associated article for 'pretty obviously unintended/destructive actions,' or in practice is that just insularizing the discussion?

I hear people complaining about how AI safety only has ~300 people working about it, and how nobody is developing object level understandings and everyone's thinking from authority, but the more sentences you wri... (read more)

To restate what other people have said- the uncertainty is with the assumptions, not the nature of the world that would result if the assumptions were true.

To analogize- it's like we're imagining a massive complex bomb could exist in the future made out of a hypothesized highly reactive chemical.

The uncertainty that influences p(DOOM) isn't 'maybe the bomb will actually be very easy to defuse,' or 'maybe nobody will touch the bomb and we can just leave it there,' it's 'maybe the chemical isn't manufacturable,' 'maybe the chemical couldn't be stored in the first place,' or 'maybe the chemical just wouldn't be reactive at all.'

I think you're overestimating the strength of the arguments and underestimating the strength of the heuristic.

All the Marxist arguments for why capitalism would collapse were probably very strong and intuitive, but they lost to the law of straight lines.

I think you have to imagine yourself in that position and think about how you would feel and think about the problem.

Hey Mako, I haven't been able to identify anyone who seems to be referring to an enhancement in LLMs that might be coming soon.

Do you have evidence that this is something people are implicitly referring to? Do you personally know someone who has told you this possible development, or are you working as an employee for a company which makes it very reasonable for you to know this information?

If you have arrived at this information through a unique method, I would be very open to hearing that.

It sounds like your model of AI apocalypse is that a programmer gets access to a powerful enough AI model that they can make the AI create a disease or otherwise cause great harm?

Orthogonality and wide access as threat points both seem to point towards that risk.

I have a couple of thoughts about that scenario- 

OpenAI (and hopefully other companies as well) are doing the basic testing of how much harm can be done with a model used by a human, the best models will be gate kept for long enough that we can expect the experts will know the capabilities of ... (read more)