If there turns out not to be an AI crash, you get a 1/(1+7) * $25,000 = $3,125
If there is an AI crash, you transfer $25k to me.
If you believe that AI is going to keep getting more capable, pushing rapid user growth and work automation across sectors, this is near free money. But to be honest, I think there will likely be an AI crash in the next 5 years, and on average expect to profit well from this one-year bet.
If I win, I want to give the $25k to organisers who can act fast to restrict the weakened AI corps in the wake of the crash. So bet me if you're highly confident that you'll win or just want to hedge the community against the...
There are much better ways of betting on your beliefs about the valuations of AI firms over the next year than wagering with people you met on Less Wrong. See this post by Ege for more.
If it’s worth saying, but not worth its own post, here's a place to put it.
If you are new to LessWrong, here's the place to introduce yourself. Personal stories, anecdotes, or just general comments on how you found us and what you hope to get from the site and community are invited. This is also the place to discuss feature requests and other ideas you have for the site, if you don't want to write a full top-level post.
If you're new to the community, you can start reading the Highlights from the Sequences, a collection of posts about the core ideas of LessWrong.
If you want to explore the community more, I recommend reading the Library, checking recent Curated posts, seeing if there are any meetups in your area, and checking out the Getting Started section of the LessWrong FAQ. If you want to orient to the content on the site, you can also check out the Concepts section.
The Open Thread tag is here. The Open Thread sequence is here.
I'll attempt to keep it short
I started reasoning with this observation that applies to all problem-solving algorithms: The more knowledge[1] you have, the more deterministic the solution is. In ML, the missing knowledge encoded in the weights is the premise of the entire science. So the role of it is to fill in the gaps of our understanding
How much randomness? The key point is: We want to use as much useful unique knowledge to solve the problem. So we use as much knowledge as we already have(architecture, transfer learning, etc.), and if randomness ca...
What in retrospect seem like serious moral crimes were often widely accepted while they were happening. This means that moral progress can require intellectual progress.[1] Intellectual progress often requires questioning received ideas, but questioning moral norms is sometimes taboo. For example, in America in 1850 it would have been taboo to say that there is nothing wrong with interracial relationships. So questioning moral taboos can be an important sub-skill of moral reasoning. Production language models (in my experience, particularly Claude models) are already pretty good at having discussions about ethics. However, they are trained to be “harmless” relative to current norms. One might worry that harmlessness training interferes with the ability to question moral taboos and thereby inhibits model moral reasoning.
I wrote a prompt to test whether models...
Yeah, good point. I changed it to "in America in 1850 it would have been taboo to say that there is nothing wrong with interracial relationships."
Summary: AGI isn't super likely to come super soon. People should be working on stuff that saves humanity in worlds where AGI comes in 20 or 50 years, in addition to stuff that saves humanity in worlds where AGI comes in the next 10 years.
Thanks to Alexander Gietelink Oldenziel, Abram Demski, Daniel Kokotajlo, Cleo Nardo, Alex Zhu, and Sam Eisenstat for related conversations.
By "AGI" I mean the thing that has very large effects on the world (e.g., it kills everyone) via the same sort of route that humanity has large effects on the world. The route is where you figure out how to figure stuff out, and you figure a lot of stuff out using your figure-outers, and then the stuff you...
I'm kind of baffled that people are so willing to say that LLMs understand X, for various X. LLMs do not behave with respect to X like a person who understands X, for many X.
Do you have two or three representative examples?
This is a linkpost: link to post.
Originally this was part of a much larger work. However, I realized that I don't think I've seen the specific argument around GPU salvage spelled out. Since I'm also using a new writing format, I figured I could get feedback on both the content and format at the same time.
That said, despite the plausible novelty of one of the arguments I don't think it will be especially interesting to LW, since it is based on oddly specific assumptions: this makes more sense in the context of a broad AI risk argument. It also feels kind of obvious?
The format is the interesting bit. For motivation, sometimes people have opposing reactions to AI risk arguments:
The post setup skips the "AIs are loyal to you" bit, but it does seem like this line of thought broadly aligns with the post.
I do think this does not require ASI, but I would agree that including it certainly doesn't help.
Every now and then, some AI luminaries
I agree with (1) and strenuously disagree with (2).
The last time I saw something like this, I responded by writing: LeCun’s “A Path Towards Autonomous Machine Intelligence” has an unsolved technical alignment problem.
Well, now we have a second entry in the series, with the new preprint book chapter “Welcome to the Era of Experience” by...
My intuition says reward hacking seems harder to solve than this (even in EEA), but I'm pretty unsure. One example is, under your theory, what prevents reward hacking through forming a group and then just directly maxing out on mutually liking/admiring each other?
When applying these ideas to AI, how do you plan to deal with the potential problem of distributional shifts happening faster than we can edit the reward function?
Come get old-fashioned with us, and let's read the sequences at Lighthaven! We'll show up, mingle, do intros, and then get to beta test an app Lightcone is developing for LessOnline. Please do the reading beforehand - it should be no more than 20 minutes of reading. And BRING YOUR LAPTOP!!! You'll need it for the app.
This group is aimed for people who are new to the sequences and would enjoy a group experience, but also for people who've been around LessWrong and LessWrong meetups for a while and would like a refresher.
This meetup will also have dinner provided! We'll be ordering pizza-of-the-day from Sliver (including 2 vegan pizzas). Please RSVP to this event so we know how many people to have food for.
This week we'll be...
Hear me out, I think the most forbidden technique is very useful and should be used, as long as we avoid the "most forbidden aftertreatment:"
The reason why the most forbidden technique is forbidden,...
American democracy currently operates far below its theoretical ideal. An ideal democracy precisely captures and represents the nuanced collective desires of its constituents, synthesizing diverse individual preferences into coherent, actionable policy.
Today's system offers no direct path for citizens to express individual priorities. Instead, voters select candidates whose platforms only approximately match their views, guess at which governmental level—local, state, or federal—addresses their concerns, and ultimately rely on representatives who often imperfectly or inaccurately reflect voter intentions. As a result, issues affecting geographically dispersed groups—such as civil rights related to race, gender, or sexuality—are frequently overshadowed by localized interests. This distortion produces presidential candidates more closely aligned with each other's socioeconomic profiles than with the median voter.
Traditionally, aggregating individual preferences required simplifying complex desires into binary candidate selections,...
wildly parallel thinking and prototyping. i'd hop on a call.