I will definitely be checking out those books, thanks, and your response clarified the intent a lot for me.
As for where new metaphors/mechanisms come from, and whether they're ever created out of nothing, I think that that is very very rare, probably even rarer than it seems. I have half-joked with many people that at some level there are only a few fundamental thoughts humans are capable of having, and the rest is composition (yes, this is metaphorically coming from the idea of computers with small instruction sets). But more seriously, I think it's mostl...
(1) I agree, but don't have confidence that this alternate approach results in faster progress. I hope I'm proven wrong.
(4) Also agreed, but I think this hinges on whether the failing plans are attempted in such a way that they close off other plans, either by affecting planning efforts or by affecting reactions to various efforts.
(5) Fair enough.
Liron: Carl Feynman. What is your P(Doom)?
Carl: 43%.
Comments like this always remind me of the Tetlock result that forecasters who report probability estimates using more-precise, less-round numbers do in fact outperform others, and are more correctly incorporating the sources of information available.
I'm curious if you have an opinion on the relatives contributions of different causes, such as:
I'm thinking (as an example...
It's kind of fun to picture AI agents working during the day and resting at night. Maybe that's the true AGI moment.
In context, this will depend on the relative costs of GPUs and energy storage, or the relative value of AI vs other uses of electricity that can be time-shifted. I would happily run my dryer or dishwasher during the daytime instead of at night in order to get paid to let OpenAI deliver a few million extra tokens. Liberalizing current electricity market participation and the ability to provide ancillary services has a lot of unrealized potenti...
This would be great to have, for sure, and I wish you luck in working on it!
I wonder if, for the specific types of discussions you point to in the first paragraph, it's necessary or even likely to help? Even if all the benchmarks today are 'bad' as described, they measure something, and there's a clear pattern of rapid saturation as new benchmarks are created. METR and many others have discussed this a lot. There have been papers on it. It seems like the meta-level approach of mapping out saturation timelines should be sufficient to convince people that fo...
Upvoted - I do think lack of a coherent, actionable strategy that actually achieves goals if successful is a general problem of many advocacy movements, not just AI. A few observations:
(1) Actually-successful historical advocacy movements that solved major problems usually did so incrementally over many iterations, taking the wins they could get at each moment while putting themselves in position to take advantage when further opportunities arose.
(2) Relatedly, don't complain about incremental improvements (yours or others'). Celebrate them, or no one will...
I can't tell which answer to this question is meant to be 'for' or 'against' the OP's point, but it sounds like the latter. Even if it's the case the neurons contain something useful nutritionally (and I'd be surprised but not too surprised if it were), consider that these shellfish have neurons, and unlike other meats, we actually eat the neurons instead of them being part of organs we remove before eating. Also, that we have very good reason to avoid eating the neural tissues of mammals.
On that first paragraph, we agree.
On the second paragraph - I could see this being an interesting approach, if you can get a good critical mass of employers with sufficiently similar needs and a willingness to try it. Certainly much better than some others I've seen, like (real example from 2009) "Let's bring in 50 candidates at once on a Saturday for the whole day, and interview them all in parallel for 2 positions."
I think one, slightly deeper problem is - who is doing the short interviews or the screenings? Do they actually know what the job entails and...
Upvoted. There's a lot I find interesting and a lot I agree with in this (not always the same things).
This one stood out to me as a non-central point I don't think I agree with, though:
Humans would probably appreciate their art, at least simple forms of their art (intended e.g. for children), more than they would appreciate the artistic value of marginal un-optimized nature on the alien planet.
I actually don't think this applies to me even on Earth, among humans. If you show me a random piece of average art made by and for humans, or for human children, an...
Agreed, and for sure it has been working well for 4 years. I just don't think it's what I want for the next 40. It's not intolerable. The benefits have been great, especially with the travel involved in my case. And good communal spaces help. But that's a different question from whether the costs of having more space outweigh the benefits, in general or for particular people.
To be clear, I disagree with your unstated assumption that anything happening in London can be used to draw conclusions about the impact of YIMBY anything. If London were building an order of magnitude more homes per year, with the same dynamic, then sure, after a decade I'll admit the point. But barring that, a world where you have a steady large influx of public funds to spend on actual residents, at no cost to those residents, is a good thing.
Boomers are selling their houses for high prices and then downsizing and paying for starter homes with cash. So young families can’t afford the homes they’re putting for sale and then get outbid the houses they can afford. They’ve totally ruined the market it’s insane.
So... who does someone like this think is buying the boomers' homes, and deciding how much they're willing and able to pay? What houses do they think those buyers are moving from, and therefore vacating?
They also seem to think the concept of a starter home is new?
I'm not sure what I think about the value of spaciousness. I'd love to hear about any specific examples you have in mind.
Sure. To be clear: It doesn't apply to everything. And individual rooms don't/wouldn't need to be huge. I've looked at house floor plans with walk-in closets in the master bedroom that are larger than my whole trailer, or big empty bathrooms and entryways, and find them to be just silly for any purpose I could imagine wanting. But for some personal examples where I think this applies:
My wife is a therapeutic musician who makes online cou...
One thing I am unclear on is (if we know) why trust was decreasing.
Where the entities losing trust becoming less trustworthy?
Were they always untrustworthy but this became more widely known?
Were they still trustworthy but people began thinking they weren't for some other reason?
In other words, should the goal be to increase trust, increase trustworthiness, or increase accuracy of perceptions and evaluations of trustworthiness?
To be fair, I think they should just be banned from having no-human-in-the-loop screenings
In principle I agree. In practice I'm less sure.
save a few hours of reading
Consider that the average job opening these days receives hundreds of applications. For some people that's a few hours of reading, but it's reading hundreds of near-identical-looking submissions and analyzing for subtle distinctions.
I do think automated screenings rule out a lot of very good options because they don't look like the system thinks they should, in ways a thoughtful human wou...
From personal experience: I grew up in a much bigger house than I owned as an adult, and definitely had a tendency to hold onto stuff I didn't need. Then in 2021 I sold my house and for the past 4 years my wife and I have been living full-time in a 28' trailer, working from the road and traveling the country. So learning a minimalist mindset has been an essential life skill. I'm trying to separate the minimalism component from the travel component in what's below, but admittedly it's a big part of the value for me.
There are tradeoffs. I pay more for ...
Total, but I don't think the difference is as large as it might seem. Fundamentally, barring another collapse that stops our advancement, I don't think we have more than about a century, at the high end, before we reach a point technologically where we're no longer inescapably dependent on the climate for our survival. Which means almost all my probability for how climate could cause human extinction involves something drastic happening within the next handful of decades.
Most of that remaining probability looks something like "We were wrong to reject the m...
I agree with you that deploying AI in high-impact safety-critical applications under these conditions and relying on the outputs as though they met standards they don't meet is insane.
I would, naturally, note that (1), (2), and (3) also apply to humans. It's not like we have any option for deploying minds in these applications that don't have some version of these problems. LLMs have different versions we don't understand anywhere near as well as we do the limitations of humans, but what that means for how and whether we can use them, even now, to improve ...
(5) I look forward to it.
(2) I hope you'll dig into this more in those future posts, because I think it is extremely non-obvious.
(3) Yes, I will concede that example, you're right. For any observer in any possible world, there are an arbitrarily large number of larger universes within which it could be a perfect simulation, and these would be indistinguishable from the inside. This is a thing we cannot know, and the choice to then act as if those unknowable things don't exist is an additional choice. I definitely did not think this was the kind of metaphys...
I think this is an important point, especially when experts are talking to other experts about their respective fields. I once had a client call this "thinking in webs." If you have a conclusion that you reached via a bunch of weak pieces of evidence collected over a bunch of projects and conversations and things you've read all spread out over years, it might or might not be epistemically correct to add those up to a strong opinion. But, there may be literally no verbally compelling way to express the source of that certainty. If you try, you'll have forg...
It's sorta like collaborating with a human that you don't trust, except you can conduct as many experiments as you want to improve your understanding of their biases, and of how they respond to different ways of interacting. AI tells me I'm wrong all the time, but it takes work to make sure that stays the case.
I reminds me a little of a class where the teacher asked us how to get truly random results from a coin that may or may not be biased, with unknown bias. The answer is that even if H and T are not equiprobable, HT/TH, or HHTT/TTHH, or HHHHTTTT/TTTTHH...
Ok, fair, 'prove' is a strong word and we can have different opinions on both the probability estimate of climate-induced-extinction and the threshold for that probability being low enough to count as 'not an x-risk.'
In order to actually wipe out all humanity, such that there were no residual populations able to hang on long enough to recover and rebuild, the climate would need to change faster than any human population, anywhere in the world, could adapt or invent solutions. Even if life sucked for a decade or a millennium, or if there were only 10k of us...
If we were, no one ever told us, and no one I knew ever did. If nothing else, to do so, we would have had to skip lunch entirely, because we weren't allowed to be in the halls without a pass signed by a teacher, and there would not have been anyone in the cafeteria to write one.
Admittedly, after 9th grade I stopped taking lunch so I could fit in an extra elective. Also in 9th grade, we had 4 instances of students calling in fake bomb threats in order to get out of class, and ended up with much stricter rules about who could be where, when, than had been th...
The school essay is designed to be writable within the time constraints of an in-class exam, and to let teachers grade a whole five class's worth of essays fast enough to get them back long enough before the next exam.
As a kid I was always confused about why schools had libraries. In elementary school you got sent to them once a week and were allowed to take out one book, and the librarian taught how to use a library. Otherwise, there was never a time you could actually go to them and do research on anything. (Ok, except for once in second grade, when I co...
Jabberwocky is my favorite poem. People who know me well hear that and are completely unsurprised :-)
I am locating the meaning outside the statement, rather than applying validity or invalidity to the statement itself.
This is critical, I think. In daily life I make up words all the time, and the people around me with whom I share the necessary context and are also native English speakers have no trouble intuiting what they mean.
That was, as far as I can tell, one strong downvote from me (-7, from a starting value of 2). As my comment above hopefully indicates, I did read the whole thing. I don't know if it was as fast as five minutes after posting, but this post happened to be second on the front page when I looked, so I read through it, downvoted, then commented. It's about 2200 words, which usually means anywhere from 5-10 minutes read time for me. I did reread it slower while commenting, as well, and the second readthrough did not cause me to change my downvote.
(1) Ok, fair enough, that wasn't clear to me on first read. I do think it's worth noting that he does, in fact, consider many other viewpoints before rejecting them, and gives clear explanations of his reasons for doing so, whether you agree or not. He also in many places discusses why he thinks introducing those other viewpoints does not actually help. Others in the community have since engaged with similar ideas from many other viewpoints.
(2) That conclusion does not follow from the premises. In particular, you have not considered the set of possible wor...
Strongly downvoted, seems to not realize how deeply EY has engaged with and written about metaphysics, or at least not to engage with any of his relevant writings or those of the rest of the rationalist community over the last almost 20 years.
Besides that, though: It's not clear to me how a non-physicalist metaphysics actually helps reduce x-risk, except to the extent that there is some probability of an outside force intervening in our physical cosmos. For one example among many, consciousness is not required to run physical simulations and identify...
Upvoted, well written explanation.
Might be worth explicitly including a link back to A Human's Guide to Words.
Others have already added some good ones, here's a few more.
A few that are likely familiar throughout this community even if you have never considered them this way, but still often not believed or noticed elsewhere:
In a similar vein:
I think it's also worth keeping in mind that the overall state of the field of "People who make and publish reports forecasting the future of emerging technologies" (which was my field for over a decade) is usually really, really bad (this includes the kinds of reports executives and investors will pay $5k a pop for, to help them make big decisions). When I read AI 2027 and the accompanying documents, it was very easily within the top 1% of such reports I've seen in quality, thoughtfulness, thoroughness, and reasonableness-of-assumptions-made.
I'd also add ...
Yeah, that boundary gets very confused very fast. I've come across articles, written by professionals for a general audience, calling whole wheat flour ultra-processed, and others listing cutting as a processing method that makes food less healthy. My general opinion of most standard diet advice is that it's at about this level of reliability.
It seems like you're aware of this, but AFAICT surprisingly many people who speak out about seed oils aren't, so it may be worth stating outright: uncooked or lightly cooked PUFAs are among the healthiest fats you can eat, come primarily from nuts and seeds (and fatty fish), and include all the essential and metabolically-important fatty acids. They diffuse faster across mitochondrial membranes for energy production, esterify and de-esterify at greater rates to provide energy between meals, and are preferentially stored in parts of the body where they're l...
This seems right, and I'm glad to have book-length explanations and investigations of it. "Climate change is not an x-risk" is the kind of thing you can easily (and correctly) prove to yourself in a matter of hours, but it is surprisingly hard to get other people to both notice and admit it, especially en masse.
I will say this was not so clear 20+ years ago. Back then we hadn't yet shown the runaway warming scenarios were unlikely, clean energy was far from cheap anywhere in the world, many industrial processes had no clear path to electrification, and aff...
I appreciate that point, yes, and I have looked up standard definitions. I'm probably not looking in the right places, though, because they ones I have found are either too vague and imprecise for me to make sense of, or focus on generating hypotheses/explanations/models. If you do have a good source for a better explanation, I'd actually really like to learn more.
Basically, yeah. Most times I see abduction discussed, it's less about drawing conclusions and more about hypothesis generation. That implies different permissible levels of making and breaking assumptions, choosing and changing models. It's more fluid, less rule-bound, more willing to accept being knowingly wrong in some ways, less tied to formalisms and precise methods.
Mostly my wife handles our blog. We wrote about the trivia example in this post.
We don't tend to write about anyone's kids, for obvious reasons, and I don't have kids of my own. But there are a lot of full-time families with youtube channels and blogs that roadschool their kids. See here for a perspective similar to what I said above. I can say, in general, kids who spend months or more on the road exploring with their families seem to be more independent, more aware, more curious, and more able to interact with people and things around them, than others t...
Just inducting probabilities and then deducting the most likely outcome.
I find it's good practice to be deeply suspicious of the word "just." Small words in arguments are often load-bearing in ways that hide much of the meaning from casual readers. E.g. LLMs are just applied arithmetic, biology is just applied chemistry, chemistry is just applied physics, etc. There is a sense in which this is 'true' in each case, but that does not make the less-fundamental concepts useless or unnecessary, and straightforwardly 'believing' such 'just' statements tends to c...
Ah, I see, ok.