Often you can compare your own Fermi estimates with those of other people, and that’s sort of cool, but what’s way more interesting is when they share what variables and models they used to get to the estimate. This lets you actually update your model in a deeper way.
Popular Comments
Recent Discussion
5th generation military aircraft are extremely optimised to reduce their radar cross section. It is this ability above all others that makes the f-35 and the f-22 so capable - modern anti aircraft weapons are very good, so the only safe way to fly over a well defended area is not to be seen.
But wouldn't it be fairly trivial to detect a stealth aircraft optically?
This is what an f-35 looks like from underneath at about 10 by 10 pixels:
You and I can easily tell what that is (take a step back, or squint). So can GPT4:
...The image shows a silhouette of a fighter jet in the sky, likely flying at high speed. The clear blue sky provides a sharp contrast, making the aircraft's dark outline prominent. The
Note you don't even need high resolution in all directions, just high enough to see whether it's worth zooming in/switching to a better camera.
From a Paul Christiano talk called "How Misalignment Could Lead to Takeover" (from February 2023):
Assume we're in a world where AI systems are broadly deployed, and the world has become increasingly complex, where humans know less and less about how things work.
A viable strategy for AI takeover is to wait until there is certainty of success. If a 'bad AI' is smart, it will realize it won't be successful if it tries to take over, not a problem.
So you lose when a takeover becomes possible, and some threshold of AIs behave badly. If all the smartest AIs...
I've been going through the FAR AI videos from the alignment workshop in December 2023. I'd like people to discuss their thoughts on Shane Legg's 'necessary properties' that every AGI safety plan needs to satisfy. The talk is only 5 minutes, give it a listen:
Otherwise, here are some of the details:
All AGI Safety plans must solve these problems (necessary properties to meet at the human level or beyond):
- Good world model
- Good reasoning
- Specification of the values and ethics to follow
All of these require good capabilities, meaning capabilities and alignment are intertwined.
Shane thinks future foundation models will solve conditions 1 and 2 at the human level. That leaves condition 3, which he sees as solvable if you want fairly normal human values and ethics.
Shane basically thinks that if the above...
I think this is a great idea, except that on easy mode "a good specification of values and ethics to follow" means a few pages of text (or even just the prompt "do good things"), while other times "a good specification of values" is a learning procedure that takes input from a broad sample of humanity, and has carefully-designed mechanisms that influence its generalization behavior in futuristic situations (probably trained on more datasets that had to be painstakingly collected), and has been engineered to work smoothly with the reasoning process and not encourage perverse behavior.
Nate Silver tries to answer the question: "How do people formulate their political beliefs?"
An important epistemological question that is, he says, under-discussed.
He lays out his theory:
...I think political beliefs are primarily formulated by two major forces:
Politics as self-interest. Some issues have legible, material stakes. Rich people have an interest in lower taxes. Sexually active women (and men!) who don’t want to bear children have an interest in easier access to abortion. Members of historically disadvantaged groups have an interest in laws that protect their rights
Politics as personal identity — whose team are you on. But other issues have primarily symbolic stakes. These serve as vehicles for individual and group expression — not so much “identity politics” but politics as identity. People are trying to figure out where
Some issues have legible, material stakes.
Scrolling down... the table of how important are individual topics for young people; "student debt" is at its very bottom.
(Also, inflation on the very top? But isn't inflation a good thing if all you have is an enormous debt?)
ACX recently posted about the Rootclaim Covid origins debate, coming out in favor of zoonosis. Did the post change the minds of those who read it, or not? Did it change their judgment in favor of zoonosis (as was probably the goal of the post), or conversely did it make them think Lab Leak was more likely (as the "Don't debate conspiracy theorists" theory claims)?
I analyzed the ACX survey to find out, by comparing responses before and after the post came out. The ACX survey asked readers whether they think the origin of Covid is more likely natural or Lab Leak. The ACX survey went out March 26th and was open until about April 10th. The Covid origins post came out March 28th, and the highlights on April...
This, and also most people on ACX respect Scott and his opinions, so if he demonstrates that he has put a lot of thought into this, and then he makes a conclusion, it will sound convincing to most.
Basically, we need to consider not just how many people believe some idea, but also how strongly. The typical situation with a conspiracy theory is that we have a small group that believes X very strongly, and a large group that believes non-X with various degrees of strength, from strongly to almost zero. What happens then is that people with a strong belief typ...
This is the ninth post in my series on Anthropics. The previous one is The Solution to Sleeping Beauty.
Introduction
There are some quite pervasive misconceptions about betting in regards to the Sleeping Beauty problem.
One is that you need to switch between halfer and thirder stances based on the betting scheme proposed. As if learning about a betting scheme is supposed to affect your credence in an event.
Another is that halfers should bet at thirders odds and, therefore, thirdism is vindicated on the grounds of betting. What do halfers even mean by probability of Heads being 1/2 if they bet as if it's 1/3?
In this post we are going to correct them. We will understand how to arrive to correct betting odds from both thirdist and halfist positions, and...
First of all, it‘s certainly important to distinguish between a probability model and a strategy. The job of a probability model is simply to suggest the probability of certain events and to describe how probabilities are affected by the realization of other events. A strategy on the other hand is to guide decision making to arrive at certain predefined goals.
Of course. As soon as we are talking about goals and strategies we are not talking about just probabilities anymore. We are also talking about utilities and expected utilities. However, probabilities ...
Announcing open applications for the AI Safety Careers Course India 2024!
Axiom Futures has launched its flagship AI Safety Careers Course 2024 to equip emerging talent working on India with foundational knowledge in AI safety. Spread out across 8-10 weeks, the program will provide candidates with key skills and networking opportunities to take their first step toward an impactful career in the domain. Each week will correspond with a curriculum module that candidates will be expected to complete, and discuss with their cohort during the facilitated seminar. We expect a set of candidates to pursue applied projects of their choice.
The program is aimed at undergraduate and (Master’s/PhD) graduate students, and young professionals. If you are a high-school student with demonstrated interest in AI safety, we encourage you to apply. We expect applicants...
I am honored to be part of enabling more people from around the world to contribute to the safe and responsible development of AI.
Effective Altruism (EA) is a movement trying to invest time and money in causes that do the most possible good per some unit investment. EA was at one point called optimal philanthropy.of effort. The label applies broadly, including a philosophy, a community, a set of organisations and set of behaviours. Likewise it also sometimes means how to donate effectively to charities, choose one's career, do the most good per $, do good in general or ensure the most good happens. All of these different framings have slightly different implications.
The basic concept behind EA is that youone would really struggle to donate 100 times more money or time to charity than you currently do but, spending a little time researching who to donate to could have an impact on roughly this order of magnitude. The same argument works for doing good with your career or volunteer hours.
GPT-5 training is probably starting around now. It seems very unlikely that GPT-5 will cause the end of the world. But it’s hard to be sure. I would guess that GPT-5 is more likely to kill me than an asteroid, a supervolcano, a plane crash or a brain tumor. We can predict fairly well what the cross-entropy loss will be, but pretty much nothing else.
Maybe we will suddenly discover that the difference between GPT-4 and superhuman level is actually quite small. Maybe GPT-5 will be extremely good at interpretability, such that it can recursively self improve by rewriting its own weights.
Hopefully model evaluations can catch catastrophic risks before wide deployment, but again, it’s hard to be sure. GPT-5 could plausibly be devious enough to circumvent all of...
It probably began training in January and finished around early April. And they're now doing evals.
