Holden shares his step-by-step process for forming opinions on a topic, developing and refining hypotheses, and ultimately arriving at a nuanced view - all while focusing on writing rather than just passively consuming information.
(Disclaimer: This is my personal opinion, not that of any movement or organization.)
This post aims to show that, over the next decade, it is quite likely that most democratic Western countries will become fascist dictatorships - this is not a tail risk, but the most likely overall outcome. Politics is not a typical LessWrong topic, and for good reason:
However, like the COVID pandemic, it seems like this particular trend will be so impactful and so disruptive to ordinary Western life that it will be important to be aware of it, factor it into plans,...
Just my opinion: the concerns are valid but exaggerated.
Perhaps exaggeration is justified in order to get people's attention since most americans are still treating this as a normal election.
This is the 5th of a series of 8 blog posts, which I’m serializing weekly. (Or email or DM me if you want to read the whole thing right now.)
Dissociative Identity Disorder (DID) (previously known as “Multiple Personality Disorder”) involves a person having multiple “alters” (alternate identities), with different preferences and (in some cases) different names. A DID diagnosis also requires some nonzero amount of “inter-identity amnesia”, where an alter cannot recall events that occurred when a different alter was active. For example, DSM-V talks about patients “coming to” on a beach with no recollection of how they got there.
Anyway, just like trance in the previous post, DID was one of those things that I unthinkingly assumed was vaguely fictional for most of...
Good question! I think there are two different steps here. (The following is a bit oversimplified.)
Step 1 is auto-associative recall in the hippocampus. Neurons all over the cortex directly or indirectly activate neurons in the hippocampus. And then if something is happening in any part of the cortex that partially matches some old memory, the whole old memory can autocomplete within the hippocampus.
Step 2 is: that core of an old memory has lots of (direct and indirect) associations all around the cortex / global workspace. Like, if it’s a visual memory, t...
It's been a busy season at the Nucleic Acid Observatory, and we have a lot to share since our last update. As always, If anything here is particularly interesting or if you’re working on similar problems, please reach out!
We performed an initial analysis of untargeted sequencing data from aggregated airplane lavatory waste and municipal treatment plant influent that we collected and processed during our Fall 2023 partnership with CDC’s Traveler-based Genomic Surveillance program and Ginkgo Biosecurity. We've now analyzed viral abundance and diversity in sequencing data across multiple sample types and wastewater-processing and sequencing protocols. Next steps include further investigating how protocol and sample-type affect specific viruses and bacteria, as well as understanding pathogen temporal dynamics seen in airport versus treatment-plant samples.
We have continued to work...
That is a very cute/nice logo.
Dario Amodei is thinking about the potential. The result is a mostly good essay called Machines of Loving Grace, outlining what can be done with ‘powerful AI’ if we had years of what was otherwise relative normality to exploit it in several key domains, and we avoided negative outcomes and solved the control and alignment problems. As he notes, a lot of pretty great things would then be super doable.
Anthropic also offers us improvements to its Responsible Scaling Policy (RSP, or what SB 1047 called an SSP). Still much left to do, but a clear step forward there.
Daniel Kokotajlo and Dean Ball have teamed up on an op-ed for Time on the need for greater regulatory transparency. It’s very good.
Also, it’s worth checking out the Truth Terminal...
Are the ‘AI companion’ apps, or robots, coming? I mean, yes, obviously?
The technology for bots who are "better" than humans in some way (constructive, pro-social, compassionate, intelligent, caring interactions while thinking 2 levels meta) has been around since 2022. But the target group wouldn't pay enough for GPT-4-level inference, so current human-like bots are significantly downscaled compared to what technology allows.
It’s monthly roundup time again, and it’s happily election-free.
Propaganda works, ancient empires edition. This includes the Roman Republic being less popular than the Roman Empire and people approving of Sparta, whereas Persia and Carthage get left behind. They’re no FDA.
Polling USA: Net Favorable Opinion Of:
Ancient Athens: +44%
Roman Empire: +30%
Ancient Sparta: +23%
Roman Republican: +26%
Carthage: +13%
Holy Roman Empire: +7%
Persian Empire: +1%
Visigoths: -7%
Huns: -29%
YouGov / June 6, 2024 / n=2205
What do we do about all 5-star ratings collapsing the way Peter describes here?
...Peter Wildeford: TBH I am pretty annoyed that when I rate stuff the options are:
* “5 stars – everything was good enough I guess”
* “4 stars – there was a serious problem”
* “1-3 stars – I almost died”
I can’t
Supply side: It approaches the minimum average total, not marginal, cost. Maybe if people accounted for it finer (e.g., charging self "wages" and "rent"), cooking at home would be in the ballpark (assuming equal quality of inputs and outputs across venues..), but that just illustrates how real costs can explain a lot of the differential without having to jump to regulation and barriers to entry (yes, those are nonzero too!).
Demand side: Complaints in the OP about the uninformativeness of ratings also highlight how far we are from perfect competition (also,...
...I think and talk a lot about the risks of powerful AI. The company I’m the CEO of, Anthropic, does a lot of research on how to reduce these risks. Because of this, people sometimes draw the conclusion that I’m a pessimist or “doomer” who thinks AI will be mostly bad or dangerous. I don’t think that at all. In fact, one of my main reasons for focusing on risks is that they’re the only thing standing between us and what I see as a fundamentally positive future. I think that most people are underestimating just how radical the upside of AI could be, just as I think most people are underestimating how bad the risks could be.
In this essay I try to sketch out what that
My current belief is that this essay is optimized to be understandable by a much broader audience than any comparable public writing from Anthropic on extinction-level risk.
For instance, did you know that the word 'extinction' doesn't appear anywhere on Anthropic's or Dario's websites? Nor do 'disempower' or 'disempowerment'. The words 'existential' and 'existentially' only come up three times: when describing the work of an external organization (ARC), in one label in a paper, and one mention in the Constitutional AI. In its place they always talk a...
the following is motivated by:
I've been a long time lurker on Less Wrong and I've noticed the recurring criticism that despite its focus on rationality, the community lacks structured training to develop practical rationality skills. Eliezer Yudkowsky talks rationality as a martial art, because it's something that can be trained and refined through deliberate practice. But where is our dojo?
A model that comes to mind is a website like LeetCode, where programmers can solve coding challenges, share solutions, and see how others approach the same problems. LeetCode can sometimes encourage overfitting to specific problem types so it's not a perfect analogy. The community driven aspect would interesting to me as you can see how...
like the calibration game but for a variety of decision problems, where the person has to assign probabilities to things at different stages based on what information is available. Afterwards they get an example brier score based on the average of what people with good prediction track records set at each phase.
How can we make many humans who are very good at solving difficult problems?
I made up the made-up numbers in this table of made-up numbers; therefore, the numbers in this table of made-up numbers are made-up numbers.
If you have a shitload of money, there are some projects you can give money to that would make supergenius humans on demand happen faster. If you have a fuckton of money, there are projects whose creation you could fund that would greatly accelerate this technology.
If you're young and smart, or are already an expert in either stem cell / reproductive biology, biotech, or anything related to brain-computer interfaces, there are some projects you could work on.
If neither, think hard, maybe I missed something.
You can...
Math data!
Alright, I have a question stemming from TurnTrout's post on Reward is not the optimization target, where he argues that the premises that are required to get to the conclusion of reward being the optimization target are so narrowly applicable as to not apply to future RL AIs as they gain more and more power:
But @gwern argued with Turntrout that reward is in fact the optimization target for a broad range of RL algorithms:
https://www.lesswrong.com/posts/ttmmKDTkzuum3fftG/#sdCdLw3ggRxYik385
So my question is are there known results, ideally proofs, but I can accept empirical studies if necessary that show when RL algorithms treat the reward function as an optimization target?
And how narrow is the space of RL algorithms that don't optimize for the reward function?
A good answer will link to results known in...
I guess LLMs are model-free, so that's relevant
FWIW, I strongly disagree with this claim. I believe they are model-based, with the usual datasets & training approaches, even before RLHF/RLAIF.