There’s some chance that the patchwork AI safety strategy of the leading companies might just work well enough
I wonder if it'd be a good idea to mention that "a small number of humans have intent-aligned vastly superhuman AIs" does not yet imply "things will be nice (or tolerable) for most people"?
OTOH, then you maybe also need to point out that "superhuman AIs are open-weights and anyone can run them" also leads to death-or-worse (to pre-empt "solutions" like "so let's give everyone AI!")?
1The four probabilities given as premises are inconsistent. The first three determine the fourth. (Also, there's an arithmetic error in the p(G|M) calculation, as pointed out by Bucky.)
Given
- p(M) = 0.9
- p(G) = 0.05
- p(M|G) = 0.99
it must be that
p(M|-G) = (p(M) - p(M,G)) / (1 - p(M,G) - p(-M,G)) = 0.8505 / 0.95
which is approximately 0.895263. Not 0.02.
If this feels confusing, I suggest drawing a Venn diagram or something. If you have a box of area 1.0, containing a blob M of area 0.9 and another blob G of area 0.05, such that G is almost entirely inside M, then...
Interesting. Thanks. How did you arrive at the above picture? Any sources of information you'd recommend in particular?
After reading about Trump's actions w.r.t. Greenland, I'm updating further away from
- Trump's policies/actions are at least partially aimed at pursuing the national interests of the US (albeit possibly in a very misguided/incompetent way)
and further in favor of both
- Trump has gone insane,
- and/or Trump is intentionally acting on behalf of Putin.
I'd like to find more/better sources of evidence about "what is the US executive branch optimizing for?"; curious to hear suggestions.
(Also, to Americans: How high/low salience is the issue in the US? Also: curious to read your analysis of your chief executive's behavior.)
Dominic Cummings (former Chief Adviser to the UK PM) has written some things about nuclear strategy and how it's implemented in practice. IIUC, he's critical of (i.a.) how Schelling et al.'s game-theoretic models are (often naively/blindly) applied to the real world.
I updated a bit towards thinking that incompetence-at-reasoning is a more common/influential factor than I previously thought. Thanks.
However: Where do you think that moral realism comes from? Why is it a "thorny" issue?
1social-media-like interfaces for uncovering group wisdom and will at larger scales while eliciting more productive discourse
That seems like it might significantly help with raising the sanity waterline, and thus help with coordinating on AI x-risk, and thus be extremely high-EV (if it's successfully implemented, widely adopted, and humanity survives for a decade or two beyond widespread adoption). [1]
Do you think it would be practically possible with current LLMs to implement a version of social media that promoted/suggested content based on criteria like
- validity of reasoning,
- concrete factual claims (whether correct or incorrect), vs vibing,
- thoughtful long-form writing, vs memeing,
- good-faith debate, vs e.g. adversarial tribalistic signaling?
The "widely adopted" part seems difficult to achieve, though. The hypermajority of genpop humans would probably just keep scrolling TikTok and consuming outrage porn on X, even if Civilization 2.0 Wholesome Social Media were available. ↩︎
This discourse structure associates related claims and evidence, [...]
To make it practically possible for non-experts to efficiently make sense of large, spread-out collections of data (e.g. to answer some question about the discourse on some given topic), it's probably necessary to not only rapidly summarize all that data, but also translate it into some easily-human-comprehensible form.
I wonder if it's practically possible to have LMs read a bunch of data (from papers to Twitter "discourse") on a given topic, and rapidly/on-demand produce various kinds of concise, visual, possibly interactive summaries of that topic? E.g. something like this, or a probabilistic graphical model, or some kind of data visualization (depending on what aspect of what kind of topic is in question)?
Ideally perhaps, raw observations are reliably recorded, [...]
Do you have ideas for how to deal with counterfeit observations or (meta)data (e.g. deepfaked videos)?
I think something like what you're sketching here, viz "harnessing technology to make people and civilization saner", is probably highly valuable and possibly quite neglected. [1] Thank you for working on this.
A class of infrastructure/technologies that seem very important, but which I didn't see mentioned in this post: infrastructure for creating common knowledge of better equilibria and coordinating transitions to them. [2] Do you (have plans to) address anything matching that (vague) description anywhere?
Low-hanging dignity points! ↩︎
I.e., something that would solve problems of form "there exists a much better equilibrium, but getting there would require lots of people to have common knowledge of that better equilibrium, and also coordinate and sufficiently-credibly commit to near-simultaneously taking action that would be detrimental to them if they took it alone". Some examples: move from frequentist stats to Bayesian stats; make it easier for AI labs to (conditionally) stop racing; US voters coordinate to vote for a less sociopathic party-independent candidate (or to replace first-past-the-post with a saner voting system entirely); abolish all JavaScript forever, refactor the Internet to use a non-insane language; kill Elsevier; almost everyone simultaneously leaves (at least the more toxic platforms of) social media (and move to a less toxic new platform); journals/researchers commit to preregistering studies and publishing negative results; etc. ↩︎