Possibly look for other skills / career paths, besides math and computer science? Glancing through 80,000 Hours' list:
- AI governance and policy -- I'm guessing that seeking out "policy people" will be a non-starter in Russia, either because it's dangerous or because there are fewer such people (not whole graduating classes at Harvard, etc, waiting to become the next generation of DC elites).
- AI safety technical research -- of course you are already thinking about this via IOM, IOI, etc. Others have mentioned trying to expand to LLM-specific competi...
Satellites were also plausibly a very important military technology. Since the 1960s, some applications have panned out, while others haven't. Some of the things that have worked out:
Maybe other people have a very different image of meditation than I do, such that they imagine it as something much more delusional and hyperreligious? Eg, some religious people do stuff like chanting mantras, or visualizing specific images of Buddhist deities, which indeed seems pretty crazy to me.
But the kind of meditation taught by popular secular sources like Sam Harris's Waking Up app, (or that I talk about in my "Examining The Witness" youtube series about the videogame The Witness), seems to me obviously much closer to basic psychology or rationali...
I think there are many cases of reasonably successful people who often cite either some variety of meditation, or other self-improvement regimes / habits, as having a big impact on their success. This random article I googled cites the billionaires Ray Dalio, Marc Benioff, and Bill Gates, among others. (https://trytwello.com/ceos-that-meditate/)
Similarly you could find people (like Arnold Schwarzenegger, if I recall?) citing that adopting a more mature, stoic mindset about life was helpful to them -- Ray Dalio has this whole series of videos on "life pri...
To compare to the obvious alternative, is the evidence for meditation stronger than the evidence for prayer? I assume there are also some religious billionaires and other successful people who would attribute their success to praying every day or something like that.
It feels sorta understandable to me (albeit frustrating) that OpenPhil faces these assorted political constraints. In my view this seems to create a big unfilled niche in the rationalist ecosystem: a new, more right-coded, EA-adjacent funding organization could optimize itself for being able to enter many of those blacklisted areas with enthusiasm.
If I was a billionare, I would love to put together a kind of "completion portfolio" to complement some of OP's work. Rationality community building, macrostrategy stuff, AI-related advocacy to try an...
There are actually a number of ways that you might see a permanently stable totalitarian government arise, in addition to the simplest idea that maybe the leader never dies:
https://80000hours.org/problem-profiles/risks-of-stable-totalitarianism/
I and perhaps other LessWrongers would appreciate reading your review (of any length) of the book, since lots of us loved HPMOR, the Sequences, etc, but are collectively skeptical / on the fence about whether to dive into Project Lawful. (What's the best way to read the bizzare glowfic format? What are the main themes of the book and which did you like best? etc)
I loved Project Lawful/Planecrash (not sure which is the actual title), but I do hesitate to recommend it to others. Not everyone likes their medium-core S&M with a side of hardcore decision theory, or vice-versa. It is definitely weirder than HPMOR.
Something that threw me off at first: it takes the mechanics of the adapted setting very literally (e.g. spell slots and saving throws are non-metaphorical in-universe Things). That's not normal for (good) game fanfiction. The authors make it work anyway -- perhaps because clear rules make it easier to prod...
This is nice! I like seeing all the different subfields of research listed and compared; as a non-medical person I often just hear about one at a time in any given news story, which makes things confusing.
Some other things I hear about in longevity spaces:
- Senescent-cell-based theories and medicines -- what's up with these? This seems like something people were actually trying in humans; any progress, or is this a dud?
- Repurposing essentially random drugs that might have some effect on longevity -- most famously the diabetes drug metformin (a...
Fellow Thiel fans may be interested in this post of mine called "X-Risk, Anthropics, & Peter Thiel's Investment Thesis", analyzing Thiel's old essay "The Optimistic Thought Experiment", and trying to figure out how he thinks about the intersection of markets and existential risk.
"Americans eat more fats and oils, more sugars and sweets, more grains, and more red meat; all four items that grew the most in price since 2003."
Nice to know that you can eat healthy -- fish, veggies, beans/nuts, eggs, fresh fruit, etc -- and beat inflation at the same time! (Albeit these healthier foods still probably have a higher baseline price. But maybe not for much longer!)
The linked chart actually makes red meat look fine (beef has middling inflation, and pork has actually experienced deflation), but beverages, another generally unhealthy food, a...
A thoughtful post! I think about this kind of stuff a lot, and wonder what the implications are. If we're more pessimistic about saving lives in sub-saharan africa, should we:
Re: your point #2, there is another potential spiral where abstract concepts of "greatness" are increasingly defined in a hostile and negative way by partisans of slave morality. This might make it harder to have that "aspirational dialogue about what counts as greatness", as it gets increasingly difficult for ordinary people to even conceptualize a good version of greatness worth aspiring to. ("Why would I want to become an entrepeneur and found a company? Wouldn't that make me an evil big-corporation CEO, which has a whiff of the same f...
Feynman is imagining lots of components being made with "hand tools", in order to cut down on the amount of specialized machinery we need. So you'd want sophisticated manipulators to use the tools, move the components, clean up bits of waste, etc. Plus of course for gathering raw resources and navigating Canadian tundra. And you'd need video cameras for the system to look at what it's doing (otherwise you'd only have feed-forward controls in many situations, which would probably cause lots of cascading errors).
I don't know how big a rasberry pi would be if it had to be hand-assembled from transistors big enough to pick up individually. So maybe it's doable!
idk, you still have to fit video cameras and complex robotic arms and wifi equipment into that 1m^3 box, even if you are doing all the AI inference somewhere else! I have a much longer comment replying to the top-level post, where I try to analyze the concept of an autofac and what an optimized autofac design would really look like. Imagining a 100% self-contained design is a pretty cool intellectual exercise, but it's hard to imagine a situation where it doesn't make sense to import the most complex components from somewhere else (at least initially, until you can make computers that don't take up 90% of your manufacturing output).
This was a very interesting post. A few scattered thoughts, as I try to take a step back and take a big-picture economic view of this idea:
What is an autofac? It is a vastly simplified economy, in the hopes that enough simplification will unlock various big gains (like gains from "automation"). Let's interpolate between the existing global economy, and Feynman's proposed 1-meter cube. It's not true that "the smallest technological system capable of physical self-reproduction is the entire economy.", since I can imagine many potentia...
1950s era computers likely couldn't handle the complex AI tasks imagined here (doing image recognition; navigating rough Baffin Island terrain, finishing parts with hand tools, etc) without taking up much more than 1 meter cubed.
Socialism / communism is about equally abstract as Georgism, and it certainly inspired a lot of people to fight! Similarly, Republican campaigns to lower corporate tax rates, cut regulations, reduce entitlement spending, etc, are pretty abstract (and often actively unpopular when people do understand them!), but have achieved some notable victories over the years. Georgism is similar to YIMBYism, which has lots of victories these days, even though YIMBYism also suffers from being more abstract than conspiracy theories with obvious villains about people "...
Future readers of this post might be interested this other lesswrong post about the current state of multiplex gene editing: https://www.lesswrong.com/posts/oSy5vHvwSfnjmC7Tf/multiplex-gene-editing-where-are-we-now
Future readers of this blog post may be interested in this book-review entry at ACX, which is much more suspicious/wary/pessimistic about prion disease generally:
https://www.astralcodexten.com/p/your-book-review-the-family-that
What kinds of space resources are like "mice & cheese"? I am picturing civilizations expanding to new star systems mostly for the matter and energy (turn asteroids & planets into a dyson swarm of orbiting solar panels and supercomputers on which to run trillions of emulated minds, plus constructing new probes to send onwards to new star systems).
re: the Three Body Problem books -- I think the book series imagines that alien life is much, much more common (ie, many civilizations per galaxy) than Robin Hanson imagines in his Grabby Aliens hypot...
Yes, it does have to be fast IMO, but I think fast expansion (at least among civilizations that decide to expand much at all) is very likely.
Of course the first few starships that a civilization sends to colonize the nearest stars will probably not be going anywhere near the speed of light. (Unless it really is a paperclips-style superintelligence, perhaps.) But within a million years or so, even with relatively slow-moving ships, you have colonized thousands of solar systems, built dyson swarms around every star, have a total population in the...
I think part of the "calculus" being run by the AI safety folks is as follows:
there are certainly both some dumb ways humanity could die (ie, AI-enabled bioweapon terrorism that could have easily been prevented by some RLHF + basic checks at protein synthesis companies), as well as some very tricky, advanced ways (AI takeover by a superintelligence with a very subtle form of misalignment, using lots of brilliant deception, etc)
It seems like the dumber ways are generally more obvious / visible to other people (like military generals or the median vote
re: your comments on Fermi paradox -- if an alien super-civilization (or alien-killing AI) is expanding in all directions at close to the speed of light (which you might expect a superintelligence to do), then you mostly don't see them coming until it's nearly too late, since the civilization is expanding almost as fast as the light emitted by the civilization. So it might look like the universe is empty, even if there's actually a couple of civilizations racing right towards you!
There is some interesting cosmological evidence that we are in fact living i...
[spoilers for minor details of later chapters of the book] Isn't the book at least a little self-aware about this plot hole? If I recall correctly, the book eventually reveals (rot13 from here on out)...
gung gur fcnpr cebtenz cyna jnf rffragvnyyl qrfvtarq nf n CE fghag gb qvfgenpg/zbgvingr uhznavgl, juvpu vaqrrq unq ab erny cebfcrpg bs jbexvat (gbb srj crbcyr, gbb uneq gb qbqtr zrgrbef, rirelguvat arrqf gb or 100% erhfrq, rgp, yvxr lbh fnl). Juvyr gur fcnpr cebtenz jnf unccravat, bgure yrff-choyvp rssbegf (eryngrq gb ahpyrne fhoznevarf, qvttvat haqreteb...
In addition to the researchy implications for topics like deception and superpersuasion and so forth, I imagine that results like this (although, as you say, unsuprising in a technical sense) could have a huge impact on the public discussion of AI (paging @Holly_Elmore and @Joseph Miller?) -- the general public often seems to get very freaked out about privacy issues where others might learn their personal information, demographic characteristics, etc.
In fact, the way people react about privacy issues is so strong that it usually seems very overblown to me...
So, sure, there is a threshold effect in whether you get value from bike lanes on your complex journey from point A to point G. But other people throughout the city have different threshold effects:
Nice; Colorado recently passed a statewide law that finally does away with a similar "U+2" rule in my own town of Fort Collins (as well as other such rules in Boulder and elsewhere). To progress!
this is quality.
I don't understand this post, because it seems to be parodying Anthropic's Responsible Scaling Policies (ie, saying that the RSPs are not sufficient), but the analogy to nuclear power is confusing since IMO nuclear power has in fact been harmfully over-regulated, such that advocating for a "balanced, pragmatic approach to mitigating potential harms from nuclear power" does actually seem good, compared to the status quo where society hugely overreacted to the risks of nuclear power without properly taking a balanced view of the costs vs benefits.
Maybe you c...
I will definitely check out that youtube channel! I'm pretty interested in mechanism design and public-goods stuff, and I agree there are a lot of good ideas there. For instance, I am a huge fan of Georgism, so I definitely recognize that going all-in on the "libertarian individualist approach" is often not the right fit for the situation! Honestly, even though charter cities are somewhat an intrinsically libertarian concept, part of the reason I like the charter city idea is indeed the potential for experimenting with new ways to manage ...
Yup, there are definitely a lot of places (like 99+% of places, 99+% of the time!) which aren't interested in a given reform -- especially one as uniqely big and experimental as charter cities. This is why in our video we tried to focus on political tractability as one of the biggest difficulties -- hopefully we don't come across as saying that the world will instantly be tiled over with charter cities tomorrow! But some charter cities are happening sometimes in some places -- in addition to the examples in the video, Zambia is pretty friendly ...
Thanks, this is exciting and inspiring stuff to learn about!
I guess another thing I'm wondering about, is how we could tell apart genes that impact a trait via their ongoing metabolic activities (maybe metabolic is not the right term... what I mean is that the gene is being expressed, creating proteins, etc, on an ongoing basis), versus genes that impact a trait via being important for early embryonic / childhood development, but which aren't very relevant in adulthood. Genes related to intelligence, for instance, seem like they might show up with po...
Is there a plausible path towards gene therapies that edit dozens, hundreds, or thousands of different genes like this? I thought people were worried about off-target errors, etc? (Or at least problems like "you'll have to take 1000 different customized doses of CRISPR therapy, which will be expensive".) So my impression is that this kind of GWAS-inspired medicine would be most impactful with whole-genome synthesis? (Currently super-expensive?)
To be clear I agree with the main point this post is making about how we don't need animal models, etc, to do medicine if we have something that we know works!
(this comment is kind of a "i didn't have time to write you a short letter so I wrote you a long one" situation)
re: Infowar between great powers -- the view that China+Russia+USA invest a lot of efforts into infowar, but mostly "defensively" / mostly trying to shape domestic opinion, makes sense. (After all, it must be easier to control the domestic media/information lansdscape!) I would tend to expect that doing domestically-focused infowar stuff at a massive scale would be harder for the USA to pull off (wouldn't it be leaked? wouldn't it be ...
(Copies from EA Forum for the benefit of lesswrongers following the discussion here)
Definitely agree that empathy and other social feelings provide indirect evidence for self-awareness (ie, "modeling stuff about yourself" in your brain) in a way that optimism/pessimism or pain-avoidance doesn't. (Although wouldn't a sophisticated-enough RL circuit, interacting with other RL circuits in some kind of virtual evolutionary landscape, also develop social emotions like loyalty, empathy, etc? Even tiny mammals like mice/rats display sophisticated soci...
Why would showing that fish "feel empathy" prove that they have inner subjective experience? It seems perfectly possible to build a totally mechanical, non-conscious system that nevertheless displays signs of empathy. Couldn't fish just have some kind of built-in, not-necessarily-conscious instinct to protect other fish (for instance, by swimming together in a large school) in order to obtain some evolutionary benefit?
Conversely, isn't it possible for fish to have inner subjective experience but not feel empathy? Fish are very simple crea...
Hi Trevor! I appreciate this thread of related ideas that you have been developing about intelligence agencies, AI-augmented persuasion techniques, social media, etc.
Thanks for writing this post, I 100% share your sentiment and appreciate the depth with which you've explored this topic, including some of the political considerations.
Here are some other potentially-relevant case studies of people doing similar-ish things, trying to make the world a better place while navigating touchy political fears related to biotech:
Yeah, I am interested in this from the "about to have an infant" perspective (my wife is almost 20 weeks pregnant). Interestingly this means she will be able to get both the flu, covid, and newly-approved RSV shot.
Good point that rationalism is over-emphasizing the importance of Bayes theorem in a pretty ridiculous way, even if most of the individual statements about Bayes theorem are perfectly correct. I feel like if one was trying to evaluate Eliezer or the rationalist community on some kind of overall philosophy scorecard, there would be a lot of situations like this -- both "the salience is totally out of whack here even though it's not technically /wrong/...", and "this seems like a really important and true sentiment, but it's not really the kind of thin...
Some other potentially controversial views that a philosopher might be able to fact-check Eliezer on, based on skimming through an index of the sequences:
I suggest maybe re-titling this post to:
"I strongly disagree with Eliezer Yudkowsky about the philosophy of consciousness and decision theory, and so do lots of other academic philosophers"
or maybe:
"Eliezer Yudkowsky is Frequently, Confidently, Egregiously Wrong, About Metaphysics"
or consider:
"Eliezer's ideas about Zombies, Decision Theory, and Animal Consciousness, seem crazy"
Otherwise it seems pretty misleading / clickbaity (and indeed overconfident) to extrapolate from these beliefs, to other notable beliefs of Eliezer's -- such as cryonics, quantum mec...
This video is widely believed to be a CGI fake.
I think "Why The West Rules", by Ian Morris, has a pretty informative take on this. The impression I got from the book was that gradually accruing technologies/knowledge, like the stuff you mention, is slowly accruing in the background amid the ups and downs of history, and during each peak of civilizational complexity (most notably the Roman empire, and the medieval-era Song Dynasty in china, and then industrial-era Britain) humanity basically gets another shot-on-goal to potentially industrialize.
Britain had a couple of lucky breaks -- cheap and ab...
Thanks! Apparently I am in a mood to write very long comments today, so if you like, you can see some thoughts about addressing potential objections / difficulties in a response I made to a comment on the EA Forum version of this post.
Thanks for catching that about Singaporeans!
Re: democracy, yeah, we debated how exactly to phrase this. People were definitely aware of the democracies of ancient Greece and Rome, and democracy was sometimes used on a local level in some countries, and there were sometimes situations where the nobles of a country had some sway / constraints over the king (like with the Magna Carta). But the idea of really running an entire large country on American-style democracy seems like it was a pretty big step and must've seemed a bit crazy at the time......
Probably the charter city with the most publicity is Prospera, so you could do stuff like:
I think one problem with this concept is that the "restrictions" might turn out to be very onerous, preventing the good guys (using "restrictions) from winning a complete unilateral victory over everyone else. One of the major anticipated benefits of superhuman AI systems is the ability to work effectively even on vague, broad, difficult tasks that span multiple different domains. If you are committed to creating a totally air-gapped high-security system, where you only hand your AI the "smallest subdividable subtask" and only giving your AI ac...
[Cross-posting my comment from the EA Forum]
This post felt vague and confusing to me. What is meant by a "game board" -- are you referring to the world's geopolitical situation, or the governance structure of the United States, or the social dynamics of elites like politicians and researchers, or some kind of ethereum-esque crypto protocol, or internal company policies at Google and Microsoft, or US AI regulations, or what?
How do we get a "new board"? No matter what kind of change you want, you will have to get there starting from the current s...
Note that this story has recieved a beautifully-animated video adaptation by Rational Animations!