This is a special post for quick takes by samuelshadrach. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
Has anyone on lesswrong thought about starting a SecureDrop server?
For example to protect whistleblowers of ASI orgs.
1In 2021, Daniel Ellsberg leaked US govt plans to make a nuclear first strike on China in 1958 due to Taiwan conflict.
Daniel Ellsberg copied these papers more than 50 years ago but only released them now because he thought another conflict over Taiwan may be possible soon.
Thought it might be interesting to share.
-1
As usual, seems clear Dulles was more interested in escalating the conflict (in this case, to nuclear) than Eisenhower.
Scrape of many lesswrong blogs
(I did not put much effort in this, and am unlikely to fix errors. Please fork the list if you want something higher quality. Used only public data to make this.)
https://www.astralcodexten.com
https://www.mccaughan.org.uk/g/
https://muckrack.com/thane-ruthenis/articles
1
Update: Many of these have now been added to https://searchmysite.net/search/browse/
Lesswrong is clearly no longer the right forum for me to get engagement on topics of my interest. Seems mostly focussed on AI risk.
On which forums do people who grew up on the cypherpunks mailing list hang out today? Apart from cryptocurrency space.
2
There is still the possibility on the front page to filter out the AI tag completely.
3
Yes but then it becomes a forum within a forum kinda thing. You need a critical mass of users who all agree to filter out the AI tag, and not have to preface their every post with "I dont buy your short timelines worldview, I am here to discuss something different".
Building critical mass is difficult unless the forum is conducive to it. There's is ultimately only one upvote button and one front-page so the forum will get taken over by the top few topics that its members are paying attention to.
I don't think there's anything wrong with a forum that's mostly focussed on AI xrisk and transhumanist stuff. Better to do one thing well than half ass ten things. But it also means I may need to go elsewhere.
3
Yeah. I proposed a while ago that all the AI content was becoming so dominant that it should be hived off to the Alignment Forum while LessWrong is for all the rest. This was rejected.
I've been informed that getting approval of 10-20 people on lesswrong for a project is a good way of getting funding from the bigger EA funders.
Can I just pay $50 each to 20 people for 0.5 hours of their time? Has this been tried?
(Ofcourse they can choose to disapprove the project, and accept the payment regardless.)
5
I suspect it matters quite a bit WHICH 10-20 people support the project. And I further suspect that the right people are unlikely to sell their time in this way.
That's not to say that it's a bad idea to offer to compensate people for the time and risk of analyzing your proposal. But you should probably filter for pre-interest with a brief synopsis of what you actually want to do (not "get funding", but "perform this project that does ...").
1
This makes sense. Paying someone to do more of something they care about is very different from paying someone to do something they don't otherwise want to do. That would include providing regular feedback.
3
Please post your hourly rate either here or even better, on your personal website. It'll make it easier to contact people for this.
3
Related: Beta Readers by Holden Karnofsky
-3
I suppose that philanthropy funds mainly driven by public relations and their managers are looking for projects that would look good on their portfolio. Your idea may buy your way to the attention of the said managers, but they will decide based on comparison with similar projects that got maybe more viral and may seem more public-appealing.
If your project is simple enough for a non-specialist to evaluate its worthyness in 30 minutes, than perhaps the best course of action is to seek for more appealing presentations of your ideas that would catch the attention and make it viral. Then it will work for fund managers as well.
1
I don't think this is a good description of how big EA funders operate? This might be true for other orgs. Do you have any data to back this up?
0
Not really, apart from the absense of feedback on my proposal.
I think it is a universal thing. Imagine the works of such a fund and you are a manager peeking proposals from a big flow. Even if you personally feel that some proposal is cool, but it doesn't have public support and you feel that others won't support it. Will you be heavily pushing it forward when you have to review dozens of other proposals? If you do, won't it look like you are somehow affiliated with the project?
So, you either look for private money, or seek public support, I see no other way.
1
I think EA funders are more willing than many other non-profit funders to be the first person to fund your org, without anyone else supporting.
P.S. I didn't downvote your comments.
1
That's just me trying to analyze why it doesn't work. The lack of feedback is really frustrating. I would rather prefer insults to the silence.
1
I feel same.
Ban on ASI > Open source ASI > Closed source ASI
This is my ordering.
Yudkowsky's worldview in favour of closed source ASI is sitting in multiple shaky assumptions. One of these assumptions is that getting a 3-month to 3-year lead is necessary and sufficient condition for alignment to be solved. Yudkowsky!2025 himself doesn't believe alignment can be solved in 3 years.
Why does anybody on lesswrong want closed source ASI?
Is there a single person on Earth in the intersection of these?
- received $1M funding
- non-profit
- public advocacy
- AI xrisk
My general sense is that most EA / rationalist funders avoid public advocacy. Am I missing anything?
Polymarket is not liquid enough to justify betting full-time.
Optimistically I expect if I invested $5k and 4 days per month for 6 months, I could make $7k +- $2k expected returns. Or $0-4k profit. I would have to split up the $5k into 6-7 bets and monitor them all separately.
I could probably make similar just working at a tech job.
Does anyone have a good solution to avoid the self-fulfilling effect of making predictions?
Making predictions often means constructing new possibilities from existing ideas, drawing more attention to these possibilities, creating common knowledge of said possibilities, and inspiring people to work towards these possibilities.
One partial solution I can think of so far is to straight up refuse to talk about visions of the future you don't want to see happen.
Search engine for books
http://booksearch.samuelshadrach.com
Aimed at researchers
Technical details (you can skip this if you want):
Dataset size: libgen 65 TB, (of which) unique english epubs 6 TB, (of which) plaintext 300 GB, (from which) embeddings 2 TB, (hosted on) 256+32 GB CPU RAM
Did not do LLM inference after embedding search step because human researchers are still smarter than LLMs as of 2025-03. This tool is meant for increasing quality for deep research, not for saving research time.
Main difficulty faced during project - disk throughput is a b...
4
1
use http not https
2
Okay, that works in Firefox if I change it manually. Though the server seems to be configured to automatically redirect to HTTPS. Chrome doesn't let me switch to HTTP.
1
Thanks for your patience. I'd be happy to receive any feedback. Negative feedback especially.
2
I see you fixed the https issue. I think the resulting text snippets are reasonably related to the input question, though not overly so. Google search often answers questions more directly with quotes (from websites, not from books), though that may be too ambitious to match for a small project. Other than that, the first column could be improved with relevant metadata such as the source title. Perhaps the snippets in the second column could be trimmed to whole sentences if it doesn't impact the snippet length too much. In general, I believe snippets currently do not show line breaks present in the source.
1
Thanks for feedback.
I’ll probably do the title and trim the snippets.
One way of getting a quote would to be to do LLM inference and generate it from the text chunk. Would this help?
2
I think not, because in my test the snippet didn't really contain such a quote that would have answered the question directly.
1
Can you send the query? Also can you try typing the query twice into the textbox? I'm using openai text-embedding-3-small, which seems to sometimes work better if you type the query twice. Another thing you can try is retry the query every 30 minutes. I'm cycling subsets of the data every 30 minutes as I can't afford to host the entire data at once.
2
I think my previous questions were just too hard, it does work okay on simpler questions. Though then another question is whether text embeddings improve over keyword search or just an LLMs. They seem to be some middle ground between Google and ChatGPT.
Regarding data subsets: Recently there were some announcements of more efficient embedding models. Though I don't know what the relevant parameters here are vs that OpenAI embedding model.
3
Cool!
Useful information that you’d still prefer using ChatGPT over this. Is that true even when you’re looking for book recommendations specifically? If so yeah that means I failed at my goal tbh. Just wanna know.
Since Im spending my personal funds I can’t afford to use the best embeddings on this dataset. For example text-embedding-3-large is ~7x more expensive for generating embeddings and is slightly better quality.
The other cost is hosting cost, for which I don’t see major differences between the models. OpenAI gives 1536 float32 dims per 1000 char chunk so around 6 KB embeddings per 1 KB plaintext. All the other models are roughly the same. I could put in some effort and quantise the embeddings, will update if I do it.
3
I think in some cases an embedding approach produces better results than either a LLM or a simple keyword search, but I'm not sure how often. For a keyword search you have to know the "relevant" keywords in advance, whereas embeddings are a bit more forgiving. Though not as forgiving as LLMs. Which on the other hand can't give you the sources and they may make things up, especially on information that doesn't occur very often in the source data.
1
Got it. As of today a common setup is to let the LLM query an embedding database multiple times (or let it do Google searches, which probably has an embedding database as a significant component).
Self-learning seems like a missing piece. Once the LLM gets some content from the embedding database, performs some reasoning and reaches a novel conclusion, there’s no way to preserve this novel conclusion longterm.
When smart humans use Google we also keep updating our own beliefs in response to our searches.
P.S. I chose not to build the whole LLM + embedding search setup because I intended this tool for deep research rather than quick queries. For deep research I’m assuming it’s still better for the human researcher to go read all the original sources and spend time thinking about them. Am I right?
1
Update: HTTPS should work now
Human genetic engineering targetting IQ as proposed by GeneSmith is likely to lead to an arms race between competing individuals and groups (such as nation states).
- Arms races can destabilise existing power balances such as nuclear MAD
- Which traits people choose to genetically engineer in offspring may depend on what's good for winning the race rather than what's long-term optimal in any sense.
- If maintaining lead time against your opponent matters, there are incentives to bribe, persuade or even coerce people to bring genetically edit...
9
If you convince your enemies that IQ is a myth, they won't be concerned about your genetically engineered high IQ babies.
1
Superhumans that are actually better than you at making money will eventually be obvious. Yes, there may be some lead time obtainable before everyone understands, but I expect it will only be a few years at maximum.
8
Standard objection: Genetic engineering takes a lot of time till it has any effect. A baby doesn't develop into an adult over night. So it will almost certainly not matter relative to the rapid pace of AI development.
1
I agree my point is less important if we get ASI by 2030, compared to if we don’t get ASI.
That being said, the arms race can develop over the timespan of years not decades. 6-year superhumans will prompt people to create the next generation of superhumans, and within 10-15 years we will have children from multiple generations where the younger generation have edits with stronger effect sizes. Once we can see the effects on these multiple generations, people might go at max pace.
4
PSA
Popularising human genetic engineering is also by default going to popularise lots of neighbouring ideas, not just the idea itself. If you are attracting attention to this idea, it may be useful for you to be aware of this.
The example of this that has already played out is popularising "ASI is dangerous" also popularises "ASI is powerful hence we should build it".
1
P.S. Also we don't know the end state of this race. +5 SD humans aren't necessarily the peak, it's possible these humans further do research on more edits.
This is unlikely to be careful controlled experiment and is more likely to be nation states moving at maximum pace to produce more babies so that they control more of the world when a new equilibrium is reached. And we don't know when if ever this equilibrium will be hit.
All the succeeding paths to superintelligence seem causally downstream of Moore's law:
- AI research - which is accelerated by Moore's law as per scaling laws
- Human genetic engineering - which is accelerated by next generation sequencing and nanopore sequencing, which is accelerated by circuit miniaturisation, which is accelerated by Moore's law
- Human brain connectome research - which is accelerated by fruitfly connectome, which is accelerated by electron microscopy, which is accelerated by Moore's law
Succeeding path to cheap energy also follows same:
- So
1
Do you mean Moore’s law in the literal sense of transistors on a chip, or something more general like “hardware always gets more efficient”?
I’m mentioning this because much of what I’ve been hearing in the past few years w.r.t Moore’s law has been “Moore’s law is dead.”
And, assuming you’re not referring to the transistor thing: what is your more specific Moore’s Law definition? Any specific scaling law, or maybe scaling laws specific to each of the examples you posted?
1
I mean R&D of packing more transistors on a chip, and the casually downstream stuff such as R&D of miniaturisation of detectors, transducers, diodes, amplifiers etc
More lesswrong AI debates happening on youtube instead of lesswrong would be nice.
I have a hypothesis that Stephen Krashen's comprehensible input stuff applies not just to learning new languages, but also new professions and new cultures. Video is better than text for that.
Has anyone tried duncan sabien's colour wheel thing?
https://homosabiens.substack.com/p/the-mtg-color-wheel
My colours: Red, followed by Blue, followed by Black
I don't know, I also see them as the traits needed in different stages of a movement.
- Red = freedom from previous movement.
- Blue = figuring out how to organise a new movement.
- Black = executing the movement to its intended ends.
2
I had a white-blue upbringing and a blue-green career (see below); my hobbies are black-green-white (compost makes my garden grow to feed our community); my vices are green-red; and my politics are five-color (at least to me).
Almost all of my professional career has been in sysadmin and SRE roles: which is tech (blue) but cares about keeping things reliable and sustainable (green) rather than pursuing novelty (red). Within tech's blue, it seems to me that developer roles run blue-red (build the exciting new feature!); management roles run blue-white (what orderly social rules will enable intelligence?); and venture capital runs blue-black (how do we get cleverness to make money for us?); while SRE and similar roles run blue-green.
My garden runs on blood, bones, rot, and worm poop: green-black Golgari territory, digesting the unwanted to feed the wanted. I stick my hands in the dirt to feel the mycorrhizae. But the point of the garden is to share food with others (green-white) because it makes me feel good (black-red). I'm actually kinda terrible at applying blue to my hobbies, and should really collect some soil samples for lab testing one of these days.
1
It’s not yet good enough to find zero days all by itself but it may be of some use in pentesting, static analysis or reverse engineering.
3
O3 is famously good enough to find zero days by itself.
1
Thanks! Any forums you’d recommend for more on this?
2
There was a good discussion on hacker news, and there was quite a bit of posting of highly variable quality on xitter.
1
Thanks! I saw the hackernews post and I avoid twitter for mental health reasons. I should find some solution for the latter.
2
FWIW I did not see any high-valur points made on Twitter that were not also made on HN.
Oh, one more source for that one though - there was some coverage on the Complex Systems podcast - the section titled "AI's impact on reverse engineering" (transcript available at that URL).
Any human with above room temperature IQ can design a utopia. The reason our current system isn’t a utopia is that it wasn’t designed by humans. Just as you can look at an arid terrain and determine what shape a river will one day take by assuming water will obey gravity, so you can look at a civilization and determine what shape its institutions will one day take by assuming people will obey incentives.
Possible corrolary: If you have any slack in the system, use it to dig a new canal, not endlessly row your boat upriver. Non-profit capital is an example of slack.
1
Also: Use your slack to call up that which you cannot put down. That's how you know you've dug a canal.
1
Wait I realised I no longer believe this.
This seems interesting and worth writing more on. Maybe later.
1
I'm very interested in hearing counterarguments. I have not put a lot of thought into it.
2025-05-12
Samuel x Saksham AI timelines (discussion on 2025-05-09)
- top-level views
- samuel top-level: 25% AI!2030 >= ASI, >50% ASI >> AI!2030 >> AI!2025, <25% AI!2030 ~= AI!2025
- saksham top-level: medium probability AI!2030 >= ASI
- samuel bullish on model scaling, more uncertain on RL scaling
- saksham bullish on RL/inference scaling, saksham bullish on grokking
- samuel: does bullish on grokking mean bullish on model scaling. saksham: unsure
- agreements
- samuel and saksham agree: only 2024-2025 counts as empirical data to extrapol
Anyone has a GPT2 fine-tuned API?
I might wanna ship an app comparing GPT2, GPT3.5 and o3, to explain scaling laws to non-technical folks.
1
Update: I figured it out and hosted it. Clear difference in capabilities visible.
I need atleast $100/mo to host 24x7 though.
TGI makes it trivial.
Can host openai-community/gpt2 (125M, 2019), EleutherAI/gpt-neox-20b (20B, 2022), gpt-3.5-turbo (175B?, 2020) and o3 (2T?, 2025).
If you support an international ban on building ASI, please consider making a short video stating this.
A low quality video recording made in 15 minutes is better than no video at all. Consider doing it right now if you are convinced.
Optional:
- make a long video instead of a short one, explaining your reasoning
- make videos on other topics to increase viewership
Here's mine: https://youtube.com/shorts/T40AeAbGIcg?si=OFCuD37Twyivy-oa
Why?
- Video has orders of magnitude more reach than text. Most people on earth don't have the attention span for lengthy text p
5
It could also be worse than no video all, if it gives people negative associations around the whole concept.
1
In theory, yes.
In practice, I think bad publicity is still publicity. Most people on earth still haven't heard about xrisk. I trust that sharing the truth has hard-to-predict positive effects over long time horizons even if not over short. I think average LW user is too risk-averse relative to the problem they wish to solve.
I'd love to hear your reasoning for why making a video is bad. But I do vaguely suspect this disagreement comes down to some deeper priors of how the world works and hence may not get resolved quickly.
2
I didn't say that making a video would always be bad! I agree that if the median person reading your comment would make a video, it would probably be good. I only disputed the claim that making a video would always be good.
1
Oh, cool
Do you have a clear example of a blunder someone should not make when making such a video?
Obviously you can't forecast all the effects of making a video, there could be some probability mass of negative outcome while the mean and median are clearly positive.
2
Suppose Echo Example's video says, "If ASI is developed, it's going to be like in The Terminator - it wakes up to its existence, realizes it's more intelligent than humans, and then does what more intelligent species do to weaker ones. Destroys and subjugates them, just like humans do to other species!"
Now Vee Viewer watches this and thinks "okay, the argument is that the ASIs would be a more intelligent 'species' than humans, and more intelligent species always want to destroy and subjugate weaker ones".
Having gotten curious about the topic, Vee mentions this to their friends, and someone points them to Yann LeCun claiming that people imagine killer robots because people fail to imagine that we could just build an AI without the harmful human drives. Vee also runs into Steven Pinker arguing that history "does turn up the occasional megalomaniacal despot or psychopathic serial killer, but these are products of a history of natural selection shaping testosterone-sensitive circuits in a certain species of primate, not an inevitable feature of intelligent systems".
So then Vee concludes that oh, that thing about ASI's risks was just coming from a position of anthropomorphism and people not really understanding that AIs are different from humans. They put the thought out of their head.
Then some later time Vee runs into Denny Diligent's carefully argued blog post about the dangers of ASI. The beginning reads: "In this post, I argue that we need a global ban on developing ASI. I draw on the notion of convergent instrumental goals, which holds that all sufficiently intelligent agents have goals such as self-preservation and acquiring resources..."
At this point, Vee goes "oh, this is again just another version of the Terminator argument, LeCun and Pinker have already disproven that", closes the tab, and goes do something else. Later Vee happens to have a conversation with their friend, Ash Acquaintance.
Ash: "Hey Vee, I ran into some people worried about artifici
1
Thanks for reply.
If you are making a video, I agree it's not a good idea to put weaker arguments there if you know stronger arguments.
I strongly disagree with the idea that therefore you should defer to EA / LW leadership (or generally, anyone with more capital/attention/time), and either not publish your own argument or publish their argument instead of yours. If you think an argument is good and other people think it's bad, I'd say post it.
2
I also strongly disagree with that idea.
Anyone on lesswrong writing about solar prices?
Electricity from coal and crude oil has stagnated at $0.10/kWh for over 50 years, meaning the primary way of increasing your country's per capita energy use reserve is to trade/war/bully other countries into giving you their crude oil.
Solar electricity is already at $0.05/kWh and is forecasted to go as low as $0.02/kWh by 2030.
If a new AI model comes out that's better than the previous one and it doesn't shorten your timelines, that likely means either your current or your previous timelines were inaccurate.
1
Here's a simplified example for people who have never traded in the stock market. We have a biased coin with 80% probability of heads. What's the probability of tossing 3 coins and getting 3 heads? 51.2%. Assuming first coin was heads, what's the probability of getting other two coins also heads? 64%
Each coin toss is analogous to whether the next model follows or does not follow scaling laws.
2
With coin, the options are "head" and "tails", so "head" moves you in one direction.
With LLMs, the options are "worse than expected", "just as expected", "better than expected", so "just as expected" does not have to move you in a specific direction.
1
I made a reply. You're referring to situation b.
2
I don't think this analogy works on multiple levels. As far as I know, there isn't some sort of known probability that scaling laws will continue to be followed as new models are released. While it is true that a new model continuing to follow scaling laws is increased evidence in favor of future models continuing to follow scaling laws, thus shortening timelines, it's not really clear how much evidence it would be.
This is important because, unlike a coin flip, there are a lot of other details regarding a new model release that could plausibly affect someone's timelines. A model's capabilities are complex, human reactions to them likely more so, and that isn't covered in a yes/no description of if it's better than the previous one or follows scaling laws.
Also, following your analogy would differ from the original comment since it moves to whether the new AI model follows scaling laws instead of just whether the new AI model is better than the previous one (It seems to me that there could be a model that is better than the previous one yet still markedly underperforms compared to what would be expected from scaling laws).
If there's any obvious mistakes I'm making here I'd love to know, I'm still pretty new to the space.
1
I've made a reply formalising this.
1
Update based on the replies:
I basically see this as a Markov process.
X(t+1) = P(x(t+1) | x(t), x(t-1), x(t-2), ...) = F(x(t))
where x(t) is a value is sampled from X(t) distribution for all t.
In plain English, given the last value you get a probability distribution for the next value.
In the AI example: Given x(2025), estimate probability distribution X(2030) where x is the AI capability level.
Possibilities
a) x(t+1) value is determined by x(t) value. There is no randomness. No new information is learned from x(t).
b) X(t+1) distribution is conditional on the value of x(t). Learning which value x(t) was sampled from distribution X(t) distribution gives you new information. However you sampled one of those values such that P(x(t+1) | x(t-1), x(t-2), ...) = P(x(t+1) | x(t), x(t-2) ). You got lucky, and the value sampled ensures distribution remains the same.
c) You learned new information and the probability distribution also changed.
a is possible but seems to imply overconfidence to me.
b is possible but seems to imply extraordianry luck to me, especially if it's happening multiple times.
c seems like the most likely situation to me.
1
Another way of operationalizing the objections to your argument are: what is the analogue to the event "flips heads"? If the predicate used is "conditional on AI models achieving power level X, what is the probability of Y event?" and the new model is below level X, by construction we have gained 0 bits of information about this.
Obviously this example is a little contrived, but not that contrived, and trying to figure out what fair predicates are to register will result in more objections to your original statement.
1
I've made a reply formalising this.
Suppose you are trying to figure out a function f(x,y,z | a,b,c) where x, y ,z are all scalar values and a, b, c are all constants.
If you knew a few zeroes of this function, you could figure out good approximations of this function. Let's say you knew
U(x,y, a=0) = x
U(x,y, a=1) = x
U(x,y, a=2) = y
U(x,y, a=3) = y
You could now guess U(x,y) = x if a<1.5, y if a>1.5
You will not be able to get a good approximation if you did not know enough zeroes.
This is a comment about morality. x, y, z are agent's multiple possibly-conflicting values and a, b, c ar...
My current guess for least worst path of ASI development that's not crazy unrealistic:
open source development + complete surveillance of all citizens and all elites (everyone's cameras broadcast to the public) + two tier voting.
Two tier voting:
- countries's govts vote or otherwise agree at global level on a daily basis what the rate of AI progress should be and which types of AI usage are allowed. (This rate can be zero.)
- All democratic countries use daily internet voting (liquid democracy) to decide what stance to represent at the global level. All other
2
Certainly at a human level this is unrealistic. In a way it's also overkill - if use of an AI is an essential step towards doing anything dangerous, the "surveillance" can just be of what AIs are doing or thinking.
This assumes that you can tell whether an AI input or output is dangerous. But the same thing applies to video surveillance - if you can't tell whether a person is brewing something harmless or harmful, having a video camera in their kitchen is no use.
At a posthuman level, mere video surveillance actually does not go far enough, again because a smart deceiver can carry out their dastardly plots in a way that isn't evident until it's too late. For a transhuman civilization that has values to preserve, I see no alternative to enforcing that every entity above a certain level of intelligence (basically, smart enough to be dangerous) is also internally aligned, so that there is no disposition to hatch dastardly plots in the first place.
This may sound totalitarian, but it's not that different to what humanity attempts to instill in the course of raising children and via education and culture. We have law to deter and punish transgressors, but we also have these developmental feedbacks that are intended to create moral, responsible adults that don't have such inclinations, or that at least restrain themselves.
In a civilization where it is theoretically possible to create a mind with any set of dispositions at all, from paperclip maximizer to rationalist bodhisattva, the "developmental feedbacks" need to extend more deeply into the processes that design and create possible minds, than they do in a merely human civilization.
I'm currently vaguely considering working on a distributed version of wikileaks that reduces personal risk for all people involved.
If successful, it will forcibly bring to the public a lot of information about deep tech orgs like OpenAI, Anthropic or Neuralink. This could, for example, make this a top-3 US election issue if most of the general public decides they don't trust these organisations as a result of the leaked information.
Key uncertainty for me:
- Destroying all the low trust institutions (and providing distributed tools to keep destroying them) i
3
I have partial ideas on the question of "how to build world govt"? [1]
But in general yeah I still lack a lot of clarity on how high trust political institutions are actually built.
"Trust" and "attention" seem like the key themes that come up whenever I think about this. Aggregate attention towards common goal then empower a trustworthy structure to pursue that goal.
----------------------------------------
1. For example build decentralised social media stack so people can form consensus on political questions even if there is violence being used to suppress it. Have laws and culture in favour of live-streaming leader's lives. Multi-party not two-party system will help. Ensuring weapons are distributed geographically and federally will help. (Distributing bioweapons is more difficult than distributing guns.) ↩︎
IMO a good way to explain how LLMs work to a layman is to print the weights on sheets of paper and compute a forward pass by hand. Anyone wanna shoot this video and post it on youtube?
Assuming humans can do one multiplication 4bit per second using a lookup table,
1.5B 4bit weights => ~1.5B calculations => 1.5B seconds = 47.5 years (working 24x7) = 133 years (working 60 hours/week)
So you'll need to hire ~100 people for 1 year.
You don't actually have to run the entire experiment for people to get the concept, just run a small fraction of it. Although it'll be cool to run the whole thing as well.
Update: HTTPS issue fixed. Should work now.
Books Search for Researchers
Project idea for you
Figure out why don't we build one city with one billion population
- Bigger cities will probably accelerate tech progress, and other types of progress, as people are not forced to choose between their existing relationships and the place best for their career
- Assume end-to-end travel time must be below 2 hours for people to get benefits of living in the same city. Seems achievable via intra-city (not inter-city) bullet-train network. Max population = (200 km/h * 2h)^2 * (10000 people/km^2) = 1.6 billion people
&...
2
Vr might be cheaper
2
I agree VR might be one-day be able to do this (make online meetings as good as in-person ones). As of 2025, bullet trains are more proven tech than VR. I'd be happy if both were investigated in more depth.
1
A few notes on massive cities:
Cities of 10Ms exist, there is always some difficulty in scaling, but scaling 1.5-2 OOMs doesn't seem like it would be impossible to figure out if particularly motivated.
China and other countries have built large cities and then failed to populate them
The max population you wrote (1.6B) is bigger than china, bigger than Africa, similar to both American Continents plus Europe .
Which is part of why no one really wants to build something so big, especially not at once.
Everything is opportunity cost, and the question of alternate routes matters alot in deciding to pursue something. Throwing everything and the kitchen sink at something costs a lot of resources.
Given that VR development is currently underway regardless, starting this resource intense project which may be made obsolete by the time it's done is an expected waste of resources. If VR hit a real wall that might change things (though see above).
If this giga-city would be expected to 1000x tech progress or something crazy then sure, waste some resources to make extra sure it happens sooner rather than later.
Tl;dr:
Probably wouldn't work, there's no demand, very expensive, VR is being developed and would actually be able to say what you're hoping but even better
1
It could be built in stages. Like, build a certain number of bullet train stations at a time and wait to see if immigrants + real estate developers + corporations start building the city further, or do the stations end up unused?
I agree there is opportunity cost. It will help if I figure out the approx costs of train networks, water and sewage plumbing etc.
I agree there are higher risk higher reward opportunities out there, including VR. In my mind this proposal seemed relatively low risk so I figured it’s worth thinking through anyway.
This is demonstrably false. Honestly the very fact that city rents in many 1st world countries are much higher than rural rents proves that if you reduced the rents more people would migrate to the cities.
1
Lower/Higher risk and reward is the wrong frame.
Your proposal is high cost.
Building infrastructure is expensive. It may or may not be used, and even if used it may not be worthwhile.
R&D for VR is happening regardless, so 0 extra cost or risk.
Would you invest your own money into such a project?
"This is demonstrably false. Honestly the very fact that city rents in many 1st world countries are much higher than rural rents proves that if you reduced the rents more people would migrate to the cities."
Sure, there is marginal demand for living in cities in general. You could even argue that there is marginal demand to live in bigger vs smaller cities.
This doesn't change the equation: where are you getting one billion residents - all of Africa? There is no demand for a city of that size.
1
If I were a billionaire I might.
I also have (maybe minor, maybe not minor) differences of opinion with standard EA decision-making procedures of assigning capital across opportunities. I think this is where our crux actually is, not on whether giant cities can be built with reasonable amounts of funding.
And sorry I won’t be able to discuss that topic in detail further as it’s a different topic and will take a bunch of time and effort.
1
Our cruxes is whether the amount of investment to build one has a positive expected return on investment, breaking down into
1. If you could populate such a city
2. Whether this is a "try everything regardless of cost" issue, given that a replacent is being developed for other reasons.
I suggest focusing on 1, as it's pretty fundamental to your idea and easier to get traction on
1
1 is going to take a bunch of guesswork to estimate. Assuming it were possible to migrate to the US and live at $200/mo for example, how many people worldwide will be willing to accept that trade? You can run a survey or small scale experiment at best.
What can be done is expand cities to the point where no more new residents want to come in. You can expand the city in stages.
1
Definitely an interesting survey to run.
I don't think the US wants to triple the population with immigrants, and $200/month would require a massive subsidy. (Internet says $1557/month average rent in US)
How many people would you have to get in your city to justify the progress?
100 Million would only be half an order of magnitude larger than Tokyo, and you're unlikely to get enough people to fill it in the US (at nearly a third of the population, you'd need to take a lot of population from other cities)
How much do you have to subsidize living costs, and how much are you willing to subsidize?
0
If I understand correctly it is possible to find $300/mo/bedroom accommodation in rural US today, and a large enough city will compress city rents down to rural rents. A govt willing to pursue a plan as interesting as this one may also be able to increase immigrant labour to build the houses and relax housing regulations. US residential rents are artificially high compared to global average. (In some parts of the world, a few steel sheets (4 walls + roof) is sufficient to count as a house, even water and sewage piping in every house is not mandatory as long as residents can access toilets and water supply within walking distance.)
(A gigacity could also increase rents because it'll increase the incomes of even its lowest income members. But yeah in general now you need to track median incomes of 1B people to find out new equilibrium.)
1
That dichotomy is not exhaustive, and I believe going through with the proposal will necesarily make the city inhabitants worse off.
1. Humans' social machinery is not suited to live in such large cities, as of the current generations. Who to get acquainted with, in the first place? Isn't there lots of opportunity cost to any event?
2. Humans' biomachinery is not suited to live in such large cities. Being around lots and lots of people might be regulating hormones and behaviour to settings we have not totally explored (I remember reading something that claims this a large factor to lower fertility).
3. Centralization is dangerous because of possibly-handmade mass weapons.
4. Assuming random housing and examining some quirk/polar position, we'll get a noisy texture. It will almost certainly have a large group of people supporting one position right next to group thinking otherwise. Depending on sizes and civil law enforcement, that may not end well.
After a couple hundred years, 1) and 2) will most probably get solved by natural selection so the proposal will be much more feasible.
1
Sorry I didn’t understand your comment at all. Why are 1, 2 and 4 bigger problems in 1 billion population city versus say a 20 million population city?
1
I'd maintain that those problems already exist in 20M-people cities and will not necessarily become much worse. However, by increasing city population you bring in more people into the problems, which doesn't seem good.
2
Got it. I understood what you're trying to say. I agree living in cities has some downsides compared to living in smaller towns, and if you could find a way to get the best of both instead it could be better than either.
Has anyone considered video recording streets around offices of OpenAI, Deepmind, Anthropic? Can use CCTV or drone. I'm assuming there are some areas where recording is legal.
Can map out employee social graphs, daily schedules and daily emotional states.
4
Did you mean to imply something similar to the pizza index?
If so, I think it's a decent idea, but your phrasing may have been a bit unfortunate - I originally read it as a proposal to stalk AI lab employees.
2
Update: I'll be more specific. There's a power buys you distance from the crime phenomena going on if you're okay with using Google maps data acquired on about their restaurant takeout orders, but not okay asking the restaurant employee yourself or getting yourself hired at the restaurant.
2
Pizza index and stalking employees are both the same thing, it's hard to do one without the other. If you choose to declare war against AI labs you also likely accept that their foot soldiers are collateral damage.
I agree that (non-violent) stalking of employees is still a more hostile technique than writing angry posts on an internet forum.
Forum devs including lesswrong devs can consider implementing an "ACK" button on any comment, indicating I've read a comment. This is distinct from
a) Not replying - other person doesn't know if I've read their comment or not
b) Replying something trivial like "okay thanks" - other person gets a notification though I have nothing of value to say
3http://tokensfortokens.samuelshadrach.com
Pay for OpenAI API usage using cryptocurrency.
Currently supported: OpenAI o1 model, USDC on Optimism Rollup on ethereum.
Why use this?
- You want anonymity
- You want to use AI for cheaper than the rate OpenAI charges
How to use this?
- You have to purchase a few dollars of USDC and ETH on Optimism Rollup, and install Metamask browser extension. Then you can visit the website.
More info:
- o1 by OpenAI is the best AI model in the world as of Jan 2025. It is good for reasoning especially on problems ...
If I got $1M in funding, I'd use it towards some or all of the following projects.
The objective is to get secret information out of US ASI orgs (including classified information) and host it in countries outside the US. Hopefully someone else can use this info to influence US and world politics.
Black DAQ
- whistleblower/spy guide
- hacker guide
Grey DAQ
- internet doxxing tool
- drones/cctv outside offices/ datacentres
High attention
- persuade Indian, Russian, Chinese journalists to run a SecureDrop-like system
- digital journalism guide
- OR run a journalist outl
4
Why do you want to do this as a lone person rather than e.g. directly working with the intelligence service of some foreign adverary?
1
I think for a lot of societal change to happen, information needs to be public first. (Then it becomes common knowledge, then an alternate plan gets buy-in, then that becomes common knowledge and so on.)
A foreign adversary getting the info doesn't mean it's public, although it has increased the number of actors N who now have that piece of info in the world. Large N is not stable so eventually the info may end up public anyway.
Im selling $1000 tier5 OpenAI credits at discount. DM me if interested.
You can video call me and all my friends to reduce the probably I end up scamming you. Or vice versa I can video call your friends. We can do the transaction in tranches if we still can’t establish trust.
Update: HTTPS issue fixed. Should work now.
booksearch.samuelshadrach.com
Books Search for Researchers