All of lemonhope's Comments + Replies
I'm guessing you live in a country with a US military base? Are you more free than the average Chinese citizen?
I am unsure how free the average Chinese person is, nor how to weigh freedom of speech with certain economic freedoms and competent local government, low crime, the tendency of modern democracies to rent seek from the young in favour of the old, zoning laws, restriction on industrial development, a student loan system that seems to be a weird form of indenture. I do come from a country with rather strict hate speech laws. And we do not, in fact, have freedom of speech by any strict definition. And this is a policy American elites in and out of government s...
It deliberately doesn’t assume anything especially different or weird happens, only that trend lines keep going.
Of course they are fitting an exponential curve, and only one thing happens when you do that. (Newborn on track to swallow the sun by 2040.) You can get a hyperbolic curve to fit about equally as well [citation needed] and predict negative infinity resources on Jan 2 2028. I wish they had defended this choice a bit more clearly. Like plot binomial and sigmoid best fit for comparison, to show it really does look like an exponential. (Y axis can be...
Almost every time I use Claude Code (3.7 I think) it ends up cheating at the goal. Optimizing performance by replacing the API function with a constant, deleting test cases, ignoring runtime errors with silent try catch, etc. It never mentions these actions in the summary. In this narrow sense, 3.7 is the most misaligned model I have ever used.
I think Alibaba has not made any crazy developments yet. So let's consider DeepSeek. I think almost nobody had heard of DeepSeek before v3. Before v3, predicting strong AI progress in China would probably sound like "some AI lab in China will appear from nowhere and do something great. I don't know who or what or when or where, but it will happen soon." That was roughly my opinion, at least in my memory. Maybe making that kind of prediction does not match the tastes of people who are good at predicting things? Awfully vague claim to make I guess.
There was ...
So unbelievably convenient I don't even believe it
Could you do all the research on a boat in the ocean? Excuse the naive question.
Women/girls with big heads tend to hit their heads but you can solve that with bigger arms.
use of a genotyping pipeline poorly suited to ancient DNA which meant that 80% of the genetic variants they "analysed" were likely completely artefactual and did not exist.
Brutal!! I didn't know this gotcha existed. I hope there aren't too many papers silently gotch'd by it. Sounds like the type of error that could easily be widespread and unnoticed, if the statistical trace it leaves isn't always obvious.
To dumb it down a bit, here's my made up example: you get +1 IQ if your brain has surplus oxygen in the blood flowing through it. There's 1000 ways to get a bit more oxygen in there, but with +1000 oxygen, you still only get +1 IQ.
Is that the idea?
Good point! I didn't think that far ahead
I would vote to be ruled by their carbon children instead of their silicon children for certain
GeneSmith forgot to explicitly say that you can and should weight against sociopathy. Parents will be motivated to do this because if your kid is a jerk then your life will be miserable. (I do think if you select for success without selecting against sociopathy then you'll get lots of sociopaths.)
I would bet against some weird disease manifesting, especially if you are weighting for general health.
The wikipedia page picture has some evidence that you are right
1Someone please tell Altman and Musk they can spend their fortunes on millions of uber-genius children if they please, and they don't have to spend it all on their contest to replace ourselves with the steel & copper successors.
You could also make people grow up a bit faster. Some kids are more mature, bigger, etc than others at the same wall-clock age. If this doesn't conflict with lifespan then it would allow the superbabies to be productive sooner. Wouldn't want to rob someone of their childhood entirely, but 12 years of adolescence is long enough for lots of chase tag and wrestling.
What do you think is the ideal use-case for steering? Or is it not needed
What do you think of the cards held by TSMC and Samsung?
Skipping 8th grade is also a good option. And taking college courses during high school also good. I did both and finished BS 3 years early.
OK I'll bite. Memes and genes are obvious enough, but why is the rate of technological improvement proportional to the current technological level (or basically zero)? Don't ideas get harder to find?
Well Big Ideas do get harder to find, but if you make a 1% improvement to the US's steel production, then you get an extra 800,000 tons of steel. That doesn't help you think up new improvements but it does mean that the next 1% improvement will yield 808,000 tons.
Basically, any cost reduction or speedup or quality improvement is on top of what you have. How wou...
I have been using raindrop.io for my bookmarks for seven years or so and it is pretty good. Comments all have permalinks as you know.
in some sense that’s just hiring you for any other job, and of course if an AGI lab wants you, you end up with greater negotiating leverage at your old place, and could get a raise (depending on how tight capital constraints are, which, to be clear, in AI alignment are tight).
Could you say more about where the whole sequence is going / what motivated it? I am curious.
I believe I did explain/decompose the underlying mechanism
A simple way to see this: even a relatively crappy factory will make more goods in a month than it & its employees consume in a month. Likewise for farms, mines, fishing vessels, etc etc.
I could also have mentioned that it's relatively easy for two people to make three.
If someone prints money for themselves, they'll devalue their currency, but they won't be making factories less productive.
Intel makes more stuff than they use, no technical progress required.
I should've said "a dollar's worth of stuff can produce 1.03 dollar's worth of stuff". That would have been more clear.
Easy to say when you're already known by almost everyone in your world, have total career security, and have a full-sized family! I've never really done teaser links, but I can see why anyone would. You're more likely to gain some reputation or a job or a spouse if the reader goes to your website and sees your name there at the top.
Also, in terms of value to the reader: my life has changed in a big way because of a blog post I read two times that I can think of, but never from Twitter, despite spending more time reading Twitter than blogs by now. When I s...
No that should be one of the fastest and most cachable queries
I appreciate this post very much! What a great question you have found. Some old thoughts I had on this topic came back to me.
Personally I think of wealth as neither extractive nor hyper-efficient. I think of it as blind dumb compounding growth. A dollar has a lifespan of 1 year and bears 1.07 offspring in that time. A simple way to see this: even a relatively crappy factory will make more goods in a month than it & its employees consume in a month. Likewise for farms, mines, fishing vessels, etc etc. If nothing major goes wrong, money just grows, like...
I am glad to see somebody make the point properly. It's a weird state of affairs. We know the models can implement PoCs for CVEs better than most coders. We know the models can persuade people pretty effectively. Obviously the models can spread and change very easily. It's also easy for a rogue deployment to hide because datacenter GPUs draw 70W idle and update scripts constantly use tons of bandwidth. There's just no urgency to any of it.
How do you measure results?
If you wanted to take this idea to an absurd level, you could install a dropped ceiling made partially of furnace filters, and a grid of fans above it. Maybe have the outer perimeter of fans blowing up and the inner area blowing down, to try to get one large convection through the entire room.
How do you figure out the optimal filter thickness? If you hypothetically had a very weak fan then it wouldn't push much air through even furnace filters. If you had a magic constant air flow source then you would want the thickest filter possible.
I guess I am just wondering if you could use something better-looking and cheaper, like semi-transparent paper with lights behind it or a washable sheet/tapestry.
Have you heard of Big Ass Fans? It's a company that makes what you would expect. Do you think your ceiling fan filter could work with a 30ft fan?
What is the current popular (or ideally wise) wisdom wrt publishing demos of scary/spooky AI capabilities? I've heard the argument that moderately scary demos drive capability development into secrecy. Maybe it's just all in the details of who you show what when and what you say. But has someone written a good post about this question?
Einstein started doing research a few years before he actually had his miracle year. If he started at 26, he might have never found anything. He went to physics school at 17 or 18. You can't go to "AI safety school" at that age, but if you have funding then you can start learning on your own. It's harder to learn than (eg) learning to code, but not impossibly hard.
I am not opposed to funding 25 or 30 or 35 or 40 year olds, but I expect that the most successful people got started in their field (or a very similar one) as a teenager. I wouldn't expect funding an 18-year-old to pay off in less than 4 years. Sorry for being unclear on this in original post.
I don't have a witty, insightful, neutral-sounding way to say this. The grantmakers should let the money flow. There are thousands of talented young safety researchers with decent ideas and exceptional minds, but they probably can't prove it to you. They only need one thing and it is money.
They will be 10x less productive in a big nonprofit and they certainly won't find the next big breakthrough there.
(Meanwhile, there are becoming much better ways to make money that don't involve any good deeds at all.)
My friends were a good deal sharper and more motivate...
1
1In general people should feel free to DM me with pitches for this sort of thing.
Just wanted to flag quickly that Open Philanthropy's GCR Capacity Building team (where I work) has a career development and transition funding program.
The program aims to provide support—in the form of funding for graduate study, unpaid internships, self-study, career transition and exploration periods, and other activities relevant to building career capital—for individuals at any career stage who want to pursue careers that could help reduce global catastrophic risks (esp. AI risks). It’s open globally and operates on a rolling basis.
I realize that this ...
My friends were a good deal sharper and more motivated at 18 than now at 25.
How do you tell that there were sharper back then?
I think I disagree. This is a bandit problem, and grantmakers have tried pulling that lever a bunch of times. There hasn't been any field-changing research (yet). They knew it had a low chance of success so it's not a big update. But it is a small update.
Probably the optimal move isn't cutting early-career support entirely, but having a higher bar seems correct. There are other levers that are worth trying, and we don't have the resources to try every lever.
Also there are more grifters now that the word is out, so the EV is also declining that way.
(I feel bad saying this as someone who benefited a lot from early-career financial support).
2
1Wasted opportunity to guarantee this post keeps getting holywar comments for the next hundred years.
This is pretty inspiring to me. Thank you for sharing.
The other day I was trying to think of information leaks that a competent conspiracy couldn't prevent, regarding this. I just thought of one small one: people will sometimes randomly die or have their homes raided. If the slavery is common, then sometimes the slaves will be discovered during these events. Even if the escapees wanted to silence the story out of shame, cops would probably gossip to the press.
So you can probably tally such events, crunch the numbers, and get a decent conspiracy-resistant estimate.
As a layman, I have not seen much unrealistic hype. I think the hype-level is just about right.
You should not bury such a good post in a shortform
Maybe it should be a game that everyone can play
Yeah I just wanted to check that nobody is giving away money before I go do the exact opposite thing I've been doing. I might try to tidy something up and post it first
Yes.
I do think I could put a good team together and make decent contributions quickly
I wonder if anybody has tried to quantify how much it's worth to be a swing voter. I imagine if you are the government contractor up for renewal then it's worth quite a lot, but I wonder how much of the money/benefits the average Joe sees.
I don't know much about swing state benefits except that Milwaukee, Wisconsin got their lead pipes replaced by the fed and the workers were required to be local and they say they were paid quite well https://youtube.com/watch?v=4VpwgG0P8VU
Makes sense. Those were real questions, to be clear.