All of Neil 's Comments + Replies

The original post, the actual bet, and the short scuffle in the comments is exactly the kind of epistemic virtue, basic respect, and straight-talking object-level discussion that I like about LessWrong. 

I'm surprised and saddened that there aren't more posts like this one around (prediction markets are one thing; loud, public bets on carefully written LW posts are another).

Having something like this occur every ~month seems important from the standpoint of keeping the garden on its toes and remind everyone that beliefs must pay rent, possibly in the form of PayPal cash transfers. 

I wrote this after watching Oppenheimer and noticing with horror that I wanted to emulate the protagonist in ways entirely unrelated to his merits. Not just unrelated but antithetical: cargo-culting the flaws of competent/great/interesting people is actively harmful to my goals! Why would I do this!? The pattern generalized, so I wrote a rant against myself, then figured it'd be good for LessWrong, and posted it here with minimal edits. 

I think the post is crude and messily written, but does the job. 

Meta comment: I notice I'm surprised that out ... (read more)

Neil 92

I think you're right, but I rarely hear this take. Probably because "good at both coding and LLMs" is a light tail end of the distribution, and most of the relative value of LLMs in code is located at the other, much heavier end of "not good at coding" or even "good at neither coding nor LLMs". 

(Speaking as someone who didn't even code until LLMs made it trivially easy, I probably got more relative value than even you.) 

Neil 10

need any help on post drafts? whatever we can do to reduce those trivial inconveniences 

Neil 32

I'm very pro- this kind of post. Whatever this is, I think it's important for ensuring LW doesn't get "frozen" in a state where specific objects are given higher respect than systems. Strong upvoted.

Neil 60

I think you could get a lot out of adding a temporary golden dollar sign with amount donated next to our LW names! Upon proof of donation receipt or whatever.

Seems like the lowest hanging fruit for monetizing vanity— benches being usually somewhat of a last resort!

(The benches seem still underpriced to me, given expected amount raised and average donation size in the foreseeable future).

Neil 10

I've been at Sciences Po for a few months now. Do you have any general advice? I seem to have trouble taking the subjects seriously enough to any real effort in them, which you seem to point out as a failure mode you skirted. Asking as many people I can for this, as I'm going through a minor existential crisis. Thanks!

Neil 83

Yeah that'd go into some "aesthetic flaws" category which presumably has no risk of messing with your rationality. I agree these exist. And I too am picky.

Neil 52

I agree about the punchline. Chef's kiss post

Good list!

I personally really like Scott Alexander's Presidential Platform, it hits the hilarious-but-also-almost-works spot so perfectly. He also has many Bay Area house party stories in addition to the one you link (you can find a bunch (all?) linked at the top of this post). He also has this one from a long time ago, which has one of the best punchlines I've read.

Neil 10

Can I piggy-back off your conclusions so far? Any news you find okay?

3AnthonyC
The first few that came to mind have, it turns out, already retired since I last talked to them. The next few are basically all bloggers with a tighter focus that I learned about either here on LW or through recommendations that ultimately chain back to SSC/ASX.  There are a lot of good sources of data in the world, and very few good sources of analysis, and those that exist have very little relationship to popularity or price or prestige. Beyond that, it really is "buyer beware," and learning to know your own limited and improve your own speed at sorting the nonsense and spotting the bad assumptions and wrong inferences. That's why I'm not being more specific - without knowing your own habits of thought, it's hard to guess whose habitual mistakes and quirks will be transparent to you, and which will mislead you or just not paint the intended picture for you. Probably my advice is, if something you read seems worth understanding, try to spot and discount (what Zvi calls) the Obvious Nonsense. Back in 2008 my aunt sent me a NYT article that (seemingly) claimed having granite countertops was like smoking a pack of cigarettes a day. Obvious nonsense. It was basically pretending the cancer risk from cigarettes was radiological and not chemical, but unless you knew enough about biology and physics it was easy to miss. I was primed for that, because I was a physics undergrad and because I'd recently been reading about how coal plants release more radiation than nuclear plants even if you ignore all other chemical pollution. Eventually this kind of thinking became habitual for me, and it became easier to learn useful things from inadequate sources.  It's also been educational to go through a few healthcare situations where you need to find doctors that are Actually Good and not just cargo-cult-style going through the motions. There's a vibe, a style, that just shines through regardless of specific topic, related to curiosity and excitement about something new and diff
Neil 10

Well then, I can update a little more in the direction not to trust this stuff.

3AnthonyC
To be fair, there are exceptions, and some reporters or publications consistently do better at oarticular kinds of reporting. It just takea a lot of work to reliably figure out which are which.
Neil 10

Ah right, the decades part--I had written about the 1930 revolution, commune, and bourbon destitution, then checked the dates online and stupidly thought "ah, it must be just 1815 then" and only talked about that. Thanks

Neil 10

"second" laughcries in french

Neil 5-2

Ahem, as one of LW's few resident Frenchmen, I must interpose to say that yes, this was not the Big Famous Guillotine French revolution everyone talks about, but one of the ~ 2,456^2 other revolutions that went on in our otherwise very calm history. 

Specifically, we refer to the Les Mis revolution as "Les barricades" mostly because the people of Paris stuck barricades everywhere and fought against authority because they didn't like the king the other powers of Europe put into place after Napoleon's defeat. They failed that time, but succeeded 15 years... (read more)

7David Matolcsi
That's not quite true. Les Mis starts in 1815, but the book spans decades and the revolution is in 1832, a short-lived uprising against the king who got in power two years before, in the 1830 revolution against the dynasty the other European powers restored after Napoleon's defeat in 1815.
Neil 90

Do we know what side we're on? Because I opted in and don't know whether I'm East or West, it just feels Wrong. I guess I stand a non-trivial chance of losing 50 karma ahem please think of the daisy girl and also my precious internet points.

Neil 30

Anti-moderative action will be taken in response if you stand in the way of justice, perhaps by contacting those hackers and giving them creative ideas. Be forewarned.

Neil 10

Fun fact: it's thanks to Lucie that I ended up stumbling onto PauseAI in the first place. Small world + thanks Lucie.

Neil 71

Update everyone: the hard right did not end up gaining a parliamentary majority, which, as Lucie mentioned, could have been the worse outcome wrt AI safety.

Looking ahead, it seems that France will end up being fairly confused and gridlocked as it becomes forced to deal with an evenly-split parliament by playing German-style coalition negociation games. Not sure what that means for AI, except that unilateral action is harder.

For reference, I'm an ex-high school student who just got to vote for the first 3 times in his life because of French political turmoi... (read more)

1Neil
Fun fact: it's thanks to Lucie that I ended up stumbling onto PauseAI in the first place. Small world + thanks Lucie.
Neil 10

I'm working on a non-trivial.org project meant to assess the risk of genome sequences by comparing them to a public list of the most dangerous pathogens we know of. This would be used to assess the risk from both experimental results in e.g. BSL-4 labs and the output of e.g. protein folding models. The benchmarking would be carried out by an in-house ML model of ours. Two questions to LessWrong: 

1. Is there any other project of this kind out there? Do BSL-4 labs/AlphaFold already have models for this? 

2. "Training a model on the most dangerous pa... (read more)

Neil 10

I'm taking this post down, it was to set up an archive.org link as requested by Bostrom, and no longer serves that purpose. Sorry, this was meant to be discreet.

Neil 40

Poetry and practicality

I was staring up at the moon a few days ago and thought about how deeply I loved my family, and wished to one day start my own (I'm just over 18 now). It was a nice moment.

Then, I whipped out my laptop and felt constrained to get back to work; i.e. read papers for my AI governance course, write up LW posts, and trade emails with EA France. (These I believe to be my best shots at increasing everyone's odds of survival).

It felt almost like sacrilege to wrench myself away from the moon and my wonder. Like I was ruining a moment of poetr... (read more)

Neil 54

Too obvious imo, though I didn't downnvote. This also might not be an actual rationalist failure mode; in my experience at least, rationalists have about the same intuition all the other humans have about when something should be taken literally or not.

As for why the comment section has gone berserk, no idea, but it's hilarious and we can all use some fun.

Neil 2-13

Can we have a black banner for the FHI? Not a person, still seems appropriate imo.

Neil 86

FHI at Oxford
by Nick Bostrom (recently turned into song):

the big creaky wheel
a thousand years to turn

thousand meetings, thousand emails, thousand rules
to keep things from changing
and heaven forbid
the setting of a precedent

yet in this magisterial inefficiency
there are spaces and hiding places
for fragile weeds to bloom
and maybe bear some singular fruit

like the FHI, a misfit prodigy
daytime a tweedy don
at dark a superhero
flying off into the night
cape a-fluttering
to intercept villains and stop catastrophes

and why not base it here?
our spandex costumes
blend in wi... (read more)

Neil 10

I've come to think that isn't actually the case. E.g. while I disagree with Being nicer than clippy, it quite precisely nails how consequentialism isn't essentially flawless:

I haven't read that post, but I broadly agree with the excerpt. On green did a good job imo in showing how weirdly imprecise optimal human values are. 

It's true that when you stare at something with enough focus, it often loses that bit of "sacredness" which I attribute to green. As in, you might zoom in enough on the human emotion of love and discover that it's just an endless ti... (read more)

Neil 0-2

Interesting! Seems like you put a lot of effort into that 9,000-word post. May I suggest you publish it in little chunks instead of one giant post? You only got 3 karma for it, so I assume that those who started reading it didn't find it worth the effort to read the whole thing. The problem is, that's not useful feedback for you, because you don't know which of those 9,000 words are presumably wrong. If I were building a version of utilitarianism, I would publish it in little bursts of 2-minute posts. You could do that right now with a single section of your original post. Clearly you have tons of ideas. Good luck! 

Neil 21

You know, I considered "Bob embezzled the funds to buy malaria nets" because I KNEW someone in the comments would complain about the orphanage. Please don't change. 

Actually, the orphanage being a cached thought is precisely why I used it. The writer-pov lesson that comes with "don't fight the hypothetical" is "don't make your hypothetical needlessly distracting". But maybe I miscalculated and malaria nets would be less distracting to LWers. 

Anyway, I'm of course not endorsing fund-embezzling, and I think Bob is stupid. You're right in that failu... (read more)

8tailcalled
I think your position here is approximately-optimal within the framework of consequentialism. It's just that I worry that consequentialism itself is the reason we have problems like AI x-risk, in the sense that the thing that drives x-risk scenarios may be the theory of agency that is shared with consequentialism. I've been working on a post - actually I'm going to temporarily add you as a co-author so you can see the draft and add comments if you're interested - where I discuss the flaws and how I think one should approach it differently. One of the major inspirations is Against responsibility, but I've sort of taken inspiration from multiple places, including critics of EA and critics of economics. I've come to think that isn't actually the case. E.g. while I disagree with Being nicer than clippy, it quite precisely nails how consequentialism isn't essentially flawless: Unbounded utility maximization aspires to optimize the entire world. This is pretty funky for just about any optimization criterion people can come up with, even if people are perfectly flawless in how well they follow it. There's a bunch of attempts to patch this, but none have really worked so far, and it doesn't seem like any will ever work.
3faul_sname
Fixed, thanks
Neil 30

Re: sociology. I found a meme you might enjoy, which would certainly drive your teacher through the roof: https://twitter.com/captgouda24/status/1777013044976980114 

Neil 10

Yeah, that's an excellent idea. I often spot typos in posts, but refrain from writing a comment unless I collect like three. Thanks for sharing!

Neil 6254

A functionality I'd like to see on LessWrong: the ability to give quick feedback for a post in the same way you can react to comments (click for image). When you strong-upvote or strong-downvote a post, a little popup menu appears offering you some basic feedback options. The feedback is private and can only be seen by the author. 

I've often found myself drowning in downvotes or upvotes without knowing why. Karma is a one-dimensional measure, and writing public comments is a trivial inconvience: this is an attempt at middle ground, and I expect it to ... (read more)

6papetoast
Relatedly, in-line private feedback. I saw a really good design for alerting typos here.
Yoav Ravid*1912

I suggested something similar a few months back as a requirement for casting strong votes.

Strong upvote, but I won't tell you why.

Neil 10

I'm not clear on what you're calling the "problem of superhuman AI"?

1NickH
I've heard much about the problems of misaligned superhuman AI killing us all but the long view seems to imply that even a "well aligned" AI will prioritise inhuman instrumental goals.
Neil 10

I was given clear instructions from a math phd about how to dump random lean files into the repository I created to confuse lesswrongers for at least a few minutes. But then I got confused while attempting to follow the instructions. There’s only so much my circuits can handle. I’m running most of my code on a Chromebook! Fear me.

Neil 133

Bonus song in I have been a good Bing: "Claude's Anguish", a 3-minute death-metal song whose lyrics were written by Claude when prompted with "how does the AI feel?": https://app.suno.ai/song/40fb1218-18fa-434a-a708-1ce1e2051bc2/ (not for the faint of heart)

2Heron
I hate death metal. This is a great song!
Neil 89

I'm glad "thought that faster" is the slowest song of the album. Also where's the "Eliezer Yudkowsky" in the "ft. Eliezer Yudkowsky"? I didn't click on it just to see Eliezer's writing turned into song, I came to see Eliezer sing. Missed opportunity. 

Neil 63

I'm not convinced. I felt the training video was incomplete, and the deadline too short.

Thanks for the feedback Neil! At LM we know that insights can come from anywhere. We appreciate your input regarding the training video's completeness and the deadline duration. In the meantime please feel free to apply for one of our graduate positions, where presumably one can feel better working on capabilities since 'someone else will just take the job anyway': https://www.lockheedmartinjobs.com/job/aguadilla/software-engineer-fire-control-weapons-early-career/694/53752768720

Neil 30

I think that's fair. Public transport is a lot more important in France than in the US, for example, and is usually the first casually in political upheavals. As with the retirement age debacle a few months ago, railway and bus operators (along with other public services like garbage collectors and school administration) went on mass strikes. It's easier here to make big, daring political actions than in the US where eg cars are the default mode of transport. 

5ChristianKl
Even when it comes to cars, there are plenty of French protests where tractors are used to block roads. You don't see similar blockages in the US and when you saw it in Canada their premier was essentially claiming dictatorship powers for himself to fight the protests.
Neil 10

Certainly I would expect people to grow up relatively normal, even in a crazy climate. What I see for religion, I expect to see here. Beyond the natural "immunity" I think my peers will develop over time, I imagine that whatever revolutionary fervor they get from youth will fade as well. My communist friend is going to be a high school philosophy teacher soon enough; by then his "glorious revolution" won't stretch much further than in a few academic dissertations (read by literally no one). 

That story with the sociology teacher is certainly crazy. I t... (read more)

5TeaTieAndHat
I’m not 100% sure the relevant lesson is to avoid sociology (or some other social sciences) entirely. The way I see it, it’s about as reliable as psychology if there had never been anything like a replication crisis: loads of nonsense at the very core of the field, and that everyone seems to think is gospel, but with a few good insights and useful approaches hidden in it—okay, maybe sociology has significantly more people who have been made actually crazy by politics, though. Then, either you avoid it entirely, or you engage with it knowing that you’re on a quest to find as much actually useful things in it as you can. If you do what I did as a 1st year student and engage with them only for your brain to immediately conflate the misunderstood, the immature, and the many genuinely crazy beliefs into "everyone’s completely nutty in that school!", you might make yourself more miserable than needed :-)  Really cool projects, though! Good luck with those! I’m not in SciencesPo currently, but I’ve heard that some folks had started an EA association which seems to be growing pretty fast, and the (pre-existing) cybersecurity association seems to be moving a little toward AI risk, and to do it well (they’re often in touch with the main people working on AI safety here). edit: if I had wanted to summarise my comments above in one sentence (I might have wanted to do that, right? ;-) ), it would be something like: SciencesPo is weird because it’s a great place to work on X-risk governance and policy, and quite a few folks in EA/rationalist circles do just that, but the vibes of the place  are just completely opposed to LW-style rationality. Not throwing the baby with the bathwater, then, is surprisingly hard. 
Neil 32

More of... whatever this is on LessWrong, please! Great humor! Imma go open sheets now and optimally estimate turtle weights (as one does on a good friday night). 

Edit: hot damn, you've got a whole sequence of this stuff!

Neil 23

They took it down real quick for some reason.

3habryka
Oh, it seemed like the kind of thing you would only keep up around the time of death, and we kept it up for a bit more than a day. Somehow it seemed inappropriate to keep it up for longer.
5the gears to ascension
it's still there for me
Neil 32

Concept creep is a bastard. >:(

Neil 20

This reminds me of when Charlie Munger died at 99, and many said of him "he was just a child". Less of a nod to transhumanist aspirations, and more to how he retained his sparkling energy and curiosity up until death. There are quite a few good reasons to write "dead far too young". 

Neil 91

More French stories: So, at some point, the French decided what kind of political climate they wanted. What actions would reflect on their cause well? Dumping manure onto the city center using tractors? Sure! Lining up a hundred stationary taxi cabs in every main artery of the city? You bet! What about burning down the city hall's door, which is a work of art older than the United States? Mais évidemment!

"Politics" evokes all that in the mind of your average Frenchman. No, not sensible strategies that get your goals done, but the first shiny thing the prot... (read more)

Neil 1-1

The new designs are cool, I'd just be worried about venturing too far into insight porn. You don't want people reading the posts just because they like how they look (although reading them superficially is probably better than not reading them at all). Clicking on the posts and seeing a giant image that bleeds color into the otherwise sober text format is distracting. 

I guess if I don't like it there's always GreaterWrong.

Neil 10

Yeah I think I'm wrong about this. Thanks to all of you commenters for feedback. I'm updating.

Load More