LESSWRONG
LW

Comment Permalink
mingyuan3616

I do have some general sense here that those aren't emotionally realistic options for people with my emotional makeup.

Here's my take: From the inside, Nate feels like he is incapable of not becoming very frustrated, even angry. In a sense this is true. But this state of affairs is in fact a consequence of Nate not being subject to the same rules as everybody else.

I think I know what it's like, to an extent — I've had anger issues since I was born, and despite speaking openly about it to many people, I've never met anyone who's been able to really understand the feeling of being overwhelmed with rage (especially not as an adult). That feeling can be very hard to control. 

However, I am constantly aware that having an angry outburst is massively socially unacceptable, to the point where if I let such things happen regularly I would lose my job / my standing in the community / all my friends / everyone close to me. This creates an extremely strong incentive for me to self-regulate at least my outward reactions, even when it's really hard. But because Nate is so high-status, he is allowed to make such outbursts without being faced with losing his job, his standing in the community, or his friends. This means he is insufficiently incentivized to self-regulate, and thus has been unable to learn.

Reply3
See in context
by Raemon
1 min read
140

71

One of the subthreads in Thomas Kwa's MIRI research experience was about his experience working with Nate. In the comments, some other people brought up their own negative experiences. There was a lot of ensuing discussion about it.

Thomas felt this was distracting from the points he was most interested in (e.g. how infohazard policies slow down research, how new researchers can stop flailing around so much, whether deconfusion is a bottleneck to alignment, or the sharp left turn, etc). 

I also somewhat regretted curating the post since we normally avoid curating "community politics" posts, and while the post had a lot of timeless content in both the OP and the discussion, it ended up being a major focus of the comments.

So, I'm moving those comments to this escape-valve-post, where the discussion can continue in whatever direction people end up taking while leaving the original post to focus on more timeless topics that are relevant whether or not you're plugged into particular social scenes.

epistemic status: in that sweet spot myself ↩︎

Drama1Community2Site Meta2AI1
Personal Blog

71

Related Discussion from Thomas Kwa's MIRI Research Experience
110TurnTrout
New Comment


139 comments, sorted by
top scoring
Click to highlight new comments since:
Some comments are truncated due to high volume. (⌘F to expand all)Change truncation settings
[-]TurnTrout11074

"Whelp, people are spiky. Often, the things that are inconvenient about people are tightly entwined with the things that are valuable about them. Often people can change, but not in many of the obvious ways you might naively think." So I'm sort of modeling Nate (or Eliezer, although he doesn't sound as relevant) as sort of a fixed cognitive resource, without too much flexibility on how they can deploy that resource.

I perceive some amount of "giving up" on maintaining social incentives in this comment. I think that's a mistake, especially when the people are in positions of great power and status in this community. 

I think the quoted passage advances an attitude which, in general, allows community invasion by malefactors and other bad actors. Social norms are present for a reason. I think it's reasonable and healthy to expect people to engage in a respectful and professional manner.

If some individual (like Nate) finds these norms costly for some reason, then that shouldn't mean "banishment" or "conclude they have bad intent" so much as—at the minimum—"they should clearly communicate their non-respectful/-kind alternative communication protocols beforehand, and they should help ... (read more)

Reply4
[-]KurtB*21661

Alright, I’ll say it.

I did office operations at MIRI from Sep 2017 to June 2018 as a contractor and it finally feels right to share. All views herein are my own and not meant to represent anyone else. I intended to write a few paragraphs here but ended up with several pages.

Okay, so...my gut wants me to shout, “He’s not simply overly blunt in math arguments! He’s mean and scary[1] toward ops workers! Doesn’t anyone notice this?! I feel like I’m taking crazy pills!” That’s my gut reaction. My reasoned words below will be longer and more nuanced[2]. 

I wish some people at MIRI had explicitly told me up front something like, “Hey, if you mess up a lunch order, you may want to avoid Nate until the next day. He is a very high-performing researcher, and you should not expect typical levels of patience or anger management from him. Also, if you try to stand up for yourself, he may simply cut you off and storm out of the room. Such is the price of having world-savers…do you have thick enough skin to work here?” And I would have said, “Ah, I appreciate the candor and respect. Seems like you guys are making a reasonable tradeoff--after all Newton was notoriously prickly too. But I’m... (read more)

Reply35159532
[-]So8res*7613

  1. Thanks for saying so!

  2. My intent was not to make you feel bad. I apologize for that, and am saddened by it.

    (I'd love to say "and I've identified the source of the problem and successfully addressed it", but I don't think I have! I do think I've gotten a little better at avoiding this sort of thing with time and practice. I've also cut down significantly on the number of reports that I have.)

  3. For whatever it's worth: I don't recall wanting you to quit (as opposed to improve). I don't recall feeling ill will towards you personally. I do not now think poorly of you personally on account of your efforts on the MIRI ops team.

As to the question of how these reports hit my ear: they sound to me like accurate recountings of real situations (in particular, I recall the bike pump one, and suspect that the others were also real events).

They also trigger a bunch of defensiveness in me. I think your descriptions are accurate, but that they're missing various bits of context.

The fact that there was other context doesn't make your experience any less shitty! I reiterate that I would have preferred it be not-at-all shitty.

Speaking from my sense of defensiveness, and adding in some of th... (read more)

Reply632
[-]KurtB*8728

I have some replies to Nate's reply. 

Overview:

  • I’m not asking anyone to modify their personality at all. I mainly wish I had been warned about what was in store for me when I joined MIRI, and I want such warnings to be a visible part of Nate's reputation.
  • I feel some pressure to match Nate's conciliatory tone, but something feels incongruous about it. I'm concerned that people will read Nate's calm, kindly replies and come away with the wrong idea of how he presented himself at MIRI.
  • I find Nate’s additional context to be, well…missing some important context. See below...

 

More context and recollections

It’s true that I didn’t report directly to Nate, and there could be a reasonable expectation that I refrain from bothering him without at least talking to my manager first. My memory is that this was a practical emergence, and not an explicit rule. Regardless, it seemed that Nate was sort of having it both ways, because he did in fact sometimes directly ask me questions (while quite angry), for example why we had ordered lunch from a restaurant he didn’t like, or where the soy sauce was. I now have to wonder what would have happened if I had refused to answer those angrily pose... (read more)

Reply5
[-]mingyuan3616

I do have some general sense here that those aren't emotionally realistic options for people with my emotional makeup.

Here's my take: From the inside, Nate feels like he is incapable of not becoming very frustrated, even angry. In a sense this is true. But this state of affairs is in fact a consequence of Nate not being subject to the same rules as everybody else.

I think I know what it's like, to an extent — I've had anger issues since I was born, and despite speaking openly about it to many people, I've never met anyone who's been able to really understand the feeling of being overwhelmed with rage (especially not as an adult). That feeling can be very hard to control. 

However, I am constantly aware that having an angry outburst is massively socially unacceptable, to the point where if I let such things happen regularly I would lose my job / my standing in the community / all my friends / everyone close to me. This creates an extremely strong incentive for me to self-regulate at least my outward reactions, even when it's really hard. But because Nate is so high-status, he is allowed to make such outbursts without being faced with losing his job, his standing in the community, or his friends. This means he is insufficiently incentivized to self-regulate, and thus has been unable to learn.

Reply3
[-]PoignardAzur316

I think it's cool that you're engaging with criticism and acknowledging the harm that happened as a result of your struggles.

And, to cut to the painful part, that's about the only positive thing that I (random person on the internet) have to say about what you just wrote.

In particular, you sound (and sorry if I'm making any wrong assumption here) extremely unwilling to entertain the idea that you were wrong, or that any potential improvement might need to come from you.

You say:

For whatever it's worth: I don't recall wanting you to quit (as opposed to improve).

But you don't seem to consider the idea that maybe you were more in a position to improve than he was.

I don't want to be overly harsh or judgmental. You (eventually) apologize and acknowledge your responsibility in employees having a shitty time, and it's easy for an internet stranger to over-analyze everything you said.

But. I do feel confident that you're expressing a lack of curiosity here. You're assuming that there's nothing you possibly have done to make Kurt's experience better, and while you're open to hearing if anyone presents you with a third option, you don't seem to think seeking out a third option is a problem... (read more)

Reply432
[-]Aella844

I've been dating Nate for two years (tho wanna clarify we are not doing marriage-kids and we're both actively looking for more serious other partners).

Nate is profoundly wonderful in many ways, like often surprises me in new ways of wonderfulness, and has raised my standards in partners. He's deeply caring, attentive, competent, hilarious, and of course brilliant.

Also, many of the complaints about him in the comments resonate with my experience, particularly your description above. I often find that in disputes I feel dismissed, I perceive him as having a significant lack of curiosity about my worldview (and believe he's explicitly said he's not curious about perspectives he anticipates to have no value to him). 

Iirc he's explicitly said he doesn't respect my thinking (edit: he clarifies he respects it in some areas but not others), and from my perspective this radiates off him whenever we fight. I often feel like I have trouble trusting my own mind, I doubt myself, and despite my best attempts I somehow come out of disputes thinking I must be the one who's wrong. It's weird to have a partner who's so shockingly good in so many ways, yet we have maybe the worst fights I've eve... (read more)

Reply997
[-]So8res*120

Thanks <3

(To be clear: I think that at least one other of my past long-term/serious romantic partners would say "of all romantic conflicts, I felt shittiest during ours". The thing that I don't recall other long-term/serious romantic partners reporting is the sense of inability to trust their own mind or self during disputes. (It's plausible to me that some have felt it and not told me.))

Reply3
[-]sim350

Chiming in to provide additional datapoints. (Apologies for this being quite late to the conversation; I frequent The Other Forum regularly, and LW much less so, and only recently read this post/comments.) My experience has been quite different to a lot of the experiences described here, and I was very surprised when reading. 

I read all of the people who have had (very) negative experiences as being sincere and reporting events and emotions as they experienced them. I could feel what I perceived to be real distress and pain in a lot of the comments, and this was pretty saddening. 

Note: I really don’t like posting personal information on the public internet, for both personal preference and professional reasons. (I felt sure I wanted to post this, though.)

Background: I dated Nate on-and-off for ~6 years (from 2016-2022). We’re now friends on good terms. 

How I experienced Nate’s communication over the years:

  • During disputes, I felt Nate respected my views and my feelings. I felt Nate tried to communicate well with me (though of course this is often hard for both parties in the midst of a dispute and we both failed a bunch).
  • During disputes, and generally, I felt like Nat
... (read more)
Reply
[-]So8res3525

In particular, you sound [...] extremely unwilling to entertain the idea that you were wrong, or that any potential improvement might need to come from you.

you don't seem to consider the idea that maybe you were more in a position to improve than he was.

Perhaps you're trying to point at something that I'm missing, but from my point of view, sentences like "I'd love to say "and I've identified the source of the problem and successfully addressed it", but I don't think I have" and "would I have been living up to my conversational ideals (significantly) better, if I'd said [...]" are intended indicators that I believe there's significant room for me to improve, and that I have desire to improve.

At to be clear: I think that there is significant room for improvement for me here, and I desire to improve.

(And for the record: I have put a decent amount of effort towards improving, with some success.)

(And for the record: I don't recall any instances of getting frustrated-in-the-way-that-turntrout-and-KurtB-are-recounting with Thomas Kwa, or any of Vivek's team, as I think is a decent amount of evidence about those improvements, given how much time I spent working with them. (Which isn... (read more)

Reply1
7Vaniver
So I've been thinking about this particular branch for a while and I think I have a slightly different diagnosis from PoignardAzur, which I think nearly lines up with yours but has an important difference. I think this is the important part: Even if you are not tracking who is Wrong is any particular interaction, if other people are tracking who is Wrong, that seems like an important thing for you to handle because it will be a large part of how they interpret communication from you. (For the bike pump example, the thing where you saw Kurt as "begging pardon" seems like evidence this was plausibly up for Kurt / you could have guessed this was up for Kurt in the moment.) One way to interpret the situation is: Kurt: I am Wrong but would like to displace that to the bike pump Nate: Rejected! >:[ Kurt: :(   I am imagining that you were not asking for this sort of situation (and would have been less interested in a "save your time" deal if "do emotional labor for people helping you" had explicitly been part of the deal), but my guess is attention to this sort of thing is the next place to look for attacking the source of the problem. [Also, I'm not trying to confidently assert this is what was actually up for Kurt in the moment--instead I'm asking "if this story made me side with Kurt, why did that happen?"]
3PoignardAzur
I don't know if "dense" is the word I use, but yeah, I think you missed my point. My ELI5 would be "You're still assuming the problem was 'Kurt didn't know how to use a pump' and not 'there was something wrong with your pump'". I don't want to speculate too much beyond that eg about the discretionary budget stuff. Happy to hear that!
5So8res
(I had used that pump that very day, shortly before, to pump up the replacement tire.)
[-]TurnTrout*1713

The best life-hack I have is "Don't be afraid to come back and restart the discussion once you feel less frustration or exasperation".

I talked to Kurt in some detail. Nate never apologized or acknowledged the bike pump incident (until now). After that incident, Nate never came back and said e.g. “wow, I was really frustrated earlier, sorry for taking that out on you!” The next time Kurt was alone with him was in the elevator later that week, and there was a cold silence that neither of them broke.

Reply1
[-]Thomas Kwa1215

Perhaps I'm missing some obvious third alternative here, that can be practically run while experiencing a bunch of frustration or exasperation. (If you know of one, I'd love to hear it.)

One alternative could be to regulate your emotions so you don't feel as intense frustration from a given epistemic position? I think this is what most people do.

Reply
[-][anonymous]124

I suspect that lines like this are giving people the impression that you [Nate] don't think there are (realistic) things that you can improve, or that you've "given up". 

I do have some general sense here that those aren't emotionally realistic options for people with my emotional makeup.

I have a sense that there's some sort of trap for people with my emotional makeup here. If you stay and try to express yourself despite experiencing strong feelings of frustration, you're "almost yelling". If you leave because you're feeling a bunch of frustration and people say they don't like talking to you while you're feeling a bunch of frustration, you're "storming out".

My understanding is that your perspective is something like "I feel like I recognize that there's stuff I can do to improve, and I've tried to put a lot of energy into finding those improvements, and I'm pretty open to others suggesting specific things I could do. But a lot of things that other people think would be easy fixes actually don't work or don't work for someone with my emotional makeup (e.g., because they're super costly or because I don't end up being able to implement them well.)

Like, my guess is that some peop... (read more)

Reply22
8Elizabeth
How do you/MIRI communicate about working with you now?
8So8res
Insofar as you're querying the near future: I'm not currently attempting work collaborations with any new folk, and so the matter is somewhat up in the air. (I recently asked Malo to consider a MIRI-policy of ensuring all new employees who might interact with me get some sort of list of warnings / disclaimers / affordances / notes.) Insofar as you're querying the recent past: There aren't many recent cases to draw from. This comment has some words about how things went with Vivek's hires. The other recent hires that I recall both (a) weren't hired to do research with me, and (b) mentioned that they'd read my communication handbook (as includes the affordance-list and the failure-modes section, which I consider to be the critcial pieces of warning), which I considered sufficient. (But then I did have communication difficulties with one of them (of the "despair" variety), which updated me somewhat.) Insofar as you're querying about even light or tangential working relationships (like people asking my take on a whiteboard when I'm walking past), currently I don't issue any warnings in those cases, and am not convinced that they'd be warranted. To be clear: I'm not currently personally sold on the hypothesis that I owe people a bunch of warnings. I think of them as more of a sensible thing to do; it'd be lovely if everyone was building explicit models of their conversational failure-modes and proactively sharing them, and I'm a be-the-change-you-wanna-see-in-the-world sort of guy. (Perhaps by the end of this whole conversation I will be sold on that hypothesis! I've updated in that direction over the past couple days.) (To state the obvious: I endorse MIRI institutionally acting according to others' conclusions on that matter rather than on mine, hence asking Malo to consider it independently.)
[-]Elizabeth5331

One frame I want to lay out is that it seems like you're not accounting for the organizational cost of how you treat employees/collaborators. An executive director needing to mostly not talk to people, and shaping hiring around social pain tolerance, is a five alarm fire for organizations as small as MIRI. Based on the info here, my first thought is you should be in a different role, so that you have fewer interactions and less implied power.  That requires someone to replace you as ED, and I don't know if there are any options available,  but at a minimum I think you/MIRI should be treating the status quo as potentially extremely costly, and taking steps to assess the total cost and potential fixes. 

I could be wrong here, 98% of my information is from this post + comments, but I get the sense you/MIRI haven't looked sufficiently hard to even assess what the costs are. It sounds like you have asked people, which is great and more than most orgs do, but I get the sense you haven't grappled with the magnitude of the costs beyond the personal and social. 

Reply2
[-]Thomas Kwa*130

That requires someone to replace you as ED

Nate stepped down as ED shortly after [edit: actually before] our project ended, the website just hasn't been updated. I'm not sure what exactly the organizational structure is now, but you can probably message @lisathiergart for an update.

Edit: there is now an announcement.

Reply21
[-]Quinn*35-1

This comment's updates for me personally:

  • The overall "EA is scary / criticizing leaders is scary" meme is very frequently something I roll my eyes at, I find it alien and sometimes laughable when people say they're worried about being bold and brave cuz all I ever see are people being rewarded for constructive criticism. But man, I feel like if I didn't know about some of this stuff then I'm missing a huge piece of the puzzle. Unclear yet what I'll think about, say, the anon meta on forums after this comment sinks in / propagates, but my guess is it'll be very different than what I thought before.
  • People are way too quick to reward themselves for trying (my update is my priority queue in doing a proper writeup): Nate & enablers saying that productivity / irreplaceability is an excuse to triage out fundamental interpersonal effort is equivalent (as far as I'm concerned) to a 2022 University Community Builder (TM) deciding that they're entitled to opulent retreats the moment they declare stated interest in saving the world. "For the greater good" thinking is fraught and dicey even when you're definitely valuable enough for the case to genuinely be made, but obviously there's pr
... (read more)
Reply
1Oliver Sourbut
This is a generally constructive comment. One bit left me confused, and I wonder if you can unpack what it means? What was the misfire? (I mean literally what does 'it' stand for in this sentence?) Also, what kind of points and karma are we talking about, presumably metaphorical?
[-]Quinn142

https://www.lesswrong.com/posts/BGLu3iCGjjcSaeeBG/related-discussion-from-thomas-kwa-s-miri-research?commentId=fPz6jxjybp4Zmn2CK This brief subthread can be read as "giving nate points for trying" and is too credulous about if "introspection" actually works--- my wild background guess is that roughly 60% of the time "introspection" is more "elaborate self-delusion" than working as intended, and there are times when someone saying "no but I'm trying really hard to be good at it" drives that probability up instead of down. I didn't think this was one of those times before reading Kurt's comment. A more charitable view is that this prickliness (understatement) is something that's getting triage'd out / deprioritized, not gymnastically dodged, but I think it's unreasonable to ask people to pay attention to the difference.

That's besides the point: the "it" was just the gdoc. "it would be a steep misfire" would mean "the gdoc tries to talk about the situation and totally does not address what matters". The subtraction of karma was metaphorical (I don't think I even officially voted on lesswrong!). I want to emphasize that I'm still very weak, cuz for instance I can expect people in that ... (read more)

Reply
[-]So8res180

Separately, a friend pointed out that an important part of apologies is the doer showing they understand the damage done, and the person hurt feeling heard, which I don't think I've done much of above. An attempt:

I hear you as saying that you felt a strong sense of disapproval from me; that I was unpredictable in my frustration as kept you feeling (perhaps) regularly on-edge and stressed; that you felt I lacked interest in your efforts or attention for you; and perhaps that this was particularly disorienting given the impression you had of me both from my in-person writing and from private textual communication about unrelated issues. Plus that you had additional stress from uncertainty about whether talking about your apprehension was OK, given your belief (and the belief of your friends) that perhaps my work was important and you wouldn't want to disrupt it.

This sounds demoralizing, and like it sucks.

I think it might be helpful for me to gain this understanding (as, e.g., might make certain harms more emotionally-salient in ways that make some of my updates sink deeper). I don't think I understand very deeply how you felt. I have some guesses, but strongly expect I'm missing a bunch of important aspects of your experience. I'd be interested to hear more (publicly or privately) about it and could keep showing my (mis)understanding as my model improves, if you'd like (though also I do not consider you to owe me any engagement; no pressure).

Reply
[-]Raemon*154

(quick mod note confirming this is Kurt Brown who worked at MIRI)

Reply1
[-]lc*143

With Eliezer, my experience has been the opposite. When I showed up in Berkeley, people who knew Eliezer tripped over themselves to tell me how arrogant and difficult they found him. I’ve talked to him for 5-10 minutes on 5-10 occasions, and every single time he was somewhere between neutral and friendly.

I have only met Eliezer once for about ~60 minutes, but I had the same experience. We talked in a group about alignment, and even though he ended up repeating many concepts he had already written about extensively online, he failed to explain those concepts condescendingly at all, which is not what I've come to expect the median person to do in that situation. It just seemed like he really wanted us to understand the problem.

I was sort of unsurprised at the mismatch between perception and reality, frankly, because Eliezer is a very awkward, sorta funny looking dude. In this vein I will note that I was mildly disappointed a few years back when I checked out MIRI's team page and saw that (IMO), with the exception of the founder, all of the most attractive people were the ones in leadership positions. In my experience, in very relatively nerdy environments, people who look and sound not-like-nerds get social leeway to be domineering and dismissive if they choose. That might explain part of what happened with Nate, though I am not a bay resident and have virtually no inside info here.

Reply1
[-]TekhneMakre1412

This updated me, thank you. A fair amount, from "IDK, this sounds like it's fairly likely to mainly be just people being sensitive about blunt confrontational communication in a context where blunt confrontational communication is called for" to "Maybe that, but sure sounds a lot like Nate has a general disregard for fellows--maybe there's some internal story he has where his behavior would make sense if other people shared that story, but they don't and that should be obvious and he should have not behaved that way given that they don't".

Reply
5PoignardAzur
I'll make a tentative observation: it seems that you're still being euphemistic and (as you kind of note yourself) you're still self-censoring a bit. The words that you say are "he's mean and scary" and "he was not subject to the same behavioral regulation norms as everyone else". The words I would have said, given your description and his answer below is "he acts like an asshole and gets away with it because people enable him". I've known bosses that were mean and scary, but otherwise felt fair and like they made the best of a tough situation. That's not what you're describing. Maybe Nate is an amazing person in other ways, and amazingly competent in ways that make him important to work with, but. He sounds like a person with extremely unpleasant behavior.
-12[comment deleted]
[-]Bird Concept467

Wanted to briefly add a perspective I didn't see mentioned yet -- 

First -- seems like you had a particularly rough interaction, and I do want to express empathy for that. I feel like I recognise some of the things you point to, and think it's plausible that I might have been similarly demoralised by that situation, and that would really suck for me and I'd be really sad. So, genuinely sorry about that. I hope you'll find ways to regain motivation that was unfairly lost, and the ability to draw on insights that ended up involuntarily screened off from you. 

Second, the perspective I've come to hold for these situations is...  Basically the world does seem full of people who are extraordinarily productive in important ways, and who also... are kind of d*cks. (Important footnote: [1]) 

As such:

  • I think exceptional people are often sufficiently rare that, as things go, I'd rather take a bunch of productive d*cks than tune down their cognitive spikiness at the cost of mulling the productive peaks
  • I observe and am strategic about how I allocate my soul and motivation points to things. In the past I would kind of always pour full soul into things, but that led to a lo
... (read more)
Reply31
[-]TurnTrout2010

Basically the world does seem full of people who are extraordinarily productive in important ways, and who also... are kind of d*cks.

I tried to communicate "we should indeed subtract points for people being rude and aggressive" and "stop double-counting evidence by reminding yourself that someone might also be productive; that's already factored into your assessment of them." 

It seems like you're saying "I can imagine many cases where rude people have net positive points." If that's an accurate summary, that's not in conflict with my point.

I'd rather take a bunch of productive d*cks than tune down their cognitive spikiness at the cost of mulling the productive peaks

Can you be more specific about what part of "socially penalize people for being rude in their interactions" would tune down their "cognitive spikiness"? This seems like a false dichotomy, but I'm open to hearing about costs of my proposal I was unaware of.

But I do not think they have a responsibility to proactively inform people about their style. 

So if some boss often drove his employees to tears, as long as he was pretty insightful, you don't think that the employees should be able to know before taking the job? Surely that's not your position. But then what is? 

Reply1
7Bird Concept
I wanted to add a perspective to the conversation that I didn't see mentioned, moreso than advocating a very thought out position. I have conflicting intuitions, and the territory seems messy!  On the one hand, it does seem to me like there should be some kind of "heads up about intensity". It's real bad to create hidden slippery slopes along the intensity scale. It's real bad to first make people dependent on you (by, say, paying most of their salary in yet-to-be-vested equity, making them work long enough that they can't explore external opportunities and maintain outside friends, ...) and then shifting into a potentially abusive stance (heavily frame controlling, demoralising, etc). It is when these kinds of pressures are applied that I think things move into unacceptable territory. (And my suggested community response would probably be something like "Sandbox culprit in ways where they're able to remain highly productive while doing less damage, give people accurate indications about their style (conveying this might actually fall on someone else than the culprit to do -- that division of labor might just be our only way to get all the good stuff here!), and avoid giving people inaccurate impressions or being a wide-eyed feeder school." For comparison, when I imagine pursuing a career in investment banking, it seems like I'd be opting into a shark tank. I'm just kind of accepting there'll be some real abusive folks around, following the $$$, and I'll be betting on my ability to navigate that without losing myself in the process. Being part of a healthy community means somehow having people around me who can help me see these things. I do think there are some young undergrads who naively will believe the faceless Goldman ads. I feel like Taleb would have a word for them -- the "sucker" or the "Intellectual Yet Idiot". They'll get hurt, and this is bad, and the recruiting ads that led them into this are immoral.  (From that perspective, I'm pretty into my straw
[-]Viliam107

give people accurate indications about their style (conveying this might actually fall on someone else than the culprit to do

In my opinion, this is a responsibility of the person who made the decision that Nate works for their organization. They should either do it, or delegate it to someone and verify that it is done.

Reply
0Said Achmiz
Seconding this. A very large part, perhaps the majority, of the problem with everything that’s been described in this discussion seems to me to boil down to “poor management”. Totally ignoring that and instead focusing either on Nate or on any of the “complainants” here seems just wildly misguided.
-2Thoth Hermes
My take is that they (those who make such decisions of who runs what) are pretty well-informed about these issues well before they escalate to the point that complaints bubble up into posts / threads like these.  I would have liked this whole matter to have unfolded differently. I don't think this is merely a sub-optimal way for these kinds of issues to be handled, I think this is a negative one.  I have a number of ideological differences with Nate's MIRI and Nate himself that I can actually point to and articulate, and those disagreements could be managed in a way that actually resolve those differences satisfactorily. Nate's MIRI - to me - seemed to be one of the most ideologically conformist iterations of the organization observed thus far.  Furthermore, I dislike that we've converged on the conclusion that Nate is a bad communicator, or that he has issues with his personality, or - even worse - that it was merely the lack of social norms imposed on someone with his level of authority that allowed him to behave in ways that don't jive with many people (implying that literally anyone with such authority would behave in a similar way, without the imposition of more punitive and restrictive norms).  Potentially controversial take: I don't think Nate is a bad communicator. I think Nate is incorrect about important things, and that incorrect ideas tend to appear to be communicated badly, which accounts for perceptions that he is a bad communicator (and perhaps also accounts for observations that he seemed frustrated and-or distressed while trying to argue for certain things). Whenever I've seen him communicate sensible ideas, it seems communicated pretty well to me.  I feel that this position is in fact more respectful to Nate himself.  If we react on the basis of Nate's leadership style being bad, his communication being bad, or him having a brusque personality, then he's just going to be quietly replaced by someone who will also run the organization in a simi
4Viliam
It is possible to be a good communicator in some situations (e.g. when you write a blog post) and a bad communicator in other situations (e.g. when someone randomly interrupts you when you were worried about something else). For example, when I talk, I am much less coherent, and my English sucks. If I remember the details correctly (sorry, I am not going to read the entire thread again), this seems like a mistake that could be avoided in the future. -- Someone tried to make Nate happy by telling Kurt to do something for him; Nate didn't ask for any help, but when an attempt was made regardless, he got angry at Kurt because he perceived the help as unreliable, worse than nothing. Kurt was hurt, because this wasn't his idea in the first place, and he tried to communicate a problem with his task, unsuccessfully. -- I think a possible lesson is to just leave Nate alone, unless he explicitly asks for help, and even then think twice whether you chose the right person for the job. And maybe have someone managing your employees, whom they can ask for advice, if needed. (Yes, I would prefer if Nate just magically stopped being angry at people who are trying to help, even if he is not satisfied with the outcome. But it is not wise to rely on magic to happen.) More meta, when people have a bad experience with Nate (or anyone else), don't ignore that fact. Stop and think about the situation. If people felt hurt interacting with me, I would want to know it, get some advice how to prevent this outcome, and if the advice doesn't feel actionable then at least how to avoid such people and/or situations. It doesn't necessarily mean that someone is a bad person, sometimes people just rub each other the wrong way, but in such case there should be an option to avoid each other.
[-]So8res352

I hereby push back against the (implicit) narrative that I find the standard community norms costly, or that my communication protocols are "alternative".

My model is closer to: the world is a big place these days, different people run on different conversation norms. The conversation difficulties look, to me, symmetric, with each party violating norms that the other considers basic, and failing to demonstrate virtues that the other considers table-stakes.

(To be clear, I consider myself to bear an asymmetric burden of responsibility for the conversatiosn going well, according to my seniority, which is why I issue apologies instead of critiques when things go off the rails.)

Separately but relatedly: I think the failure-mode I had with Vivek & co was rather different than the failure-mode I had with you. In short: in your case, I think the issue was rooted in a conversational dynamic that caused me frustration, whereas in Vivek & co's case, I think the issue was rooted in a conversational dynamic that caused me despair.

Which is not to say that the issues are wholly independent; my guess is that the common-cause is something like "some people take a lot of damage from having co... (read more)

Reply4
[-]TurnTrout*3323

I sure don't buy a narrative that I'm in violation of the local norms.

This is preposterous.

I'm not going to discuss specific norms. Discussing norms with Nate leads to an explosion of conversational complexity.[1] In my opinion, such discussion can sound really nice and reasonable, until you remember that you just wanted him to e.g. not insult your reasoning skills and instead engage with your object-level claims... but somehow your simple request turns into a complicated and painful negotiation. You never thought you'd have to explain "being nice."

Then—in my experience—you give up trying to negotiate anything from him and just accept that he gets to follow whatever "norms" he wants.

So, in order to evaluate whether Nate is "following social norms", let's not think about the norms themselves. I'm instead going to share some more of the interactions Nate has had:

  1. "Flipping out" at Kurt Brown because there wasn't enough sourdough bread.
  2. Storming out of the room because Kurt had a question about pumping Nate's tires.
  3. A chat with me, where Nate describes himself as "visibly flustered, visibly frustrated, had a raised voice, and was being mean in various of his replies."
  4. An employee rep
... (read more)
Reply
[-]TurnTrout3615

Regarding my own experience: I would destroy 10% of my liquidity to erase from my life that conversation and its emotional effects. 

I don't think it's reasonable to expect Nate to have predicted that I in particular would be hurt so much. But in fact, being unexpectedly and nonconsensually mean and aggressive and hurtful has heavy negative tails. 

And so statistically, this level of harm is totally predictable, both on priors and based off of past experience which I know Nate has. 

Reply147
-3M. Y. Zuo
This seems like a fairly extreme statement, so I was about to upvote due to the courage required to post it publicly and stand behind it. But then I stopped and thought about the long term effects and it's probably best not to encourage this.  As ideally, you, along with the vast majority of potential readers, should become less emotionally reactive over time to any real or perceived insults, slights, etc... If it's the heat of the moment talking, that's fine, but letting thoughts of payback, revenge, etc., linger on for days afterwards likely will not lead to any positive outcome. 
[-]TurnTrout*177

As ideally, you, along with the vast majority of potential readers, should become less emotionally reactive over time to any real or perceived insults, slights, etc...

If it's the heat of the moment talking, that's fine, but letting thoughts of payback, revenge, etc., linger on for days afterwards likely will not lead to any positive outcome. 

I have had these thoughts many times. I would berate myself for letting it get on my nerves so much. It was just an hour-and-a-half chat. But I don't think it's a matter of "letting" thoughts occur, or not. Certain situations are damaging to certain people, and this situation isn't a matter of whether people are encouraged to be damaged or not (I certainly had no expectation of writing about this, back in July–October 2022.) 

EDIT: Moving another part elsewhere.

Reply
1M. Y. Zuo
I upvoted for effort because it's clear you put in quite a bit of effort in writing this comment, but skipped expressing agreement or disagreement. I had thought of several possible responses, and it is worthy of a substantial response, but since it's not my role to be the adjudicator or corrector of LW users, I'll pose you this question: Consider, is it possible for him to take offence in return and then retaliate via some mean(s)? If that does occur, what's the range of likely outcomes?
6TurnTrout
(Note that I moved most of the original comment and plan to put it elsewhere in the thread.) I don't follow. I'm not going to behave differently in the face of any possible retaliation, nor do I in fact expect Nate to retaliate in an inappropriate manner. So I'm not worried about this?
[-]Lukas_Gloor*1313

[...] I was about to upvote due to the courage required to post it publicly and stand behind it. But then I stopped and thought about the long term effects and it's probably best not to encourage this. [...] As ideally, you, along with the vast majority of potential readers, should become less emotionally reactive over time to any real or perceived insults, slights, etc...

It seems weird to single out this specific type of human limitation (compared to perfect-robot instrumental rationality) over the hundreds of others. If someone isn't in top physical shape or cannot drive cars under difficult circumstances or didn't renew their glasses and therefore doesn't see optimally, would you also be reluctant to upvote comments you were otherwise tempted to upvote (where they bravely disclose some limitation) because of this worry about poor incentives? "Ideally," in a world where there's infinite time so there are no tradeoffs for spending self-improvement energy, rationalists would all be in shape, have brushed up their driving skills, have their glasses updated, etc. In reality, it's perfectly fine/rational to deprioritize many things that are "good to have" because other issues are more... (read more)

Reply
[-]TurnTrout2523

I say all of this as though it's indeed "very uncommon" to feel strongly hurt and lastingly affected by particularly harsh criticism. I don't even necessarily think that this is the case: If the criticism comes from a person with high standing in a community one cares about, it seems like a potentially quite common reaction?

This is relevant context for my strong reaction. I used to admire Nate, and so I was particularly upset when he treated me disrespectfully. (The experience wasn't so much "criticism" as "aggression and meanness", though.)

FWIW, I also reject the framing that this situation is reasonably understood as an issue with my own instrumental rationality.

Going back to the broader point about incentives, it's not very rewarding to publicly share a distressing experience and thereby allow thousands of internet strangers to judge my fortitude, and complain if they think it lacking. I'm not walking away from this experience feeling lavished and reinforced for having experienced an emotional reaction. 

Furthermore, the reason I spoke up was mostly not to litigate my own experience. It's because I've spent months witnessing my friends take unexpected damage from a powerful individual who appears to have faced basically no consequences for his behavior.

Reply
-1M. Y. Zuo
  This is a minor error but I feel the need to correct it for future readers, as it's in the first sentence. There are infinitely many 'specific types' of human limitations, or at least an uncountable quantity , depending on the reader's preferred epistemology. The rest of your thesis is interesting though a bit difficult to parse. Could you isolate a few of the key points and present them in a list? 
8Lukas_Gloor
I wasn't the one who downvoted your reply (seems fair to ask for clarifications), but I don't want to spend much more time on this and writing summaries isn't my strength. Here's a crude attempt at saying the same thing in fewer and different words: IMO, there's nothing particularly "antithetical to LW aims/LW culture" (edit: "antithetical to LW aims/LW culture" is not a direct quote by anyone; but it's my summary interpretation of why you might be concerned about bad incentives in this case) about neuroticism-related "shortcomings." "Shortcomings" compared to a robotic ideal of perfect instrumental rationality. By "neuroticism-related "shortcomings"", I mean things like having triggers or being unusually affected by harsh criticism. It's therefore weird and a bit unfair to single out such neuroticism-related "shortcomings" over things like "being in bad shape" or "not being good at common life skills like driving a car." (I'm guessing that you wouldn't be similarly concerned about setting bad incentives if someone admitted that they were bad at driving cars or weren't in the best shape.) I'm only guessing here, but I wonder about rationalist signalling cascades about the virtues of rationality, where it gets rewarded to be particularly critical about things that least correspond to the image of what an ideally rational robot would be like. However, in reality, applied rationality isn't about getting close to some ideal image. Instead, it's about making the best out of what you have, taking the best next move step-by-step for your specific situation, always prioritizing what actually gets you to your goals rather than prioritizing "how do I look as though I'm very rational." Not to mention that high emotionality confers advantages in many situations and isn't just an all-out negative. (See also TurnTrout's comment about rejecting the framing that this is an issue of his instrumental rationality being at fault.)
-1M. Y. Zuo
I don't mind the occasional downvote or negative karma, it even has some positive benefits, such as being a useful signalling function. As it's decent evidence I haven't tailored my comments for popularity or platitudes. In regards to your points, I'll only try to respond to them one at a time, since this is already pretty far down the comment chain. Who suggested that there was a relation between being "antithetical to LW aims/LW culture" and "neuroticism-related "shortcomings""? i.e. Is it supposed to be my idea, TurnTrout's, your's, a general sentiment, something from the collective unconscious, etc.?
2Lukas_Gloor
I made an edit to my above comment to address your question; it's probably confusing that I used quotation marks for something that wasn't a direct quote by anyone.
1M. Y. Zuo
I appreciate the edit though can you clarify why you put so many quotes in when they are your own thoughts? Is it just an idiosyncratic writing style or is it also meant to convey some emotion, context, direction, etc.? But to clarify, this is not the reason why I 'might be concerned about bad incentives in this case', if you were wondering. 
4Viliam
Not Lukas, but I also sometimes use quotes: * as a kind of semantic brackets; I think the official way to do this is to write-the-words-connected-by-hyphens, but that just seems hard to read; * to remove a possible connotation, i.e. to signal that I am using the word not exactly as most people would probably use it in a similar situation; * or as a combination of both, something like: I am using these words to express an idea, but these are probably not the right words, but I can't find any better, so please do not take this part literally and don't start nitpicking (don't assume that I used a specific word because I wanted to hint at something specific). For example, as I understand it, Means: things that technically are shortcomings (because they deviate from some ideal), but also a reasonable person wouldn't call them so (because it is a normal human behavior, and I would actually be very suspicious about anyone who claimed to not have any of them), so the word is kinda correct but also kinda incorrect. But it is a way to express what I mean.
2Lukas_Gloor
Sounds like I misinterpreted the motivation behind your original comment!  I ran out of energy to continue this thread/conversation, but feel free to clarify what you meant for others (if you think it isn't already clear enough for most readers).
[-]TurnTrout192

(For completeness, I want to note that I've talked with a range of former/current MIRI employees, and a non-trivial fraction did have basically fine interactions with Nate.)

Reply
4TekhneMakre
More detail here seems like it could be good. What form did the insult take? Other relevant context?
4TurnTrout
According to my notes and emails, Nate repeatedly said things like "I have not yet ruled out [uncharitable hypothesis about how TurnTrout is reasoning]" in order to—according to him—accomplish his conversational objectives / because his "polite" statements were apparently not getting his point across. I don't remember what specific uncharitable things he said (the chat was on 7/19/22).  I might come back and add more context later.
6TekhneMakre
Thank you.
[-][anonymous]2516

Huh, I initially found myself surprised that Nate thinks he's adhering to community norms. I wonder if part of what's going on here is that "community norms" is a pretty vague phrase that people can interpret differently. 

Epistemic status: Speculative. I haven't had many interactions with Nate, so I'm mostly going off of what I've heard from others + general vibes. 

Some specific norms that I imagine Nate is adhering to (or exceeding expectations in):

  • Honesty
  • Meta-honesty
  • Trying to offer concrete models and predictions
  • Being (internally) open to acknowledging and recognizing mistakes, saying oops, etc.

Some specific norms that I think Nate might not be adhering to:

  • Engaging with people in ways such that they often feel heard/seen/understood
  • Engaging with people in ways such that they rarely feel dismissed/disrespected
  • Something fuzzy that lots of people would call "kindness" or "typical levels of warmth"

I'm guessing that some people think that social norms dictate something like "you are supposed to be kind and civil and avoid making people unnecessarily sad/insecure/defensive." I wonder if Nate (a) believes that these are community norms and thinks he's following them or (b) just... (read more)

Reply
[-]TekhneMakre2217

Engaging with people in ways such that they often feel heard/seen/understood

This is not a reasonable norm. In some circumstances (including, it sounds like, some of the conversations under discussion) meeting this standard would require a large amount of additional effort, not related to the ostensible reason for talking in the first place.

Engaging with people in ways such that they rarely feel dismissed/disrespected

Again, a pretty unreasonable norm. For some topics, such as "is what you're doing actually making progress towards that thing you've arranged your life (including social context) around making progress on?", it's very easy for people to feel this way, even if they are being told true, useful, relevant things.

Something fuzzy that lots of people would call "kindness" or "typical levels of warmth"

Ditto, though significantly less strongly; I do think there's ways to do this that stay honest and on-mission without too much tradeoff.

Reply
[-]Thomas Kwa179

I think it's not a reasonable norm to make sure your interlocutors never e.g. feel dismissed/disrespected, but it is reasonable to take some measures to avoid having someone consistently feel dismissed/disrespected if you spend over 200 hours talking with their team and loosely mentoring them (which to be clear Nate did, it's just difficult in his position and so was only mildly successful).

I'm not sure kindness/warmth should even be a norm because it's pretty difficult to define.

Reply
[-]TekhneMakre1814

The details matter here; I don't feel I can guess from what you've said whether we'd agree or not.

For example:

Tam: says some idea about alignment

Newt: says some particular flaw "...and this is an instance of a general problem, which you'll have to address if you want to make progress..." gestures a bit at the general problem

Tam: makes a tweak to the proposal that locally addresses the particular flaw

Newt: "This still doesn't address the problem."

Tam: "But it seems to solve the concrete problem, at least as you stated it. It's not obvious to me that there's a general problem here; if we can solve instances of it case-by-case, that seems like a lot of progress."

Newt: "Look, we could play this game for some more rounds, where you add more gears and boxes to make it harder to see that there's a problem that isn't being addressed at all, and maybe after a few rounds you'll get the point. But can we just skip ahead to you generalizing to the class of problem, or at least trying to do that on your own?"

Tam: feels dismissed/disrespected

I think Newt could have been more graceful and more helpful, e.g. explicitly stating that he's had a history of conversations like this, and setting boundar... (read more)

Reply1
[-]Viliam119

You can choose to ignore all these "unreasonable norms", but they still have consequences. Such as people thinking you are an asshole. Or leaving the organization because of you. It is easy to underestimate these costs, because most of the time people won't tell you (or they will, but you will ignore them and quickly forget).

This is a cost that people working with Nate should not ignore, even if Nate does.

I see three options:

  • try making Nate change -- this may not be possible, but I think it's worth trying;
  • isolate Nate from... well, everyone else, except for volunteers who were explicitly warned;
  • hire a separate person whose full time job will be to make Nate happy.

Anything else, I am afraid, will mean paying the costs and most likely being in denial about them.

Reply
3Said Achmiz
I see at least two other options (which, ideally, should be used in tandem): * don’t hire people who are so terribly sensitive to above-average blutness * hire managers who will take care of ops/personnel problems more effectively, thus reducing the necessity for researchers to navigate interpersonal situations that arise from such problems
1Viliam
If I translate it mentally to "don't hire people from the bottom 99% of thick skin", I actually agree. Though they may be difficult to find, especially in combination with other requirements. Are you available for the job? ;-)
2Said Achmiz
Do you really think it’d take 99th percentile skin-thickness to deal with this sort of thing without having some sort of emotional breakdown? This seems to me to be an extraordinary claim. While I probably qualify in this regard, I don’t think that I have any other relevant qualifications.
[-]Ben Pace283

My experience is that people who I think of as having at least 90th percentile (and probably 99th if I think about it harder) thick-skin have been brought to tears from an intense conversation with Nate.

My guess is that this wouldn't happen for a lot of possible employees from the broader economy, and this isn't because they've got thicker skin, but it's because they're not very emotionally invested in the organization's work, and generally don't bring themselves to their work enough to risk this level of emotion/hurt.

Reply
[-]Said Achmiz154

My experience is that people who I think of as having at least 90th percentile (and probably 99th if I think about it harder) thick-skin have been brought to tears from an intense conversation with Nate.

This is a truly extraordinary claim! I don’t know what evidence I’d need to see in order to believe it, but whatever that evidence is, I sure haven’t seen it yet.

My guess is that this wouldn’t happen for a lot of possible employees from the broader economy, and this isn’t because they’ve got thicker skin, but it’s because they’re not very emotionally invested in the organization’s work, and generally don’t bring themselves to their work enough to risk this level of emotion/hurt.

This just can’t be right. I’ve met a decent number of people who are very invested in their work and the mission of whatever organization they’re part of, and I can’t imagine them being brought to tears by “an intense conversation” with one of their co-workers (nor have I heard of such a thing happening to the people I have in mind).

Something else is going on here, it seems to me; and the most obvious candidate for what that “something else” might be is simply that your view of what the distribution of “thick-skinned-ness” is like, is very mis-calibrated.

Reply1
7TekhneMakre
To me the obvious candidate is that people are orienting around Nate in particular in an especially weird way.
6Ben Pace
(Don't know why some folks have downvoted the above comment, seems like a totally normal epistemic state for Person A not to believe what Person B believes about something after simply learning that Person B believes it, and to think Person B is likely miscalibrated. I have strong upvoted the comment back to clearly positive.)
[-]So8res2111

In academia, for instance, I think there are plenty of conversations in which two researchers (a) disagree a ton, (b) think the other person's work is hopeless or confused in deep ways, (c) honestly express the nature of their disagreement, but (d) do so in a way where people generally feel respected/valued when talking to them.

My model says that this requires them to still be hopeful about local communication progress, and happens when they disagree but already share a lot of frames and concepts and background knowledge. I, at least, find it much harder when I don't expect the communciation attempt to make progress, or have positive effect.

("Then why have the conversation at all?" I mostly don't! But sometimes I mispredict how much hope I'll have, or try out some new idea that doesn't work, or get badgered into it.)

Some specific norms that I think Nate might not be adhering to:

  • Engaging with people in ways such that they often feel heard/seen/understood
  • Engaging with people in ways such that they rarely feel dismissed/disrespected
  • Something fuzzy that lots of people would call "kindness" or "typical levels of warmth"

These sound more to me like personality traits (that mem... (read more)

Reply
2Viliam
It seems to me that in theory it should be possible to have very unusual norms and make it work, but that in practice you and your organization horribly underestimate how difficult it is to communicate such things clearly (more than once, because people forget or don't realize the full implications at the first time). You assume that the local norms were made perfectly clear, but they were not (expecting short inferential distances, double illusion of transparency, etc.). Did you expect KurtB to have this kind of reaction, to post this kind of comment, and to get upvoted? If the answer is no, it means your model is wrong somewhere. (If the answer is yes, maybe you should print that comment, and give a copy to all new employees. That might dramatically reduce a possibility of misunderstanding.)
2TurnTrout
My original comment is not talking about communication norms. It's talking about "social norms" and "communication protocols" within those norms. I mentioned "basic respectfulness and professionalism." 
[-]TekhneMakre219

But I think some people possess the skill of "being able to communicate harsh truths accurately in ways where people still find the interaction kind, graceful, respectful, and constructive." And my understanding is that's what people like TurnTrout are wishing for.

This is a thing, but I'm guessing that what you have in mind involves a lot more than you're crediting of not actually trying for the crux of the conversation. As just one example, you can be "more respectful" by making fewer "sweeping claims" such as "you are making such and such error in reasoning throughout this discussion / topic / whatever". But that's a pretty important thing to be able to say, if you're trying to get to real cruxes and address despair and so on.

Reply
[-]TurnTrout170

But I think some people possess the skill of "being able to communicate harsh truths accurately in ways where people still find the interaction kind, graceful, respectful, and constructive." And my understanding is that's what people like TurnTrout are wishing for.

Kinda. I'm advocating less for the skill of "be graceful and respectful and constructive" and instead looking at the lower bar of "don't be overtly rude and aggressive without consent; employ (something within 2 standard deviations of) standard professional courtesy; else social consequences." I want to be clear that I'm not wishing for some kind of subtle mastery, here. 

Reply
[-]TurnTrout*213

I'm putting in rather a lot of work (with things like my communication handbook) to making my own norms clearer, and I follow what I think are good meta-norms of being very open to trying other people's alternative conversational formats.

Nate, I am skeptical.

As best I can fathom, you put in very little work to proactively warn new hires about the emotional damage which your employees often experience. I've talked to a range of people who have had professional interactions with you, both recently and further back. Only one of the recent cases reported that you warned them before they started working with you. 

In particular, talking to the hires themselves, I have detected no evidence that you have proactively warned most of the hires[1] you've started working with since July 2022, which is when:

  1. I told you that your anger and ranting imposed unexpected and large costs on me,
  2. And you responded with something like "Sorry, I'll make sure to tell people I have research conversations with -- instead of just my formal collaborators. Obvious in hindsight."

And yet you apparently repeatedly did not warn most of your onboarded collaborators.

EDIT: The original version of this comment c... (read more)

Reply
[-]So8res*2212

On the facts: I'm pretty sure I took Vivek aside and gave a big list of reasons why I thought working with me might suck, and listed that there are cases where I get real frustrated as one of them. (Not sure whether you count him as "recent".)

My recollection is that he probed a little and was like "I'm not too worried about that" and didn't probe further. My recollection is also that he was correct in this; the issues I had working with Vivek's team were not based in the same failure mode I had with you; I don't recall instances of me getting frustrated and bulldozey (though I suppose I could have forgotten them).

(Perhaps that's an important point? I could imagine being significantly more worried about my behavior here if you thought that most of my convos with Vivek's team were like most of my convos with you. I think if an onlooker was describing my convo with you they'd be like "Nate was visibly flustered, visibly frustrated, had a raised voice, and was being mean in various of his replies." I think if an onlooker was describing my convos with Vivek's team they'd be like "he seemed sad and pained, was talking quietly and as if choosing the right words was a struggle, and would o... (read more)

Reply1
[-]Vivek Hebbar222

I've been asked to clarify a point of fact, so I'll do so here:

My recollection is that he probed a little and was like "I'm not too worried about that" and didn't probe further.

This does ring a bell, and my brain is weakly telling me it did happen on a walk with Nate, but it's so fuzzy that I can't tell if it's a real memory or not.  A confounder here is that I've probably also had the conversational route "MIRI burnout is a thing, yikes" -> "I'm not too worried, I'm a robust and upbeat person" multiple times with people other than Nate.

In private correspondence, Nate seems to remember some actual details, and I trust that he is accurately reporting his beliefs.  So I'd mostly defer to him on questions of fact here.

I'm pretty sure I'm the person mentioned in TurnTrout's footnote.  I confirm that, at the time he asked me, I had no recollection of being "warned" by Nate but thought it very plausible that I'd forgotten.

Reply31
7TurnTrout
This is a slight positive update for me. I maintain my overall worry and critique: chats which are forgettable do not constitute sufficient warning.  Insofar as non-Nate MIRI personnel thoroughly warned Vivek, that is another slight positive update, since this warning should reliably be encountered by potential hires. If Vivek was independently warned via random social connections not possessed by everyone,[1] then that's a slight negative update.  1. ^ For example, Thomas Kwa learned about Nate's comm doc by randomly talking with a close friend of Nate's, and mentioning comm difficulties.
[-]TurnTrout1010

I think I'd also be more compelled by this argument if I was more sold on warnings being the sort of thing that works in practice.

Like... (to take a recent example) if I'm walking by a whiteboard in rosegarden inn, and two people are like "hey Nate can you weigh in on this object-level question", I don't... really believe that saying "first, be warned that talking techincal things with me can leave you exposed to unshielded negative-valence emotions (frustration, despair, ...), which some people find pretty crappy; do you still want me to weigh in?" actually does much. I am skeptical that people say "nope" to that in practice.

I think there are several critical issues with your behavior, but I think the most urgent is that people often don't know what they're getting into. People have a right to make informed decisions and to not have large, unexpected costs shunted onto them. 

It's true that no one has to talk with you. But it's often not true that people know what they're getting into. I spoke out publicly because I encountered a pattern, among my friends and colleagues, of people taking large and unexpected emotional damage from interacting with you. 

If our July interact... (read more)

Reply
2TurnTrout
(I don't know who strong disagree-voted the parent comment, but I'm interested in hearing what the disagreement is. I currently think the comment is straightforwardly correct and important.)
[-]Zack_M_Davis216

The 9-karma disagree-vote is mine. (Surprise!) I thought about writing a comment, and then thought, "Nah, I don't feel like getting involved with this one; I'll just leave a quick disagree-vote", but if you're actively soliciting, I'll write the comment.

I'm wary of the consequences of trying to institute social norms to protect people from subjective emotional damage, because I think "the cure is worse than the disease." I'd rather develop a thick skin and take responsibility for my own emotions (even though it hurts when some people are mean), because I fear that the alternative is (speaking uncharitably) a dystopia of psychological warfare masquerading as kindness in which people compete to shut down the expression of perspectives they don't like by motivatedly getting (subjectively sincerely) offended.

Technically, I don't disagree with "people should know what they're getting into" being a desirable goal (all other things being equal), but I think it should be applied symmetrically, and it makes sense for me to strong-disagree-vote a comment that I don't think is applying it symmetrically: it's not fair if "fighty" people need to to make lengthy disclaimers about how their blunt... (read more)

Reply1
1[comment deleted]
2TurnTrout
You told me you would warn people, and then did not.[1]  1. ^ Do I have your permission to quote the relevant portion of your email to me?
8So8res
I warned the immediately-next person. It sounds to me like you parsed my statement "One obvious takeaway here is that I should give my list of warnings-about-working-with-me to anyone who asks to discuss their alignment ideas with me, rather than just researchers I'm starting a collaboration with." as me saying something like "I hereby adopt the solemn responsibility of warning people in advance, in all cases", whereas I was interpreting it as more like "here's a next thing to try!". I agree it would have been better of me to give direct bulldozing-warnings explicitly to Vivek's hires.
[-]TurnTrout140

Here is the statement:

(One obvious takeaway here is that I should give my list of warnings-about-working-with-me to anyone who asks to discuss their alignment ideas with me, rather than just researchers I'm starting a collaboration with. Obvious in hindsight; sorry for not doing that in your case.)

I agree that this statement does not explicitly say whether you would make this a one-time change or a permanent one. However, the tone and phrasing—"Obvious in hindsight; sorry for not doing that in your case"—suggested that you had learned from the experience and are likely to apply this lesson going forward. The use of the word "obvious"—twice—indicates to me that you believed that warnings are a clear improvement.

Ultimately, Nate, you wrote it. But I read it, and I don't really see the "one-time experiment" interpretation. It just doesn't make sense to me that it was "obvious in hindsight" that you should... adopt this "next thing to try"..? 

Reply
[-]So8res*251

I did not intend it as a one-time experiment.

In the above, I did not intend "here's a next thing to try!" to be read like "here's my next one-time experiment!", but rather like "here's a thing to add to my list of plausible ways to avoid this error-mode in the future, as is a virtuous thing to attempt!" (by contrast with "I hereby adopt this as a solemn responsibility", as I hypothesize you interpreted me instead).

Dumping recollections, on the model that you want more data here:

I intended it as a general thing to try going forward, in a "seems like a sensible thing to do" sort of way (rather than in a "adopting an obligation to ensure it definitely gets done" sort of way).

After sending the email, I visualized people reaching out to me and asking if i wanted to chat about alignment (as you had, and as feels like a reconizable Event in my mind), and visualized being like "sure but FYI if we're gonna do the alignment chat then maybe read these notes first", and ran through that in my head a few times, as is my method for adopting such triggers.

I then also wrote down a task to expand my old "flaws list" (which was a collection of handles that I used as a memory-aid for having the "ways... (read more)

Reply11
2TurnTrout
I appreciate the detail, thanks. In particular, I had wrongly assumed that the handbook had been written much earlier, such that even Vivek could have been shown it before deciding to work with you. This also makes more sense of your comments that "writing the handbook" was indicative of effort on your part, since our July interaction. Overall, I retain my very serious concerns, which I will clarify in another comment, but am more in agreement with claims like "Nate has put in effort of some kind since the July chat."  Noting that at least one of them read the handbook because I warned them and told them to go ask around about interacting with you, to make sure they knew what they were getting into.
4So8res
Yep! I've also just reproduced it here, for convenience:
[-]Zack_M_Davis2922

Surely the problem with "state non-obvious takes in an obvious tone, and decline to elaborate" bulldozing is that it violates the maxim that beliefs should be supported with arguments and evidence? (And the reason for the maxim is that even the smartest human experts aren't infallible; if not subjected to the rigor of the debate algorithm, they're going to get things wrong.) It seems misplaced to focus on emotionally bad experiences and punctured excitement.

Reply11
[-]TurnTrout*1815

That's relevant, but I'm largely not discussing group epistemics. I'm discussing the community impact of social norms. That impact is measured in human well-being, productivity, and happiness, as well as the height of the sanity waterline. Concretely—if I treat my colleagues in a rude and angry manner, that imposes costs on them. In that situation, whether or not I'm making correct verbal claims, that's generally not a good community to be a part of, and it's not a good way to treat people. 

Emotionally bad experiences are an extremely relevant quantity to discuss.

(I don't expect to engage further due to our past unfruitful interactions on similar topics.)

EDIT: Also, clarification that the "bulldozing" incident did not primarily consist of "state non-obvious takes in an obvious tone, and decline to elaborate."

Reply
3Said Achmiz
What did it consist of? Have you described it somewhere?
[-]Jeremy Gillen261

"they should clearly communicate their non-respectful/-kind alternative communication protocols beforehand, and they should help the other person maintain their boundaries;"

Nate did this.

By my somewhat idiosyncratic views on respectful communication, Nate was roughly as respectful as Thomas Kwa. 

I do seem to be unusually emotionally compatible with Nate's style of communication though.

Reply11
[-]Thomas Kwa*222

By my somewhat idiosyncratic views on respectful communication, Nate was roughly as respectful as Thomas Kwa.

I agree with a couple of caveats. Even though Nate did try pretty hard and even send us a guide on communicating with him, he's still difficult (for me) to work with even after reading the guide. I think I'm more capable of minimizing unintentional emotional damage than Nate is just due to our demeanors.

Edit: I misremembered; Nate didn't send us the guide proactively, someone who knew him sent it to us after we asked for tips on communicating with Nate.

Reply11
[-]Bird Concept3418

even send us a guide on communicating with him

Woah! That's like 10x more effort than I expect >90% of difficult-to-communicate-with people will go through. 

Kudos to Nate for that.

Reply
[-]Lukas_Gloor182

Woah! That's like 10x more effort than I expect >90% of difficult-to-communicate-with people will go through. 

Kudos to Nate for that.

There are things that I really like about the document, but I feel like I'd need to know more about its reason for being created to say whether this deserves kudos.

It seems plausible that the story went something like this: "Nate had so much social standing that he was allowed/enabled to do what most 'difficult to interact with' people couldn't, namely to continue in their mannerisms without making large changes, and still not suffer from a reduction of social standing. Partly to make this solution palatable to others and to proactively address future PR risks and instances of making people sad (since everyone already expected/was planning for more such instances to come up), Nate wrote this document."

If an org is going to have this sort of approach to its most senior researcher, it's still better to do it with a document of this nature than without.

But is this overall a great setup and strategy? I'm doubtful. (Not just for the org as a whole, but also long-term for Nate himself.)

Reply
4Viliam
I think we now have an experimental verification that the document, no matter how impressive, doesn't always achieve its intended goal. So, the strategy needs an update.
1[anonymous]
Three points that might be somewhat revealing: 1. There was never an ask for reciprocal documents from employees. "Here's a document describing how to communicate with me. I'd appreciate you sending me pointers on how to communicate with you, since I am aware of my communication issues." was never considered. 2. There are multiple independent examples of people in various capacities, including his girlfriend, expressing that their opinions were not valued, and a clear hierarchical model was in play. 3. The more humble "my list of warnings" was highlighted immediately as justification but never broadcast broadly,  and there seems to be no cognizance that it's not something anyone else would ever take upon themselves to share.
8Raemon
So on one hand...  your bullet-points there are maybe pointing in a helpful direction. And I think my overall take right now is "however much effort Nate has previously put into improving on his communications or comms-onboarding, probably he (or MIRI) should put more." But, your phrasing here feels a bit like a weird demand for exceptional rigor.  Like, although I think Nate is pretty high on "can feel intense to interact with", it's not that weird for a company to have an intense manager, and I've never heard of companies-with-intense-managers having this sort of doc at all. And I know a bunch of people who are intense to interact with in regular, interpersonal interactions (i.e. while dating), and they also often don't have docs explaining that.  So, it feels pretty weird (and not particularly "revealing" of anything) that Nate made a pretty novel type of doc... and didn't (yet) do some additional followup steps with it.
[-]TurnTrout125

Like, although I think Nate is pretty high on "can feel intense to interact with", it's not that weird for a company to have an intense manager, and I've never heard of companies-with-intense-managers having this sort of doc at all. And I know a bunch of people who are intense to interact with

(I think that "intense" is euphemizing.)

Reply
[-][anonymous]118

But, your phrasing here feels a bit like a weird demand for exceptional rigor. 

No - the opposite. I was implying that there's clearly a deeper underpinning to these patterns that any amount of rigor will be insufficient in solving, but my point has been articulated within KurtB's excellent later comment, and solutions in the earlier comment by jsteinhardt.

it's not that weird for a company to have an intense manager

I agree; that's very true. However, this usually occurs in companies that are chasing zero-sum goals. Employees treated in this manner might often resort to a combination of complaining to HR, being bound by NDAs, or biting the bullet while waiting for their paydays. It's just particularly disheartening to hear of this years-long pattern, especially given the induced discomfort in speaking out and the efforts to downplay, in an organization that publicly aims to save the world.

Reply
7Ben Pace
I found that an endearing document to read. A lot of clear self-reflection communicated explicitly, describing both endorsed and unendorsed self-properties, in an effort to help others communicate and interface.
[-]Viliam150

The document is interesting, but how well does it describe Nate's actual behavior? Can you find the parts that correspond to this:

He didn’t exactly yell at me and my fellow ops coworker, according to my imaginary decibelmeter, but he was indisputably hostile and aggressive, and obviously uninterested in 2-way communication.

I saw Nate in the office kitchen later that day (a Saturday) and thought it was an appropriate time to bring up again that I was having trouble with our available pump. I didn’t know how to–“Learn!” he snapped and then stormed out of the room.

He got really angry at me when the rest of the office outvoted him on the choice of lunch catering.

Reply
8TurnTrout
As I wrote elsewhere:
4Ben Pace
The 12 section headings under "Failure modes" are: Confidence, Frustration, Lashing out, Condescension, Nonrespect, Dismissal, Disdain, Disbelieving you, Harshness, Tunnel vision, Retaliation, and Stoneface. The elements in your quoted section appear to me to come up here (e.g. Lashing out, Tunnel vision, Dismissal, etc). I don't say it's a successful document. My first guess is that a more successful version of the doc (edit: or at least a doc that successfully conveys what to expect when interacting with the author) would be shorter and focus on giving a more grounded sense of "here are concrete instances of the peak good and bad interactions I've had with people, and some sense of the modal interaction type". It might include content like "I've talked to N people who've said they've had life-changingly positive interactions with me, and M people who've said as a result they were really substantively hurt by interacting with me and wish to indefinitely avoid me, here's some properties of the interactions they described, also here's a survey on general properties of how people find me to be in conversation, split out between friends and colleagues". However I think that the sort of doc I'm describing is ~unheard of in any setting and way more effort than ~anyone I'm aware of has put into this sort of widespread expectation setting with lots of colleagues and collaborators (I'd say the same for the linked doc that Nate drafted), and people typically do not have much obligation to let people know about bad interactions. Heck, in many countries people can get criminal records expunged so that they don't have to inform their future employers about them, which is worlds apart from handing someone a doc listing times when people they've talked to have felt burned by the interaction, which reads to me like a standard being demanded elsethread.
[-]Raemon208

I wanted to briefly note for now that LW moderators are tracking this thread. Some of us may end up writing more of our thoughts later. 

I think it's good for people to be sharing their experiences. I'm glad Turntrout and KurtB shared their stories. I'm glad So8res responded with his experience/context. I generally feel quite good about people sharing information like this. 

I feel a bit wary of any specific recommendations necessarily helping – I think often situations like this are just actually pretty tricky/nuanced and involve a lot more significant tradeoffs than it may seem. (Still seems fine for people to write up their own wishes, or guesses about what would help, I just think it's just useful to keep in mind that it may be more complicated that it seems at first glance)

I haven't personally interacted with Nate much, but I've had some experiences similar to things Turntrout described with other people I respect. I might have some useful things to say about some of the patterns involved, thought it may take awhile to write up and might end up being a separate top-level post.

Reply
[-]TurnTrout117

I think often situations like this are just actually pretty tricky/nuanced and involve a lot more significant tradeoffs than it may seem. (Still seems fine for people to write up their own wishes, or guesses about what would help, I just think it's just useful to keep in mind that it may be more complicated that it seems at first glance)

For my part, I predict (85%) that this sentiment will age at least somewhat poorly, looking back one year from now. (Not operationalizing it super well, just trying to do better than nothing.)

EDIT: Obviously there's going to be some nuance, etc. This prediction is a public-facing result of a longer private dialogue I had with other parties.

Reply1
[-]peterbarnett155

I think it would have been useful to be informed about Nate's communication style and reputation here before starting the project, although I doubt this would have changed anyone's decision to work on the project (I haven't checked with the others and they might think differently). I think it's kind of hard to see how bad/annoying/sad this is until you're in it.
This also isn't to say that ex post I think joining/doing the project was a bad idea. 

Reply2
5TurnTrout
Can you say when you were made aware of Nate's communication style? It sounds here like he didn't tell you until after you'd started working together, but Jeremy's comment seems to indicate that you all were informed beforehand?
[-]peterbarnett130

I think I became most aware in December 2022, during our first set of in-person meetings. Vivek and Thomas Kwa had had more interaction with Nate before this and so might have known before me. I have some memory of things being a bit difficult before the December meetings, but I might have chalked this up to not being in-person, I don't fully remember. 
It was after these meetings that we got the communication guide etc. 
Jeremy joined in May 2023, after the earlier members of the team knew about communication stuff and so I think we were able to tell him about various difficulties we'd had. 

Reply11
[-]TurnTrout*106

My negative-feeling meeting with Nate was in July 2022, after which I emailed something like "hey I didn't consent to that and wouldn't have agreed to talk if I knew ahead of time" and his reply included something like "sorry, I'll at least make sure to forewarn future people first, even if they aren't my formal collaborators; obvious in retrospect." 

Because a) July is well before December and b) the information you just provided, it seems like he does not reliably warn his collaborators.

As Nate clarified at the end of the dialogue, he apparently considered this collaboration to be a "sad/grudging shot" and that context explains "suboptimalities" of your working relationship. But from my limited information and impressions thus far, I don't think that "neglecting to warn incoming junior researchers" falls under "reasonable suboptimality." I could imagine him thinking, "hm they're Vivek's friends, and Vivek has talked to me, so he probably let them know already." However, I'd personally consider this to be a serious lapse, given the potential for damage.

(But maybe there's some more sympathetic perspective here which I'm missing..? I'm happy to update back upwards, given additional clarification or facts.)

Reply
[-]So8res*242

Less "hm they're Vivek's friends", more "they are expressly Vivek's employees". The working relationship that I attempted to set up was one where I worked directly with Vivek, and gave Vivek budget to hire other people to work with him.

If memory serves, I did go on a long walk with Vivek where I attempted to enumerate the ways that working with me might suck. As for the others, some relevant recollections:

  • I was originally not planning to have a working relationship with Vivek's hires. (If memory serves, there were a few early hires that I didn't have any working relationship with at any point during their tenure.) (If memory serves further, I explicitly registered pessimism, to Vivek, about me working with some of his hires.)
  • I was already explicitly relying on Vivek to do vetting and make whatever requests for privacy he wanted to, which my brian implicitly lumped in with "give caveats about what parts of the work might suck".
  • The initial work patterns felt to me more like Vivek saying "can one of my hires join the call" than "would you like to also do research with my hires directly", which didn't trigger my "give caveats personally" event (in part because I was implicitly expe
... (read more)
Reply2
4[anonymous]
TurnTrout, if you're comfortable sharing, I'd be curious to hear more about the nature of your interaction with Nate in July 2022. Separately, my current read of this thread is something like "I wish it was easier for Nate to communicate with people, but it seems (at least to me) like the community is broadly aware that Nate can be difficult to communicate with & I think his reputation (at least in the AIS circles I'm in) already matches this. Also, it seems like he (at least sometimes) tries to be pretty clear/explicit in warning people about his communication/mentoring style." I do think the "giving up" vibe you mention in the original comment is tracking something real. I think the AIS field would benefit if Nate woke up one day and found ways to communicate with people that made them feel more heard/respected/validated and less dismissed/misunderstood/pained. I'm guessing Nate would agree with this, though, and I'm guessing he's probably tried a lot of things and simply hasn't found strategies that are effective (and that don't trade off against other desireada, like honesty or Nate energy/motivation). I'm not sure though-- I mostly believe this because other people have told it to me & because it seems like he put in a fair amount of effort/reflection in his communication guide. I guess my TLDR is something like "I think you're pointing at a real problem, and it would indeed be great if Nate were better at communicating, but I also feel like there's something important about Nate's perspective that feels missing here."
[-]TurnTrout1012

it seems (at least to me) like the community is broadly aware that Nate can be difficult to communicate with & I think his reputation (at least in the AIS circles I'm in) already matches this. Also, it seems like he (at least sometimes) tries to be pretty clear/explicit in warning people about his communication/mentoring style.

I disagree on both points. I wasn't aware before talking with him, and I'd been in alignment and around the Bay for years. Some of my MATS mentees had been totally unaware and were considering engaging with him about something. Peter wasn't informed before starting a work relationship, and Nate didn't tell Peter (and maybe Thomas either) before working with them. 

Reply
2Quinn
I second this--- I skimmed part of nate's comms doc, but it's unclear to me what turntrout is talking about unless he's talking about "being blunt"--- it sounds that overall there's something other than bluntness going on, cuz I feel like we already know about bluntness / we've thought a lot about upsides and downsides of bluntness people before.
6Raemon
So, I don't know what actually happened here. But I at least want to convey support for:  "There are ways of communicating other than being blunt that can... unsettlingly affect you [or, at least, some people], which are hard to explain, and their being hard to explain makes it psychologically harder to deal with because when you try to explain it or complain about it people are kinda dismissive." (I'm not expressing a strong opinion here about whether Nate should have done something different in this case, or what the best way for Turntrout, Vivek's team, or others should relate to it. I'm just trying to hold space for "I think there's a real thing people should be taking seriously as a possibility and not just rounding off to 'Turntrout should have thicker skin' or something) I have some guesses about the details but they're mostly informed by my interactions with people other than Nate, which give me sort of an existence proof, and I'm wary of speculating myself here without having actually had this sort of conversation with Nate.
4TekhneMakre
I really wish it were possible for this conversation to address what the affected people are coming in with. I suspect (from priors and the comments here) that there are social effects that are at core not located in either Nate or TurnTrout that result in this.
3TurnTrout
I might reply later, but I want to note that Nate's comms doc doesn't really track my (limited) experience of what it feels like to talk with Nate, and so (IMO) doesn't make great sense as a baseline of "what happened?".
[-]Quinn145

Ah yeah. I'm a bit of a believer in "introspection preys upon those smart enough to think they can do it well but not smart enough to know they'll be bad at it"[1], at least to a partial degree. So it wouldn't shock me if a long document wouldn't capture what matters.

  1. epistemic status: in that sweet spot myself ↩︎

Reply
5Richard_Ngo
+1, was gonna make this comment myself, but TurnTrout said it better.
1Raemon
Was this via public comments or private communications? (Regardless, sucks that this happened to you. But it changes my guesses what sorts of solutions would help)
8TurnTrout
This was a 1-on-1, in-person chat.
[-]jsteinhardt8146

While I am not close to this situation, I felt moved to write something, mostly to support junior researchers and staff such as TurnTrout, Thomas Kwa, and KurtB who are voicing difficult experiences that may be challenging for them to talk about; and partly because I can provide perspective as someone who has managed many researchers and worked in a variety of research and non-research organizations and so can more authoritatively speak to what behaviors are 'normal' and what patterns tend to lead to good or bad outcomes. Caveat that I know very little about any internal details of MIRI, but I am still reasonably confident of what I'm saying based on general patterns and experience in the world.

Based on reading Thomas Kwa's experience, as well as KurtB's experience, Nate Soares' behavior is far outside any norms of acceptable behavior that I'd endorse. Accepting or normalizing this behavior within an organization has a corrosive effect on the morale, epsistemics, and spiritual well-being of its members. The morale effects are probably obvious, but regarding epistemics, leadership is significantly less likely to get useful feedback if people are afraid to cross them (psychological s... (read more)

Reply521
4frontier64
Has anyone claimed at this point that Nate yelled at them?
-35IlyaShpitser
[-]lisathiergart4428

I’m MIRI’s new research manager and I’d like to report back on the actions we’ve taken inside MIRI in response to the experiences reported above (and others). In fact I joined MIRI earlier this year in part because we believe we can do better on this. 

First off, I’d like to thank everyone in this thread for your bravery (especially @KurtB and @TurnTrout). I know this is not easy to speak about and I’d like you to know that you have been heard and that you have contributed to a real improvement. 

Second, I’d like to say that I, personally, as well as MIRI the org take these concerns very seriously and we’ve spent the intervening time coming up with internal reforms. Across MIRI research, comms and ops, we want every MIRI staff member to have a safe environment to work in and to not have to engage in any interactions they do not consent to. For my area of responsibility in research, I’d like to make a public commitment to firmly aim for this. 

To achieve this we’ve set up the following: 

  • Nate currently does not directly manage any staff. By default, all new research staff will be managed by me (Lisa) and don’t need to interact with Nate. Further, should he ever want
... (read more)
Reply431
[-]KurtB159

I'm very happy to read this. I feel lucky that Lisa works at MIRI, and I feel major gratitude to her and everyone else who contributed behind the scenes.

Reply1
[-]TurnTrout104

I'm also really glad to see this. Seems like a very positive set of changes. Props to MIRI for taking concrete and decisive action. 

Reply1
[-]peterbarnett3012

I want to say some things about the experience working with Nate, I’m not sure how coherent this will be.

Reflections on working with Nate

I think jsteinhardt is pretty correct when he talks about psychological safety, I think our conversations with Nate often didn’t feel particularly “safe”, possibly because Nate assumes his conversation partners will be as robust as him.

Nate can pretty easily bulldoze/steamroll over you in conversation, in a way that requires a lot of fortitude to stand up to, and eventually one can just kind of give up. This could happen if you ask a question (and maybe the question was confused in some way) and Nate responds with something of a rant that makes you feel dumb for even asking the question. Or often we/I felt like Nate had assumed we were asking a different thing, and would go on a spiel that would kind of assume you didn’t know what was going on. This often felt like rounding your statements off to the dumbest version. I think it often did turn out that the questions we asked were confused, this seems pretty expected given that we were doing deconfusion/conceptual work where part of the aim is to work out which questions are reasonable to ask.

I thin... (read more)

Reply82
[-]Zvi239

I am curious to what extent you or Nate think I understand that frame? And how easy it would be to help me fully get it? I am confused about how confused I am.

Reply
[-]TurnTrout*133

even when you try to point people at the thing to look at they keep turning to look at something else (something easier, less scary, more approachable, but useless).

I understand why someone might be frustrated in his position, and it's fine to feel however. However, I want to push back on any implicit advancement of sentiments like "his intense feelings justify the behavior."[1]  The existing discussion has focused a lot on the social consequences of e.g. aggressive and mean behavior. I'll now take a more pragmatic view.

If you want to convince people of something, you should not severely punish them for talking to you. For example, I'd be more open to hearing Nate's perspective if he had conducted himself in an even somewhat reasonable manner. As I wrote in my original comment:

[Nate's behavior] killed my excitement for engaging with the MIRI-sphere.

Even from a pragmatic "world-saving" perspective, and given Nate's apparent views, Nate's behavior still doesn't make sense to me. It doesn't seem like he's making some clever but uncooperative trade whereby he effectively persuades people of true stuff, albeit at (sometimes) large emotional cost to others. It seems more like "relat... (read more)

Reply
5gjm
I'm struck by this: This might be true, and if true it might be very important. But, outside view, I think the track record of people/organizations claiming things along the lines of "we and we alone have the correct understanding of X, and your only way to understand X is to seek our wisdom" is pretty bad, and that of people/organizations about whom other people say "they and they alone have the correct understanding, etc." isn't much better. I know that MIRI expresses concern about the dangers of spreading their understanding of things that might possibly be used to advance AI capabilities. But if an important thing they have is a uniquely insightful way of framing the alignment problem then that seems like the sort of thing that (1) is very unlikely to be dangerous to reveal, (2) could be very valuable to share with others, and (3) if so shared would (a) encourage others to take MIRI more seriously, if indeed it turns out that they have uniquely insightful ways of thinking about alignment and (b) provide opportunities to correct errors they're missing, if in fact what they have is (something like) plausible rhetoric that doesn't stand up to close critical examination.
[-]Max H2218

I think the 2021 MIRI Conversations and 2022 MIRI Alignment Discussion sequences are an attempt at this. I feel like I have a relatively good handle on their frame after reading those sequences, and I think the ideas contained within are pretty insightful. 

Like Zvi, I might be confused about how confused I am, but I don't think it's because they're trying to keep their views secret. Maybe there's some more specific capabilities-adjacent stuff they're not sharing, but I suspect the thing the grandparent is getting at is more about a communication difficulty that in practice seems to be overcome mostly by working together directly, as opposed to the interpretation that they're deliberately not communicating their basic views for secrecy reasons.

(I also found Eliezer's fiction helpful for internalizing his worldview in general, and IMO it is also has some pretty unique insights.)

Reply3
[-]Raemon233

I haven't really interacted with Nate in the sorts of contexts getting discussed here. But one of Turntrout's comments reminded me of some similar experiences I've had with other people that I wanted to write up.

I think there are three high level points:

  1. "Relative Forcefulness-of-personality" is variable to be aware of.
  2. "Suddenly having to defend boundaries you didn't expect to have to" is a particular way that it can be difficult/scary interacting with someone more forceful than you. (Corollary: it's scarier to interact with forceful people who do a lot of unpredictable things)
  3. "Forcefully telling people how they think" can feel particularly violating, to somewho is less self-assured, because it can leave them doubting their sanity, and undermining their ability to figure out what they want.

I feel sold on "these are things to be aware of." I don't necessarily prescribe any particular takeaway with them.

I think rather than write a giant effort-comment I'm going to write pieces at a time. Fleshing out point #1 a bit:

People vary in how forceful a personality they are. I don't think there's a single "correct" amount of forcefulness to be, but whenever there's a mismatch it can leave some... (read more)

Reply
8Raemon
Okay, now shifting to give more details about #3: "Forcefully telling people how they think" Here's an example situation. One day, Alex, Bob and me were all having a negotiation, trying to figure out what to do about Dave. We all agreed Dave had done something wrong. Alex and Bob both thought Dave was... something like "overwhelmingly in the wrong, and had defected first." I thought that we'd all been kinda complicit in an escalation spiral, and I cut Dave more slack. At a point when Alex, Bob and I were all in a room talking about Dave, and I was pointing out a way that Bob had escalated things with Dave. Alex said, forcefully, with a bit of edge in his voice, "Ray is only saying that because he has a pathological need to listen to Dave and [I can't remember the exact phrasing here]. And... ...I think Alex was at least moderately right. There is some way in which I feel compelled to take people's stories at face value, and be kind of a bleeding heart for them, and put in extra work trying to cater to them. Often at my own expense and the expense of other important stuff I care about. I think this has in fact been the wrong call, and I've made significant updates over the years about being more stone-hearted about it. But, also, in this case, I'm pretty sure this was at most 25% of what was going on (at least in this conversation). I remember some specific ways I updated about Dave that week, and I trust my deliberate introspection about it (even if I don't trust all of my background-process-introspection). This was all during a pretty tense time period, when we were all kinda exhausted. A few things that stick out here: 1. Alex spoke really confidently. I think in general this triggers some kind of "I should listen to what this person is saying" behavior. I also do respect their opinions a lot (often more than my own), so part of me was thinking "I dunno am I the crazy one here? Am I wrong about why I'm thinking this? I'm pretty sure I'm not wrong." I f
4Raemon
I guess another thing to note is that "people telling you that you are wrong about how you think" can also be an important part of breaking out of wrong, sticky narratives you have about yourself.  Perhaps annoyingly: many of the people who I have found very helpful for learning to "see the water I was swimming in, and take it as object" seem also actively destabilizing for some people around them. (i.e. I've gotten a lot of value from Brent, Ziz, Vassar and Geoff Anders. In each of those cases I didn't actually get too close for long, but I know other people who did and had various flavors of bad experience). Habryka's gloss on all this is "telling people they are wrong about what they think is high variance and should be treated as risky, but also has important upside." I feel like "high variance" is too positive a spin on it, but there's something important there.
4Said Achmiz
I’d like to point out that of these three things, #3 is one (and, note, the only one) which is traditionally frowned upon, according to commonly accepted rules of conversational conduct, independently of any views about how “forceful” one ought to be, etc. This seems to me to be instructive. (We have many words/phrases for it, too: “Bulverism”, “psychoanalyzing your conversation partner”, etc. These are not laudatory terms!)
4TurnTrout
Thanks for sharing these factors. To add: different orientations to the conversation can produce different forcefulness-of-personality. In the first half of my chat with Nate, I was rather cooperative and truth-seeking, and it mostly felt like Nate was steamrolling me. In the second half, I oriented towards it as an adversarial sparring match with someone who was trying to beat me, and began calling out a bunch of (IMO) shaky claims, and the conversation felt pretty even. 
[-]Oliver Sourbut*63

Note I have never met Nate, but have taken some amount of value from his writings. I have met KurtB briefly around a year ago and enjoyed some excellent conversations at the time. I can't remember if I've met TurnTrout but we've certainly interacted with medium success online. This comment is not meant to add any evidential weight to any part of anyone's previous statements.

I am reminded by some of these comments and threads (especially the one with TurnTrout and KurtB) of the 'super chicken' model - researchers selecting chickens for high egg output inadv... (read more)

Reply
Moderation Log
Curated and popular this week
139Comments