All of Malo's Comments + Replies

Agree. I think Google DeepMind might actually be the most forthcoming about this kind of thing, e.g., see their Evaluating Frontier Models for Dangerous Capabilities report.

Apple Music?

I’d certainly be interested in hearing about them, though it currently seems pretty unlikely to me that it would make sense for MIRI to pivot to working on such things directly as opposed to encouraging others to do so (to the extent they agree with Nate/EYs view here).

I think this a great comment, and FWIW I agree with, or am at least sympathetic to, most of it.

If you are on an airplane or a train, and you can suddenly work or watch on a real theater screen, that would be a big game. Travel enough and it is well worth paying for that, or it could even enable more travel.

Ben Thompson agrees in a followup (paywalled):

Vision Pro on an Airplane

I tweeted about this, but I think it’s worth including in the Update as a follow-up to last week’s review of the Vision Pro: I used the Vision Pro on an airplane over the weekend, sitting in economy, and it was absolutely incredible. I called it “life-changing” on Twitter, and

There is no sign that anyone plans to actually offer MLB or other games in this mode.

NBA seems somewhat bullish.

That may be right but then the claim is wrong. The true claim would be "RSPs seem like a robustly good compromise with people who are more optimistic than me".

IDK man, this seems like nitpicking to me ¯\_(ツ)_/¯. Though I do agree that, on my read, it’s technically more accurate.

My sense here is that Holden is speaking from a place where he considers himself to be among the folks (like you and I) who put significant probability on AI posing a catastrophic/existential risk in the next few years, and “people who have different views from mine” is referring to... (read more)

Responsible scaling policies (RSPs) seem like a robustly good compromise with people who have different views from mine

2. It seems like it's empirically wrong based on the strong pushback RSPs received so that at least you shouldn't call it "robustly", unless you mean a kind of modified version that would accommodate the most important parts of the pushback. 

FWIW, my read here was that “people who have different views from mine” was in reference to these sets of people:

• Some people think that the kinds of risks I’m worried about are far off, farfetched

Our reserves increased substantially in 2021 due to a couple of large crypto donations. 

At the moment we've got ~$20M.

FWIW, I approached Gretta about starting to help out with comms related stuff at MIRI, i.e., it wasn't Eliezer's idea.

Interesting, I don't think I knew about this post until I clicked on the link in your comment.

Quickly chiming in to add that I can imagine there might be some research we could do that could be more instrumentally useful to comms/policy objectives. Unclear whether it makes sense for us to do anything like that, but it's something I'm tracking.

1. Given Nate's comment: "This change is in large part an enshrinement of the status quo. Malo’s been doing a fine job running MIRI day-to-day for many many years (including feats like acquiring a rural residence for all staff who wanted to avoid cities during COVID, and getting that venue running smoothly). In recent years, morale has been low and I, at least, haven’t seen many hopeful paths before us." (Bold emphases are mine). Do you see the first bold sentence as being in conflict with the second, at all? If morale is low, why do you see that as an indica

• Does MIRI also plan to get involved in policy discussions (e.g. communicating directly with policymakers, and/or advocating for specific policies)?

We are limited in our ability to directly influence policy by our 501(c)3 status; that said, we do have some latitude there and we are exercising it within the limits of the law. See for example this tweet by Eliezer.

To expand on this a bit, I and a couple others at MIRI have been spending some time syncing up and strategizing with other people and orgs who are more directly focused on policy work themselves. We... (read more)

• Does MIRI need any help? (Or perhaps more precisely "Does MIRI need any help from the right kind of person with the right kind of skills, and if so, what would that person or those skills look like?")

Yes, I expect to be hiring in the comms department relatively soon but have not actually posted any job listings yet. I will post to LessWrong about it when I do.

That said, I'd be excited for folks who think they might have useful background or skills to contribute and would be excited to work at MIRI, to reach out and let us know they exist, or pitch us on why they might be a good addition to the team.

MIRI used to be focused on safety research, but now it's mostly trying to stop the march towards superintelligence, by presenting the case for the extreme danger of the current trajectory.

Yeah, given the current state of the game board we think that work in the comms/policy space seems more impactful to us on the margin, so we'll be focusing on that as our top priority and see how things develop, That won't be our only focus though, we'll definitely continue to host/fund research.

Sometimes quick org updates about team changes can be a little dry. ¯\(ツ)/¯

I expect you’ll find the next post more interesting :)

(Edit: fixed typo)

My read was that his comment was in response to this part at the end of the post:

There’s a lot more we hope to say about our new (and still evolving) strategy, and about our general thinking on the world’s (generally very dire) situation. But I don’t want those announcements to further delay sharing the above updates, so I’ve factored our 2023 strategy updates into multiple posts, beginning with this one.

First two of the six volumes are out: https://www.lesswrong.com/posts/NjFgqv8bzjhXFaELP/new-edition-of-rationality-from-ai-to-zombies

Update: Added an announcement of our newest hire, Edward Kmett, as well as a list of links to relatively recent work we've been doing in Agent Foundations, and updated the post to reflect the fact that Giving Tuesday is over (though our matching opportunity continues)!

It's my understanding, thought Oliver can of course correct me if I'm wrong, that the canonical domain will be lesswrong.com, and all lesserwrong.com/* links will redirect to lesswrong.com/*, to ensure that any links on the web to lesserwrong.com continue to work.

No.

Update 2:

Professional poker players Martin Crowley, Tom Crowley, and Dan Smith, in partnership with Raising for Effective Giving, have just announced a $1 million Matching Challenge and included MIRI among the 10 organizations they are supporting!

Also, we’ve hit our first fundraising target ($625,000)!

See here for more details.

Update 2: Professional poker players Martin Crowley, Tom Crowley, and Dan Smith, in partnership with Raising for Effective Giving, have just announced a $1 million Matching Challenge and included MIRI among the 10 organizations they are supporting!

Also, we’ve hit our first fundraising target ($625,000)!

See here for more details.

Awesome! Thanks so much :)

We just passed the 1/4 mark towards our first target! Fun fact, of the ~$200k raised so far in the fundraiser, ~65% of that has come from cryptocurrency dontions.

We just passed the 1/4 mark towards our first target! Fun fact, of the ~$200k raised so far in the fundraiser, ~65% of that has come from cryptocurrency dontions.

You might also want to check out Ketolent.

It's just a really big project. It's almost an order of magnitude longer then In Eq, and it was written in a way that makes it much more challenging to turn into a paper book. E.g., links are pretty important when reading the Sequences. Said another way, the task of getting a physical book up for sale on Amazon is pretty trivial. The process of transforming the actual content of the Sequences into something that works in book form is significantly harder. In Eq doesn't have this issue.

The enormity of the task combined with other competing priorities at MIRI are the reason it's not out yet.

Yes.

I think the new font looks pretty good. I do think though for a body font the x-height is pretty small which makes is less readable.

Wording seem less clear then it could be here, what does it mean to say it “produces better problem-solving.” What about something like:

. . . that participants arrive at the correct answer more often when the problems is presented in terms of frequencies, 20 patients, rather then probabilities, 20% of patients.”

Agree. Could be replaced with “similar” or “similar in form”. The sentence could also be change to say something like “This problem is just like . . .”

Someone just snagged the last $1,001 match. Thanks to all those who donated $1,001 to secure the matching, and DeevGrape for providing it!

Another $1,001 donation has come it.

One last $1,001 match remaining.

3 of the 5 $1,001 matches have already been claimed. As additional $1,001 donations come in I'll post updates here.

The server was down, but it is back up again now.

It does: http://agentfoundations.org/rss

The link to it is the last thing in the right sidebar. It says RSS in green.

Definitely beneficial, there is no cost worth considering when it comes to the next marginal person getting the book through our site, even if their selection is $0. So don't worry about directing them there.