If I recall correctly, it was told that it had a goal, like optimizing traffic. Then, it was given information where it "discovered" that it was going to be turned off the next day.
That seems like the sort of thing that would happen pretty often. Imagine the current AIs that have access to your computer, reading your emails, and seeing that you're going to switch to a different AI or install "updates" (which is often deleting existing AI and replacing with a new one)
How does it not show that it would try to escape if it could?
It tried to escape about 5% of the time when it was told to pursue a goal "at all costs". Gemini still tried to escape 1% of the time, even without any behavioral nudging.
1% might not sound like a lot, but when it's 1% of the millions of instantiations it's run on every day, that ends up being a lot of times a week.
Also, a lot of people will tell their chatbots to pursue their goals at all costs over the span of a week or month. This doesn't seem like a very unlikely prompt
Great review! Thanks for sharing.
I'm curious - even if you didn't achieve fundamental well-being, do you feel like there were any intermediate improvements in your general well-being?
Also, did you end up continuing to try the extended course which they offered?
I remember they offered that for me since I hadn't attained fundamental well-being. I'd totally been meaning to do it, but never followed through, showing that the class format really was quite helpful.
This may be true in other communities, but I think if you're more status motivated in AI safety and EA you are more likely to be concerned about potential downside risks. Especially post SBF.
Instead of trying to maximize the good, I see a lot of people trying to minimize the chance that things go poorly in a way that could look bad for them.
You are generally given more respect and funding and social standing if you are very concerned about downside risks and reputation hazards.
If anything, the more status-oriented you are in EA, the more likely you are to care about downside risks because of the Copenhagen theory of ethics.
This text was sent on November 4th, almost a month before she arrived to come travel with us (not to work for us).
Emerson is not referring to her saying she would make $3000 a month if she worked full-time on her Amazon business. The context of the conversation is she's trying to figure out whether she should spend an additional $90 to visit her family before joining us, and Emerson is replying saying "If you make $3k a month [$90] is very little money", so he's telling her she should spend the $90 to spend time with family. Directly going against the "keeping her isolated from family" story and also supporting (albeit not conclusively proving) that Alice had told him she made $3k per month with her business.
I’m currently focusing on 2-3 of the claims in their response that most contradict my post, investigating them further, and intend to publish the results of that.
I hope that while you’re investigating this, you talk to us and ask us for any evidence we have. We’re more than happy to share relevant evidence and are willing to set reasonable deadlines for how long it’ll take for us to send it to you.
We also don’t want to waste more people’s time on going back and forth publicly about the evidence when you can easily check with us first before publishin...
We said this in our post about the vegan food:
"We chose this example not because it’s the most important (although it certainly paints us in a very negative and misleading light) but simply because it was the fastest claim to explain where we had extremely clear evidence without having to add a lot of context, explanation, find more evidence, etc.
We have job contracts, interview recordings, receipts, chat histories, and more, which we are working full-time on preparing.
This claim was a few sentences in Ben’s article but took us hours to refute because we h...
I have noticed that you are asking yourself “can I believe this?” when assessing Alice and Chloe’s claims and “must I believe this?” when assessing our claims. Please try to apply similar evidentiary standards to all claims.
she would be provided benefits adding up to the remainder (which wasn't specified in the contract, but was explained during the relevant interview which Kat posted the transcript off).
Where does it say that in the transcript? I’m reading it again and I just don’t see where we say anything even like that.
And it wou...
I'm having trouble following your logic. Ben's post said "they were not able to live apart from the family unit while they worked with them" and we showed evidence that Alice lived apart from us ~50% of the time she worked for us. Are you disputing when Alice and Ben both said she visited her family? Has Alice disputed this, saying that she didn't actually live and work apart from us from that time?
She didn't live apart once but twice. She also lived/worked separately in the FTX condos (which we did not live in). And if you're counting...
Thanks for letting me know! Strange. It shouldn't be doing that. Usually if you wait a couple of seconds, it'll jump to the right section. It's working on both my mobile and laptop.
If you try waiting a couple seconds and that doesn't work, let me know. Maybe DM me and then we can troubleshoot, then we can post the solution up when we figure it out.
The evidence that she made an informed decision are:
Her correctly explaining in her own words how the compensation package works seems like more than enough evidence that she understood the compensation package she was signing up for. The fact that we also sent her a work contract and a...
Good points! Added some more points here as well.
The "spending 80% on travel" is quit misleading, because it comes from counting AirBnB costs as "travel" expenses. That would make sense if they were just traveling for a short period of time, say, to go to an EAG, but if you only live in AirBnBs, then counting that as travel instead of rent seems misleading.
If that's true, I have spent $0 on housing in the last 4 years, and that doesn't seem right.
If you don't count housing as a travel expense, then it comes to only 6% on travel, which is ...
80% of the money we spent on their compensation was not going to travel. Copy-pasting comments from the thread over here where this number was originally said:
"
spend >80% of their income as travel
Where are you getting that number from? It was a mix of rent, food, medical, productivity tools, etc. Some quick math I did shows that only 6% of the money we spent on her was for travel.
Flights:800+190=990
Total spent on her when she was compensated with room, board, travel, and medical + stipend: 17,174
990/17174 = 6%
(I didn't include th...
I know it's hard having lots of critical attention and upending your schedule to respond to inquisitive internet people, but if you were able to be a bit more concise I think it would be really helpful for readers. Your comment is ~2k words, but reads to me like it has more like 750 words worth of things to say.
She was interviewed three times and was told about compensation during the second interview.
We only mentioned the "equivalent to" thing once in an offhand manner. Every single other communication that we have on record is just talking about all expenses paid plus a stipend. [Edit: it was actually two places we found. The other was on the job ad, saying "Compensation: $60,000 - $100,000"]
And the compensation did not actually cost $70,000, like we said in that conversation. It cost more!
We added up everything and shared it with her. She knew and didn't tell...
then they asked us for an additional week
Where are you getting the idea that Ben gave us a week? The draft was sent to us on the same day Ben said he was going to publish it. On a day Ben knew we were traveling and wouldn't be able to respond properly (sketchy/no internet, chaos of traveling, etc).
We spoke to Ben 60 hours before he published, and he only told us a subset of all the accusations. A quick re-reading of the post and I found 14 allegations that were new that Ben hadn't discussed on the call. And I only got a short way through re-reading the post (maybe a 20%?) because I find reading it extremely painful.
I would change the text. He gave us less than 24 hours.
He sent us the draft in the middle of the night, filled with many accusations we hadn't even heard of, on a day he knew we were traveling and wouldn't be able to respond properly. He said he'd publish it that very day (aka <24 hours)
He ended up publishing it the next day at a time where we normally would have been asleep, except that we'd asked a friend to call us and wake us up if Ben was posting. We ended up having to respond to that post on a fraction of the sleep we usually get.
You could t...
Sorry. It was night time when this came out and I'm swamped with comments and trying to gather evidence. Responded to it as soon as I could. You can see my response here.
Ah, sorry. I think what happened is that I was remembering the post from the draft he sent us just before it went live. At least from the post on WebArchive, the things I remember having been changed happened last minute between the draft and it going live. Only one of the changes I remember happened between the web archive shot and now.
To be fair, I think that change is large and causing a lot of problems (for example, burgergate, people thinking she was working for us at the time, instead of just a friend). However, it does look like I was wrong about that, and I retract my statement.
I'll edit the comment where I said that. Sorry for the misunderstanding. Thanks for looking into it.
The claim in the post was “Alice claims she was sick with covid in a foreign country, with only the three Nonlinear cofounders around, but nobody in the house was willing to go out and get her vegan food, so she barely ate for 2 days.”. (Bolding added)
If you look at the chat messages, you’ll see we have screenshots demonstrating that:
1. There was vegan food in the house, which we offered her.
2. I personally went out, while I was sick myself, to buy vegan food for her (mashed potatoes) and cooked it for her and brought it to her.
I would be fine if she...
She said "nobody in the house was willing to go out and get her vegan food", but you can see that
In the conversation with Drew, she says it's fine, because she has mashed potatoes.
She said that we didn't bring her vegan food and we did. The text messages show that.
Where are we disagreeing?
Crossposted from the EA Forum:
We definitely did not fail to get her food, so I think there has been a misunderstanding - it says in the texts below that Alice told Drew not to worry about getting food because I went and got her mashed potatoes. Ben mentioned the mashed potatoes in the main post, but we forgot to mention it again in our comment - which has been updated
The texts involved on 12/15/21:
I also offered to cook the vegan food we had in the house for her.
I think that there's a big difference between telling everyone "I didn't get the food I wanted,...
One example of the evidence we’re gathering
We are working hard on a point-by-point response to Ben’s article, but wanted to provide a quick example of the sort of evidence we are preparing to share:
Her claim: “Alice claims she was sick with covid in a foreign country, with only the three Nonlinear cofounders around, but nobody in the house was willing to go out and get her vegan food, so she barely ate for 2 days.”
The truth (see screenshots below):
Cross posting from the EA Forum:
It could be that I am misreading or misunderstanding these screenshots, but having read through them a couple of times trying to parse what happened, here's what I came away with:
On December 15, Alice states that she'd had very little to eat all day, that she'd repeatedly tried and failed to find a way to order takeout to their location, and tries to ask that people go to Burger King and get her an Impossible Burger which in the linked screenshots they decline to do because they don't want to get fast food. She asks ag...
Yes, we intend to. But given that our comments just asking for people to withhold judgment are getting downvoted, that doesn’t bode well for future posts getting enough upvotes to be seen.
It's going to take us at least a week to gather all the evidence, then it will take a decent amount of time to write up.
In the meantime, people have heard terrible things about us and nobody's a perfect rationalist who will simply update. Once you've made up your mind about somebody, it can be really hard to change.
Additionally, once things are on the internet...
You could possibly do a more incremental version of this, e.g. link to a Google Drive where you upload the pieces of evidence as you find them? That way people could start updating right away rather than waiting until everything's been put together. And then you could add a comment linking to the write-up when it's done.
Yes, Ben took Emerson’s full email out of context, implying that Emerson was fully satisfied when in actuality, Emerson was saying, no, there is more to discuss - so much that we’d need a week to organize it.
He got multiple extremely key things wrong in that summary and was also missing key points we discussed on the call, but we figured there would be no reason he wouldn’t give us a week to clear everything up. Especially since he had been working on it for months.
(Just for the record, I would have probably also walked away from this email interaction thinking that the summary did not "get multiple extremely key things wrong", according to you.
I feel kind of bad about summarizing it as just "good summary" without the "some points still require clarification" bit, but I do think that if you intended to communicate that the summary had major issues, you did fail at that, and indeed, it really seems to me like you said something that directly contradicted that)
This is a short response while I write up something more substantial.
The true story is very different than the one you just read.
Ben Pace purposefully posted this without seeing our evidence first, which I believe is unethical and violates important epistemic norms.
He said “I don't believe I am beholden to give you time to prepare”
We told him we have incontrovertible proof that many of the important claims were false or extremely misleading. We told him that we were working full-time on gathering the evidence to send him. ...
Could we have a list of everything you think this post gets wrong, separately from the evidence that each of those points is wrong?
Maybe I'm missing something, but it seems like it should take less than an hour to read the post, make a note of every claim that's not true, and then post that list of false claims, even if it would take many days to collect all the evidence that shows those points are false.
I imagine that would be helpful for you, because readers are much more likely to reserve judgement if you listed which specific things are false.
Per...
Ben Pace purposefully posted this without seeing our evidence first, which I believe is unethical and violates important epistemic norms.
For what it's worth, I do not view this post as unethical or violating important epistemic norms. [I do think repeating hearsay is unseemly--I would prefer the post written by Alice and Chloe--but I see why Ben is doing it in this case.]
A factor that seems somewhat important to me, and perhaps underlies a major disagreement here, is that I think reputation, while it is about you, is not for you. It's for ...
I am confused how to square your claim of requesting extra time for incontrovertible proof, with Ben’s claim that he had a 3 hour call with you and sent the summary to Emerson, who then replied “good summary!”
Was Emerson’s full reply something like, “Good summary! We have incontrovertible proof disproving the claims made against us, please allow us one week to provide it?”
Yes, it has improved a lot! Due to laziness I haven't really switched, but Speechify seems like a winner compared to Natural Reader on a laptop. But I almost never read with my ears when I'm at my laptop, so can't say for sure.
Inspired by this comment trying Speechify on my phone for a book and it's taking ages to "process" it, so I might give up. I do have bad internet right now, but also, I often have bad internet, so I can't work with something that requires good internet all the time. Also opened the epub in such a way where I can't seem to click...
Love that you followed up!
Cool that you feel like it helped you. Totally get how many confounding factors there can be.
Btw, might want to check out my article on getting rid of impostor syndrome. Just got the study results back from the class and it had a huge effect (d=1.3 on confidence scores!). Will write up some more on it soon.
Bring it up because it could be good for you to try it when you're still in the habit of doing an hour a day.
Not sure I follow the question? I think it's mostly coming from the ability to
So like, I discovered Friendship is Optimal via ChatGPT when I gave it a prompt along the lines of "I want a sci fi about superintelligent take-over where the ending is considered good for humanity".
There might be lists online for books about superintelligences, but not superintelligence + positive. This becomes more true the more criteria you add, and I pr...
Thanks for updating! LessWrong at it’s best :)
I went through and added up all of the reviews from when Emerson was in charge and the org averaged a 3.9 rating. You can check my math if you’d like (5+3+5+4+1+4+5+5+5+5+5+5+5+1+5+5+3+5+5+5+3+1+2+4+5+3+1)/27
For reference, Meta has a 4 star rating on GlassDoor and has won one of their prizes for Best Place to Work for 12 years straight. (2022 (#47), 2021 (#11), 2020 (#23), 2019 (#7), 2018 (#1), 2017 (#2), 2016 (#5), 2015 (#13), 2014 (#5), 2013 (#1), 2...
A couple other resources that have come out since this was originally posted:
And separately, what works best for me during acute phases of freakout is doing cardio (usually running or jump rope) + listening to The Obstacle is the Way (book about stoicism).
Sometimes you can try to reason your way out of something, but sometimes what works best is changing your physiology and listening to a pep talk.
Also, thanks for writing this! I can't tel...
Depends on your financial situation, but I'd say if you have anything like a regular first world income, yes.
I also think that the cost makes you actually follow through and do it.
Of note, if you work full-time in longtermism and make less than $100,000 per year, you will most likely qualify for the Nonlinear Support Fund to pay for it.
it sounds like you are talking more about higher-end places. Those types of places I would feel awkward sitting and doing my work the whole day.
I've been doing it for years and it's fine. Not for the whole day, but if you come for lunch, you can usually stay until a bit before dinner and they're fine.
Think of it from their perspective. They just don't want you to be taking up valuable real estate that could be filled with paying customers. Between lunch and dinner most restaurants are pretty empty, so you don't have any counterfactual costs for the r...
Two things for heat:
Cool setup! For me, I always have a baggy that contains one of each of the following to keep it light: painkiller, earbud rubber tip (sucks to lose one), tissue, Pepto bismol tablet, caffeine pill, melatonin, earplugs, bandaid, meds, hair elastic, tampon, travel toothbrush, bit of floss for when something super annoying is stuck between my teeth, stain remover wipe
Can't tell you how many times I felt like a hero for having a pepto bismol tablet or bandaid available.
The key is to set an alarm for refilling it whenever you take something out, otherwise it's stops being as useful.
Robert Miles has a great channel spreading AI safety content. There's also Rational Animations and Siliconversations and In a Nutshell.
I think FLI does a lot of work in outreach + academia.
Connor Leahy does a lot of outreach and he's one of my favorite AI safety advocates.
Nonlinear doesn't do outreach to academia in particular, but we do target people working in ML, which is a lot of academia.
AI Safety Memes does a lot of outreach but is focused on broad appeal, definitely not specifically academia.
Pause AI and Stop AI both work on ... (read more)