LESSWRONG
LW

All of joraine's Comments + Replies

Experiments as a Third Alternative
joraine00

Why get the pod cover? Just get a decent air conditioning system, it's far better and isn't $2500. Make the entire room cold when you sleep.

Reply
2Adam Zerner
A few reasons. Roughly in order of importance: 1. My girlfriend doesn't like it that cold. 2. I don't see $2,500 as too much money. I'm not sure how I'd value my time, but if we just ballpark it at $100/hr, $2,500 is about three 8-hour work days. Thinking about it like that, spending three 8-hour work days on something that will improve my sleep every night for many years doesn't sound too bad. (FWIW, there is another voice in my head screaming "THAT'S SO EXPENSIVE!!!". I just don't endorse that voice.) 3. In the long run, the pod cover actually will be cheaper than blasting the AC. Guesstimating, maybe it costs an extra $20/month to blast the AC. Using that number, after about 10 years, the pod cover becomes cheaper. 4. When you get out of bed and aren't under the covers anymore, it's really cold. This could somewhat be addressed by putting the thermostat on a schedule, but that doesn't addressing getting up in the middle of the night to go to the bathroom.
Feeling Progress as Motivation
joraine10

I like this post! Saved the comment about the "a day is all you need" induction to my quote bank,


I was guessing this was going in a slightly different direction, namely: tracking progress (in a spreadsheet is what I do) such that you can actually see the fact you're still making progress (this is why video-games with the clear leveling indicators are so addicting!) and you don't mistakenly believe you're stalling and get demotivated.

I like the new skill idea too though. I am already prone to starting over in new arenas a bit too much, but having a set time for a new skill is a good idea.

Reply
Dumb and ill-posed question: Is conceptual research like this MIRI paper on the shutdown problem/Corrigibility "real"
joraine30

I suppose modelling a superintelligent agent as a utility maximizer feels a bit weird but not the weirdest thing, and I'm not sure I can mount a good defense saying that a superintelligent agent definitely wouldn't be aptly modeled by that.

More importantly, the 3-step toy model with , ,  felt like a strange and unrelated leap

I don't know if it's about the not having an answer part. That is probably biasing me. But similar to the cryptography example, if someone defined what security would mean, let's say Indistinguishability under ... (read more)

Reply
3DaemonicSigil
From a pure world-modelling perspective, the 3 step model is not very interesting, because it doesn't describe reality. It's maybe best to think of it from an engineering perspective, as a test case. We're trying to build an AI, and we want to make sure it works well. We don't know exactly what that looks like in the real world, but we know what it looks like in simplified situations, where the off button is explicitly labelled for the AI and everything is well understood. If a proposed AI design does the wrong thing in the 3-step test case, then it has failed one of its unit tests, and should not be deployed to production (the real world). So the point of the paper is that a reasonable-sounding way you could design an AI with an off switch turns out to fail the unit-test. I do generally think that too many of the AI-related posts here on LessWrong are "not real" in the way you're suggesting, but this paper in particular seems "real" to me (whatever that means). I find the most "not real" posts are the verbose ones piled high with vague wordy abstractions, without an equation in sight. The equations in the corrigiblity paper aren't there to seem impressive, they're there to unambiguously communicate the math the paper is talking about, so that if the authors have made an error of reasoning, it will be as obvious as possible. The ways you keep something in contact with reality is checking either against experiment, or against the laws of mathematics. To quote Feynman, "if it disagrees with experiment, it's wrong" and similarly, there's a standard in mathematics that statements must be backed up by checkable calculations and proofs. So long as the authors are holding themselves to that standard (and so long as you agree that any well-designed AI should be able to perform well in this easy test case), then it's "real".
Dumb and ill-posed question: Is conceptual research like this MIRI paper on the shutdown problem/Corrigibility "real"
joraine20

I do like the comparison to cryptography, as that is a field I "take seriously" and does also have the issue of it being very difficult to "fairly" define terms. 

Indistinguishability under chosen plain text attack being the definition for something to be canonically "secure" seems a lot more defensible than "properly modeling this random weird utility game maybe means something for AGI ??" but I get why it's a similar sort of issue

Reply
It’s Probably Not Lithium
joraine10

How are we defining tasty foods? I'm sure if the entire world voted, chocolate would clearly be more in the "tasty food" category than rice cakes, but perhaps you really like how rice cakes taste?

Reply
Will working here advance AGI? Help us not destroy the world!
joraine20

It wasn't my suggestion it was Logan Zoellner's post

Reply
Will working here advance AGI? Help us not destroy the world!
joraine50

Can someone who downvote the agreement karma please enlighten me as to why they disagree? This really seems like the only way forward. (Trying to make my career choice right now as I am beginning my masters research this year)

Reply
1Yonatan Cale
I didn't downvote, but your suggestion seems obviously wrong to me, so:  Working in one of those companies (assuming you have added value to them) is a pretty high confidence way to get unfriendly AGI faster. If you want to build skills, there are lots of ways to do that without working at very dangerous companies.
AGI Ruin: A List of Lethalities
joraine282

This kind of post scares away the person who will be the key person in the AI safety field if we define "key person" as the genius main driver behind solving it, not the loudest person.  Which is rather unfortunate, because that person is likely to read this post at some point.

I don't believe this post has any "dignity", whatever weird obscure definition dignity has been given now. It's more like flailing around in death throes while pointing fingers and lauding yourself than it is a solemn battle stance against an oncoming impossible enemy.

For contex... (read more)

Reply
We will be around in 30 years
joraine40

You don't have to say the scenario, but was it removed because someone is going to execute it if they see it?

Reply
1mukashi
I got scolded in a different post by the LW moderators by saying that there is a policy of not brainstorming about different ways to end the world because it is considered an info hazard. I think this makes sense and we should be careful doing that
Experience LessWrong without the Time-Wasting RabbitHole Effect
joraine50

I very much enjoy that type of style of pop-up. It's quick and feels more like adding an optional paragraph versus opening a never-ending portal every 5 seconds.  Your link css is also not as jarring as a bright color on a white background compared to regular black text

Reply
Experience LessWrong without the Time-Wasting RabbitHole Effect
joraine10

So this was a phase? You went through it, wrote the article and then came out the other side?


From where I am right now it feels never-ending but I suppose if I can read so much that it stops being so provably distracting, that's a nice upside.

Reply
2Raemon
Yeah. It's maybe an unfortunate fact that the amount of stuff grows over time, but you will definitely hit a point where you've made it through the backlog.
Reflections on a year of college
joraine*140

"I realize that the tone of this post may come off as very cynical. "

I actually thought the tone of your post was very optimistic and exciting, that is until you ended it ironically by saying "since this is too cynical, let me lighten it up a bit by reminding everyone here that life is hilariously short and you'll all die so you better enjoy it!"

I do not agree that talking about the greatest sadness in life somehow is a positive nice happy thing, despite people's attempts to twist it in that direction. ("You only can appreciate things that are limited!")

I ... (read more)

Reply
1unoptimal
Thanks for this comment. I was hoping that segment would come off as a genuine reminder for undergrads to loosen up a bit and enjoy the fun parts of college after reading a chunk of heavy material that likely goes against many commonly held preconceived notions. But I appreciate this pushback; it's one I haven't encountered much in my social circles, which usually view the "life is short" sentiment in a positive light (see: Kurzgesagt's Optimistic Nihilism video). FWIW, I find that this perspective encourages me to try harder and be better.
MIRI announces new "Death With Dignity" strategy
joraine10

Yeah I saw this post:

https://www.lesswrong.com/posts/MR6cJKy2LE6kF24B7/why-hasn-t-deep-learning-generated-significant-economic

 

So I'm somewhat confused on how profitable AI is, but maybe I can just start asking random experts and researching AI startups

Reply
MIRI announces new "Death With Dignity" strategy
joraine30

Apologize for long wall of text, at the bottom I dived into your aside more as that's highly relevant to deciding the course of my next 10 years and would appreciate your weighing-in.

Pre-Lesswrong/my entire life I've been really interested in longevity, and I would do anything to help people have more time with their loved ones (and as a child I thought solving this was the only worthy kind of fame I'd ever want.)
I didn't know how to get there, but it was probably somewhere in math and science so I decided I had to do anything to get into MIT.
My hobbies en... (read more)

Reply
2jacob_cannell
Crypto generally has been unusually profitable (as a career or investment), but much of this simply stems from the rising tide raising all boats. Given that crypto market caps can only grow about another 10x or so before approaching major currency status and financial system parity at which point growth will slow - it seems likely that much of the glory days of crypto are behind us. AI/ML is pretty clearly the big thing happening on Earth. Admittedly in the short term it may be easier to cash in on a quick crypto career, but since most startups fail, consider whether you'd rather try but fail to have an impact on AI vs trying but failing at crypto wealth.
5paulfchristiano
My claim about AI vs crypto was just a misunderstanding. I still think of "cryptography" and "distributed systems" with their historical meaning rather than "cryptocurrency startup" or "cryptocurrency trading," but in the context of earning to give I think that should have been clear to me :) I'd still guess an AI career is generally the better way to make money, but I don't have a strong take / think it depends on the person and situation / am no longer confused by your position.
MIRI announces new "Death With Dignity" strategy
joraine10

I and two of my friends are on the precipice of our careers right now.  We are senior CS majors at MIT, and next year we're all doing our Master's here and have been going back and forth on what to pick.

All of us have heavily considered AI of course. I'm torn between that and Distributed Systems/Cryptography things to do Earn to Give. I've been mostly on the AI side of things until today.

This post has singlehandedly convinced two of us (myself included) to not work on AI or help with AI alignment, as if Eliezer, an expert in that field. is correct, th... (read more)

Reply
6Not Relevant
This is a real shame - there are lots of alignment research directions that could really use productive smart people.  I think you might be trapped in a false dichotomy of "impossible" or "easy". For example, Anthropic/Redwood Research's safety directions will succeed or fail in large part based on how much good interpretability/adversarial auditing/RLHF-and-its-limitations/etc. work smart people do.  Yudkowsky isn't the only expert, and if he's miscalibrated then your actions have extremely high value.
paulfchristiano210

I don't think "Eliezer is wrong about something" implies that the field is ridiculous or easily solved. Many people in the field (including the plurality with meaningful AI experience) disagree with Eliezer's basic perspective and think he is wildly overconfident.

I basically agree with Eliezer that if arguments for doom look convincing then you should focus on something more like improving the log odds of survival, and preparing to do the hard work to take advantage of the actual kind of miracles that might actually occur, rather than starting to live in f... (read more)

Reply