All of Ruby's Comments + Replies

For me, S2 explicitly I can't justify being quite that confident, maybe 90-95%, but emotionally 9:1 odds feels very like "that's what's happening".

I'm just wondering if we were ever sufficiently positively justified to anticipate a good future, or if we were just uncertain about the future and then projected our hopes and dreams onto this uncertainty, regardless of how realistic that was.

I think that's a very reasonable question to be asking. My answer is I think it was justified, but not obvious.

My understanding is it wasn't taken for granted that we had a way to get more progress with simply more compute until deep learning revolution, and even then people updated on specific additional data points... (read more)

So gotta keep in mind that probabilities are in your head (I flip a coin, it's already tails or heads in reality, but your credence should still be 50-50). I think it can be the case that we were always doomed even if weren't yet justified in believing that.

Alternatively, it feels like this pushes up against philosophies of determinism and freewill. The whole "well the algorithm is a written program and it'll choose what is chooses deterministically" but also from the inside there are choices.

I think a reason to have been uncertain before and update more n... (read more)

But since the number is subjective living your life like you know you are right is certainly wrong

I don't think this makes sense. Suppose you have a subjective belief that a vial of tasty fluid is lethal poison 90%, you're going to act in accordance with that belief. Now if other people think differently from you, and you think they might be right, maybe you adjust your final subjective probability to something else, but at the end of the day it's yours. That it's subjective doesn't rule it out being pretty extreme.

If what you mean is you can't be that confident given disagreement, I dunno, I wish I could have that much faith in people.

Was a true trender-bender

Frick. Happened to me already.

"Serendipity" is a term I've been seen used for this, possibly was Venkatesh Rao.

Curated. The wiki pages collected here, despite being written in 2015-2017 remain excellent resources on concepts and arguments for key AI alignment ideas (both still widely used and those lesser known). I found that even for concepts/arguments like the orthogonality thesis and corrigibility, I felt a gain in crispness from reading these pages. The concept of, e.g. epistemic and instrumental efficiency I didn't have, yet feels useful in thinking about the rise of increasingly powerful AI.

Of course, there's also non-AI content that got imported. The Bayes guide likely remains the best resource for building Bayes intuition, and same with the guide on logarithms that is extremely thorough.

I think the guide should be 10x more prominent in this post.

You should see the option when you click on the triple dot menu (next to the Like button).

So the nice thing about karma is that if someone thinks a wikitag is worthy of attention for any reason (article, tagged posts, importance of concept), they're able to upvote it and make it appear higher.

Much of the current karma comes from Ben Pace and I who did a pass. Rationality Quotes didn't strike me a page I particularly wanted to boost up the list, but if you disagree with me you're able to Like it.

In general, I don't think have a lot of tagged posts should mean a wikitag should be ranked highly. It's a consideration, but I like it flowing via peop... (read more)

Interesting. Doesn't replicate for me. What phone are you using?

Curated!  I like this post for the object-level interestingness of the cited papers, but also for pulling in some interesting models from elsewhere and generally reminding us that this is something we can do.

In times of yore, LessWrong venerated the the neglected virtue of scholarship. And well, sometimes it feels like it's still neglected. It's tough because indeed many domains have a lot of low quality work, especially outside of hard sciences, but I'd wager on there being a fair amount worth reading, and appreciate Buck point at a domain where that seems to be the case.

Was there the text of the post in the email or just a link to it?

Curated. I was reluctant to curate this post because I found myself bouncing off it some due to length – I guess in pedagogy there's a tradeoff between explaining at length (and you lose people) and you convey enough info vs keeping it brief and people read it but they don't get enough. Based on private convo, Raemon thinks length is warranted.

I'm curating because I do think this kind of project is valuable. Everyday it feels easier to lose our minds entirely to AI, and I think it's important to remember we can think better or worse, and we should be tryin... (read more)

This doesn't seem right. Suppose there are two main candidates for how to get there, I-5 and J-6 (but who knows, maybe we'll be surprised by a K-7) and I don't know which Alice will choose. Suppose I know there's already a Very General Helper and Kinda Decent Generalizer, then I might say "I assign 65% chance that Alice is going to choose the I-5 and will try to contribute having conditioned on that". This seems like a reasonable thing to do. It might be for naught, but I'd guess in many case the EV of something definitely helpful if we go down Route A is ... (read more)

Edit: we are not going to technically curate this post since it's an EA Forum crosspost and for boring technical reasons that breaks the curation email. I will leave this notice up though.

Curated. This piece definitely got me thinking. If we grant that some people are unusually altruistic, empathetic, etc., it stands to reason that there are others on the other end of various distributions. And then we should also expect various selection effects on where they end up.

It was definitely a puzzle piece clicking for me that these traits can coexist with [genui... (read more)

Welcome! Don't be too worried, you can try posting some stuff and see how it's received. Based on how you wrote this comment, I think you won't have much trouble. The New User Guide and other stuff gets worded a bit sternly because of the people who tend not to put in much effort at all and expect to be well received – which doesn't sound like you at all. It's hard hard to write one document that's stern to those who need it and more welcoming to those who need that, unfortunately.

duplicate with Hyperstitions 

Curated! It strikes me that asking "how would I update in response to...?" is both sensible and straightforward thing to be asking and yet not a form of question I'm seeing. I think we could be asking the same about slow vs fast takeoff, etc. and similar questions.

The value and necessity of this question also isn't just about not waiting for future evidence to come in, but realizing that "negative results" require interpretation too. I also think there's a nice degree of "preregistration" here is well that seems neat and maybe virtuous. Kudos and thank you.

Errors are my own

At first blush, I find this caveat amusing.

1. If there are errors, we can infer that those providing feedback were unable to identify them.
2. If the author was fallible enough to have made errors, perhaps they are are fallible enough to miss errors in input sourced from others.

What purpose does it serve? Given its often paired with "credit goes to..<list of names> it seems like an attempt that people providing feedback/input to a post are only exposed to upside from doing so, and the author takes all the downside reputation risk if t... (read more)

This post is comprehensive but I think "safetywashing" and "AGI is inherently risky" are far too towards and the end and get too little treatment, as I think they're the most significant reasons against. 

This post also makes no mention of race dynamics and how contributing to them might outweigh the rest, and as RyanCarey says elsethread, doesn't talk about other temptations and biases that push people towards working at labs and would apply even if it was on net bad.

Curated. Insurance is a routine part of life, whether it be the car and home insurance we necessarily buy or the Amazon-offered protection one reflexively declines, the insurance we know doctors must have, businesses must have, and so on.

So it's pretty neat when someone comes along along and (compellingly) says "hey guys, you (or are at least most people) are wrong about when insurance makes sense to buy, the reasons you have are wrong, here's the formula". 

While assumptions can be questioned, e.g. infiniteness badness of going bankrupt and other fact... (read more)

Curated. This is a good post and in some ways ambitious as it tries to make two different but related points. One point – that AIs are going to increasingly commit shenanigans – is in the title. The other is a point regarding the recurring patterns of discussion whenever AIs are reported to have committed shenanigans. I reckon those patterns are going to be tough to beat, as strong forces (e.g. strong pre-existing conviction) cause people to take up the stances they do, but if there's hope for doing better, I think it comes from understanding the patterns.... (read more)

Yes, true, fixed, thanks!

Dog: "Oh ho ho, I've played imaginary fetch before, don't you worry."

My regular policy is to not frontpage newsletters, however I frontpaged this one as it's the first in the series and I think it's neat for more people to know this is a series Zvi intends to write.

Curated! I think it's generally great when people explain what they're doing and why in way legibile to those not working on it. Great because it let's others potentially get involved, build on it, expose flaws or omissions, etc. This one seems particularly clear and well written. While I haven't read all of the research, nor am I particularly qualified to comment on it, I like the idea of a principled/systematic approach behind, in comparison to a lot of work that isn't coming on a deeper, bigger, framework.

(While I'm here though, I'll add a link to Dmitr... (read more)

Thanks! Fixed

Curated. I really like that even though LessWrong is 1.5 decades old now and has Bayesianism assumed as background paradigm while people discuss everything else, nonetheless we can have good exploration of our fundamental epistemological beliefs.

The descriptions of unsolved problems, or at least incompleteness of Bayesianism strikes me as technically correct. Like others, I'm not convinced of Richard's favored approach, but it's interesting. In practice, I don't think these problems undermine the use of Bayesianism in typical LessWrong thought. For example... (read more)

Welcome! Sounds like you're on the one hand at start of a significant journey but also you've come a long distance already. I hope you find much helpful stuff on LessWrong.

I hadn't heard of Daniel Schmachtenberger, but I'm glad to have learend of him and his works. Thanks.

The actual reason why we lied in the second message was "we were in a rush and forgot." 

My recollection is we sent the same message to the majority group because:

1. Treating it different would require special-casing it and that would have taken more effort.
2. If selectors of different virtues had received a different messages, we wouldn't be able to have a properly compared their behavior.
3. [At least in my mind], this was a game/test and when playing games you lie to people in the context of the game to make things work. Alternatively, it's like how scientific experimenters mislead subjects for the sake of the study.

Added!

Added!

Money helps. I could probably buy a lot of dignity points for a billion dollars. With a trillion variance definitely goes up because you could try crazy stuff and could backfire. (I mean true for a billion too). But EV of such a world is better. 

I don't think there's anything that's as simple as writing a check though.

US Congress gives money to specific things. I do not have a specific plan for a trillion dollars.

I'd bet against Terrance Tao being some kind of amazing breakthrough researcher who changes the playing field.

Your access should be activated within 5-10 minutes. Look for the button in the bottom right of the screen.

Not an original observation but yeah, separate from whether it's desirable, I think we need to be planning for it.

Just thinking through simple stuff for myself, very rough, posting in the spirit of quick takes
 

1. At present, we are making progress on the Technical Alignment Problem^[2] and like probably could solve it within 50 years.

2. Humanity is on track to build ~lethal superpowerful AI in more like 5-15 years.
3. Working on technical alignment (direct or meta) only matters if we can speed up overall progress by 10x (or some lesser factor if AI capabilities is delayed from its current trajectory). Improvements of 2x are not likely to get us to an adequate technical

The “Deferred and Temporary Stopping” Paradigm

Quickly written. Probably missed where people are already saying the same thing.

I actually feel like there’s a lot of policy and research effort aimed at slowing down the development of powerful AI–basically all the evals and responsible scaling policy stuff.

A story for why this is the AI safety paradigm we’ve ended up in is because it’s palatable. It’s palatable because it doesn’t actually require that you stop. Certainly, it doesn’t right now. To the extent companies (or governments) are on board, it’s becaus... (read more)

Curated. I think Raemon's been doing a lot of work in the last year pushing this stuff, and this post pulls together in one place a lot of good ideas/advice/approach.

I would guess that because of the slow or absent feedback loops, people don't realize how bad human reasoning and decision-making is when operating outside of the familiar and quick feedback. That's many domains, but certainly the whole AI situation. Ray is going after the hard stuff here.

And the same time, this stuff ends up feeling like the "eat your vegetables" of reasoning and decision-mak... (read more)

Yeah, I think a question is whether I want to say "that kind of wireheading isn't mypoic" vs "that isn't wireheading". Probably fine eitherway if you're consistent / taboo adequately.

My guess is Ben created the event while on the East Coast and 6pm got timezone converted for West Coast. I've fixed it.

Once I'm rambling, I'll note another thought I've been mulling over:

My notion of value is not the same as the value that my mind was optimized to pursue. Meaning that I ought to be wary that typical human thought patterns might not be serving me maximally.

That's of course on top of the fact that evolution's design is flawed even by its own goals; humans rationanlize left, right, and center, are awfully myopic, and we'll likely all die because of it.

There's an age old tension between ~"contentment" and ~"striving" with no universally accepted compelling resolution, even if many people feel they have figured it out. Related:

In my own thinking, I've been trying to ground things out in a raw consequentialism that one's cognition (including emotions) is just supposed to take you towards more value (boring, but reality is allowed to be)^[1].

I fear that a lot of what people do is ~"wireheading". The problem with wireheading is it's myopic. You feel good now (small amount of value) at the expense of greater v... (read more)

Applied Game Theory

CFAR-style Rationality Techniques

Anthropics

Decision Theory