All of Raelifin's Comments + Replies

Manifold market here: https://manifold.markets/MaxHarms/will-ai-be-recursively-self-improvi

Just wanted to remind folks that this is coming up on Saturday! I'm looking forward to seeing y'all at the park. It should be sunny and warm. Feel free to send me requests for snacks or whatever.

Is there a minimal thing that Claude could do which would change your mind about whether it’s conscious?

Edit: My question was originally aimed at Richard, but I like Mikhail’s answer.

Thanks! The creators also apparently have a substack: https://forecasting.substack.com/

Value of information

If you have multiple quality metrics then you need a way to aggregate them (barring more radical proposals). Let’s say you sum them (the specifics of how they combine are irrelevant here). What has been created is essentially a 25-star system with a more explicit breakdown. This is essentially what I was suggesting. Rate each post on 5 dimensions from 0 to 2, add the values together, and divide by two (min 0.5), and you have my proposed system. Perhaps you think the interface should clarify the distinct dimensions of quality, but I think UI simplicity is p... (read more)

I agree that there are benefits to hiding karma, but it seems like there are two major costs. The first is in reducing transparency; I claim that people like knowing why something is selected for them, and if karma becomes invisible the information becomes hidden in a way that people won’t like. (One could argue it should be hidden despite people’s desires, but that seems less obvious.) The other major reason is one cited by Habryka: creating common knowledge. Visible Karma scores help people gain a shared understanding of what’s valued across the site. Ra... (read more)

I suggested the 5-star interface because it's the most common way of giving things scores on a fixed scale. We could easily use a slider, or a number between 0 and 100 from my perspective. I think we want to err towards intuitive/easy interfaces even if it means porting over some bad intuitions from Amazon or whatever, but I'm not confident on this point.

I toyed with the idea of having a strong-bet option, which lets a user put down a stronger QJR bet than normal, and thus influence the community rating more than they would by default (albeit exposing them... (read more)

I agree with the expectation that many posts/comments would be nearly indistinguishable on a five-star scale. I'm not sure there's a way around this while keeping most of the desirable properties of having a range of options, though perhaps increasing it from 10 options (half-stars) to 14 or 18 options would help.

My basic thought is that if I can see a bunch of 4.5 star posts, I don't really need the signal as to whether one is 4.3 stars vs 4.7 stars, even if 4.7 is much harder to achieve. I, as a reader, mostly just want a filter for bad/mediocre posts, a... (read more)

Ah! This looks good! I'm excited to try it out.

Yep. I'm aware of that. Our karma system is better in that regard, and I should have mentioned that.

Nice. Thank you. How would you feel about me writing a top-level post reconsidering alternative systems and brainstorming/discussing solutions to the problems you raised?

I also want to note that this proposal isn't mutually exclusive with other ideas, including other karma systems. It seems fine to have there be an additional indicator of popularity that is distinct from quality. Or, more to my liking, would be a button that simply marks that you thought a post was interesting and/or express gratitude towards the writer, without making a statement about how bulletproof the reasoning was. (This might help capture the essence of Rule Thinkers In, Not Out and reward newbies for posting.)

One obvious flaw with this proposal is that the quality-indicator would only be a measure of expected rating by a moderator. But who says that our moderators are the best judges of quality? Like, the scheme is ripe for corruption, and simply pushing the popularity contest one level up to a small group of elites.

One answer is that if you don't like the mods, you can go somewhere else. Vote with your feet, etc.

A more turtles-all-the-way-down answer is that the stakeholders of LW (the users, and possibly influential community members/investors?) agree on an a... (read more)

To my mind the primary features of this system that bear on Duncan's top-level post are:

• High-reputation judges can confidently set the quality signal for a piece of writing, even if they're in the minority. The truth is not a popularity contest, even when it comes to quality.
• The emphasis on betting means that people who "upvote" low-quality posts or "downvote" high-quality ones are punished, making "this made me feel things, and so I'm going to bandwagon" a dangerous mental move. And people who make this sort of move would be efficiently sidelined.

In conce... (read more)

First of all, thank you, Duncan, for this post. I feel like it captures important perspectives that I've had, and problems that I can see and puts them together in a pretty good way. (I also share your perspective that the post Could Be Better in several ways, but I respect you not letting the perfect be the enemy of the good.)

I find myself irritated right now (bothered, not angry) that our community's primary method of highlighting quality writing is by karma-voting. It's a similar kind of feeling to living in a democracy--yes, there are lots of systems t... (read more)

Update: I decided that I like the grass south of the baseball diamond better. Let's meet there.

Hey all, Max here. I was bad/busy on the weekend when I was supposed to provide a more specific location, so I've updated the what3words to a picnic table near the dog/skate park. I reserve the right to continue to adjust the meetup location in the coming weeks if I find even better places, so be sure to check on the 18th for specifics.

I'm an AI safety researcher and author of Crystal Society. I did a bunch of community leading/organizing in Ohio, including running a rationality dojo. I moved out to the bay area in 2016, and to Grass Valley in June. If you... (read more)

Issue 2 is about to be fixed: https://github.com/Discordius/Lesswrong2/pull/188

I picked 7 Habits because it's pretty clearly rationality in my eyes, but is distinctly not LW style Rationality. Perhaps I should have picked something worse to make my point more clear.

Ah, perhaps I misunderstood the negative perception. It sounds like you see him as incompetent, and since he's working with a subject that you care about that registers as disgusting?

I can understand cringing at the content. Some of it registers that way to me, too. I think Gleb's admitted that he's still working to improve. I won't bother copy-pasting the argument that's been made elsewhere on the thread that the target audience has different tastes. It may be the case that InIn's content is garbage.

I guess I just wanted to step in and second jsteinhardt's comment that Gleb is a very growth-oriented and positive, regardless of whether his writing is good enough.

I agree! Having good intentions does not imply the action has net benefit. I tried to communicate in my post that I see this as a situation where failure isn't likely to cause harm. Given that it isn't likely to hurt, and it might help, I think it makes sense to support in general.

(To be clear: Just because something is a net positive (in expectation) clearly doesn't imply one ought to invest resources in supporting it. Marginal utility is a thing, and I personally think there are other projects which have higher total expected-utility.)

Okay well it seems like I'm a bit late to the discussion party. Hopefully my opinion is worth something. Heads up: I live in Columbus Ohio and am one of the organizers of the local LW meetup. I've been friends with Gleb since before he started InIn. I volunteer with Intentional Insights in a bunch of different ways and used to be on the board of directors. I am very likely biased, and while I'm trying to be as fair as possible here you may want to adjust my opinion in light of the obvious factors.

So yeah. This has been the big question about Intentional In... (read more)

I just wanted to interject a comment here as someone who is friends with Gleb in meatspace (we're both organizers of the local meetup). In my experience Gleb is kinda spooky in the way he actually updates his behavior and thoughts in response to information. Like, if he is genuinely convinced that the person who is criticizing him is doing so out of a desire to help make the world a more-sane place (a desire he shares) then he'll treat them like a friend instead of a foe. If he thinks that writing at a lower-level than most rationality content is currently... (read more)

Impostor entries were generally more convincing than genuine responses. I chalk this up to impostors trying harder to convince judges.

But who knows? Maybe you were a vegetarian in a past life! ;)

You're right, but I'm pretty confident that the difference isn't significant. We should probably see it as evidence that rationalists omnivores are about as capable as rationalist vegetarians.

If we look at average percent of positive predictions (predictions that earn more than 0 points):

Omnivores: 51%

Vegetarians: 46%

If we look at non-negative predictions (counting 50% predictions):

Omnivores: 52%

Vegetarians: 49%

As Douglas_Knight points out, it's only 10/12, a probability of ~0.016. In a sample of ~50 we should see about one person at that level of accuracy or inaccuracy, which is exactly what we see. I'm no more inclined to give #14 a medal than I am to call #43 a dunce. See the histogram I stuck on to the end of the post for more intuition about why I see these extreme results as normal.

I absolutely will fess up to exaggerating in that sentence for the sake of dramatic effect. Some judges, such as yourself, were MUCH less wrong. I hope you don't mind me outing y... (read more)

In retrospect I ought to have included options closer to 50%. I didn't expect that they'd be so necessary! You are absolutely right, though.

A big part of LessWrong, I think, is learning to overcome our mental failings. Perhaps we can use this as a lesson that the best judge writes down their credence before seeing the options, then picks the option that is the best match to what they wrote. I know that I, personally, try (and often fail) to use this technique when doing multiple-choice tests.

Every judge being close to 50% would be bizarre. If I flip 13 coins 53 times I would expect that many of those sets of 13 will stray from the 6.5/13 expected ratio. The big question is whether anyone scored high enough or low enough that we can say "this wasn't just pure chance".

This is a very good point, and I ought to have mentioned it in the post. The point remains about overconfidence, however. Those who did decide to try (even given that it was hard) didn't have the mental red-flag that perhaps their best try should be saying "I don't know" with or without walking away.

Great job, you two! Don't forget to give your elephant and rider some time to "discuss" the findings internally before making the final judgment. I find that my elephant will slowly come around unless there's something important I've overlooked, which is a major risk when doing explicit calculations. For instance, I notice there's no representation of location, which tends to be a very important factor in deciding where to live.

I pretty much agree with you. I think it'll be interesting to get the data out of this and see how competent the judges are compared to Leah's Christianity tests. A few people in my local group thought this would be a good topic.

Yes. I'll be providing the answer key in the stats post.

They already did. I encourage you to make your prediction, however (the full judging round will start on Monday or Tuesday depending on my schedule).

I really like this entry. Don't forget to PM me your actual opinion so I can give feedback to the judges and see how you do. ^_^

Yikes. If all responses are this good, I'm sure the judges will have a rough time! Thanks so much for your words. At some point you'll need to PM me with a description of your actual beliefs so I can give feedback to the judges and see how you do.

These are great suggestions! (As are others, suggested in other comments.) Thank you!

When I gave my presentation last night I made sure that people knew that it was called the ITT by others and that was what to search for (I also pointed them to UnEY). I'm still on the fence about pushing the name (ITT is really hard to say) but I'll keep your reservations in mind.

I'll keep you informed of the details moving forward. :-)

Survey completed! Making a note here: Huge success!

The Cleveland meetup was canceled due to people being busy and sick.

This meetup was canceled due to people being sick and busy.

(Primary author, here.)

This is a good point, and obviously there's a lot of tension between phyggish meme-sharing/codewords and a desire to be more inclusive and not so scary. An earlier draft actually made it an explicit point to talk about the perception of phyg, as I think it's one of the biggest PR issues we have.

The pamphlet was written to try and help people not feel so overwhelmed by coming into a space so loaded down with jargon, but you're right that it perpetuates the problem. I encourage people to copy and edit this, perhaps tailoring it to the ... (read more)

And as a followup, even if you're correct about the probabilities (which I'm not sure you are), it's not intrinsically optimal to vote, even if you care about the outcome. One must always weigh the opportunity cost of an action, and the opportunity cost depends on the person.

If a superintelligent AI is being built and an equal amount of Yudkowsky's time will decrease the extinction probability by the same amount as voting would increase candidate X's election probability, then it's clearly not optimal for Yudkowsky to vote, because the neg-utility of extinction far outweighs the neg-utility of an unfortunate election.

voting is rational if you...

This is a minor objection, but voting can't be rational and it can't be irrational. Voting isn't a system of thinking. You may want to rephrase your argument as "voting is optimal if you..."

Hello northern Ohio! My name is Max, and I recently moved up here from Columbus. I've attended a few LW meetups before, and would enjoy getting a regular thing happening in Cleveland. ^_^

Be aware that having tried and failed at something does not mean it does not work. That's generalizing from a single example. Remember: “The apprentice laments 'My art has failed me', while the master says 'I have failed my art'”. This is not to say you're necessarily wrong, just that we need to take a data-based approach, rather than rely on anecdotes.

The elevator pitch that got me most excited about rationality is from Raising the Sanity Waterline. It only deals with epistemic rationality, which is an issue, and it, admittedly, is best fit towards people who belong to a sanity-focused minority, like atheism or something political. It was phrased with regard to religion originally, so I'll keep it this way here, but it can easily be tailored.

"What is rationality?"

Imagine you're teaching a class to deluded religious people, and you want to get them to change their mind and become atheists, bu