All of kave's Comments + Replies

I'm inclined to agree, but at least this is an improvement over it only living in Habryka's head. It may be that this + moderation is basically sufficient, as people seem to have mostly caught on to the intended patterns.

I spent some time Thursday morning arguing with Habryka about the intended use of react downvotes. I think I now have a fairly compact summary of his position.

PSA: When to upvote and downvote a react

Upvote a react when you think it's helpful to the conversation (or at least, not antihelpful) and you agree with it. Imagine a react were a comment. If you would agree-upvote it and not karma-downvote it, you can upvote the react.

Downvote a react when you think it's unhelpful for the conversation. This might be because you think the react isn't being used for i... (read more)

there's evidence about bacteria manipulating weather for this purpose

Sorry, what?

It's been a long time since I looked at virtual comments, as we never actually merged them in. IIRC, none were great, but sometimes they were interesting (in a kind of "bring your own thinking" kind of way).

They were implemented as a Turing test, where mods would have to guess which was the real comment from a high karma user. If they'd been merged in, it would have been interesting to see the stats on guessability.

Could exciting biotech progress lessen the societal pressure to make AGI?

Suppose we reach a temporary AI development pause. We don't know how long the pause will last; we don't have a certain end date nor is it guaranteed to continue. Is it politically easier for that pause to continue if other domains are having transformative impacts?

I've mostly thought this is wishful thinking. Most people don't care about transformative tech; the absence of an alternative path to a good singularity isn't the main driver of societal AI progress.

But I've updated some her... (read more)

I think your comment is supposed to be an outside view argument that tempers the gears-level argument in the post. Maybe we could think of it as providing a base-rate prior for the gears-level argument in the post. Is that roughly right? I'm not sure how much I buy into this kind of argument, but I also have some complaints by the outside views lights.

First, let me quickly recap your argument as I understand it.

R&D increases welfare by allowing an increase in consumption. We'll assume that our growth in consumption is driven, in some fraction, by R&

From population mean or from parent mean?

Curated. Genetically enhanced humans are my best guess for how we achieve existential safety. (Depending on timelines, they may require a coordinated slowdown to work). This post is a pretty readable introduction to a bunch of the why and how and what still needs to be down.

I think this post is maybe slightly too focused on "how to genetically edit for superbabies" to fully deserve its title. I hope we get a treatment of more selection-based methods sometime soon.

GeneSmith mentioned the high-quality discussion as a reason to post here, and I'm glad we're a... (read more)

My understanding when I last looked into it was that the efficient updating of the NNUE basically doesn't matter, and what really matters for its performance and CPU-runnability is its small size.

I'm not aware of a currently published protocol; sorry for confusing phrasing!

There are various technologies that might let you make many more egg cells than are possible to retrieve from an IVF cycle. For example, you might be able to mature oocytes from an ovarian biopsy, or you might be able to turn skin cells into eggs.

Copying over Eliezer's top 3 most important projects from a tweet:

1.  Avert all creation of superintelligence in the near and medium term.

2.  Augment adult human intelligence.

3.  Build superbabies.

Thanks. Fixed.

Previous discussion

Looks like the base url is supposed to be niplav.site. I'll change that now (FYI @niplav)

With LLMs, we might be able to aggregate more qualitative anonymous feedback.

The general rule is roughly "if you write a frontpage post which has an announcement at the end, that can be frontpaged". So for example, if you wrote a post about the vision for Online Learning, that included as a relatively small part the course announcement, that would probably work.

By the way, posts are all personal until mods process them, usually around twice a day. So that's another reason you might sometimes see posts landing on personal for awhile.

Mod note: this post is personal rather than frontpage because event/course/workshop/org... announcements are generally personal, even if the content of the course, say, is pretty clearly relevant to the frontpage (as in this case)

I believe it includes some older donations:

1. Our Manifund application's donations, including donations going back to mid-May, totalling about $50k
2. A couple of older individual donations, in October/early Nov, totalling almost 200k

Mod note: I've put this on Personal rather than Frontpage. I imagine the content of these talks will be frontpage content, but event announcements in general are not.

neural networks routinely generalize to goals that are totally different from what the trainers wanted

I think this is slightly a non sequitor. I take Tom to be saying "AIs will care about stuff that is natural to express in human concept-language" and your evidence to be primarily about "AIs will care about what we tell it to", though I could imagine there being some overflow evidence into Tom's proposition.

I do think the limited success of interpretability is an example of evidence against Tom's proposition. For example, I think there's lots of work where... (read more)

Here's a little bit about the tapes in the video

I dug up my old notes on this book review. Here they are:

So, I've just spent some time going through the World Bank documents on its interventions in Lesotho. The Anti-Politics Machine is not doing great on epistemic checking

• There is no recorded Thaba-Tseka Development Project, despite the period in which it should have taken place being covered
• There is a Thaba-Bosiu development project (parts 1 and 2) taking place at the correct time.
  • Thaba-Bosiu and Thaba-Tseka are both regions of Lesotho
  • The spec doc for Thaba-Bosiu Part 2 references the alleged problems

Yup

I'm not sure I understand what you're driving at, but as far as I do, here's a response: I have lots of concepts and abstractions over the physical world (like chair). I don't have many concepts or abstractions over strings of language, apart from as factored through the physical world. (I have some, like register or language, but they don't actually feel that "final" as concepts).

As far as factoring my predictions of language through the physical world, a lot of the simplest and most robust concepts I have are just nouns, so they're already represented by tokenisation machinery, and I can't do interesting interp to pick them out.

That sounds less messy than the path from 3D physical world to tokens (and less (edit: I meant more here!) messy than the path from human concepts to tokens)

quality of tasks completed

quantity?

Just a message to confirm: Zac's leg of the trade has been executed for $810. Thanks Lucie for those $810!

This doesn't play very well with fractional kelly though

I do feel like it would be good to start with a more optimistic prior on new posts. Over the last year, the mean post karma was a little over 13, and the median was 5.

This seems unlikely to satisfy linearity, as A/B + C/D is not equal to (A+C)/(B+D)

I don't feel particularly uncertain. This EA Forum comment and its parents inform my view quite a bit.

<details>blah blah</details>

I'm guessing encounter 4 (rather than encounter 6) follows encounter 3?

You can simulate a future by short-selling the underlying security and buying a bond with the revenue. You can simulate short-selling the same future by borrowing money (selling a bond) and using the money to buy the underlying security.

I think these are backwards. At the end of your simulated future, you end up with one less of the stock, but you have k extra cash. At the end of your simulated short sell, you end up with one extra of the stock and k less cash.

A neat stylised fact, if it's true. It would be cool to see people checking it in more domains.

I appreciate that Ege included all of examples, theory, and predictions of the theory. I think there's lots of room for criticism of this model, which it would be cool to see tried. In particular, as far as I understand the formalism, it doesn't seem like it is obviously discussing the costs of the investments, as opposed to their returns.

But I still like this as a rule of thumb (open to revision).

I still think this post is cool. Ultimately, I don't think the evidence presented here bares that strongly on the underlying question: "can humans get AIs to do their alignment homework?". But I think it bares on it at all, and was conducted quickly and competently.

I would like to live in a world where lots of people gather lots of weak pieces of evidence on important questions.

Yep, if the first vote takes the score to ≤ 0, then the post will be dropped off the latest list. This is somewhat ameliorated by:

(a) a fair number of people browsing https://lesswrong.com/allPosts

(b) https://greaterwrong.com having chronological sort by default

(c) posts appearing in recent discussion in order that they're posted (though I do wonder if we filter out negative karma posts from recent discussion)

I often play around with different karma / sorting mechanisms, and I do think it would be nice to have a more Bayesian approach that started with a s... (read more)

I had a quick look in the database, and you do have some tag filters set, which could cause the behaviour you describe

• Because it's a number and a vector, you're unlikely to see anyone (other than programmers) trying to use i as a variable.

I think it's quite common to use i as index variable (for example, in a sum)

(edit: whoops, I see several people have mentioned this) 

In this case sitting down with someone doing similar tasks but getting more use out of LMs would likely help.

I would contribute to a bounty for y'all to do this. I would like to know whether the slow progress is prompting-induced or not.

Click on the gear icon next to the feed selector