LESSWRONG
LW

All of casens's Comments + Replies

Will Jesus Christ return in an election year?
casens10

The True Believers hypothesis rings false because that would be a frankly ridiculous belief to hold. Sometimes people profess ridiculous things, but very few of them put their money where their mouth is on prediction markets. [1]

  1. I’ve seen some pretty mispriced markets. At one point in 2019, PredictIt had Andrew Yang at 16% to win the Democratic presidential primary. And in 2020, Donald Trump was about 16% to become president even after he had lost the election. But the sorts of people who bet on prediction markets are not the sorts of fundamentalist Ch
... (read more)
Reply
7Eric Neyman
Empirically, the "nerd-crack explanation" seems to have been (partially) correct, see here.
when will LLMs become human-level bloggers?
casens1-1

this is a fair response, and to be honest i was skimming your post a bit. i do think my point somewhat holds, that there is no "intelligence skill tree" where you must unlock the level 1 skills before you progress to level 2.

i think a more fair response to your post is:

  1. companies are trying to make software engineer agents, not bloggers, so the optimization is towards the former.
  2. making a blog that's actually worth reading is hard. no one reads 99% of blogs.
  3. i wouldn't act so confident that we aren't surrounded by LLM comments and posts. are you really
... (read more)
Reply
when will LLMs become human-level bloggers?
casens21

Surely it would be exceptionally good at those kinds of writing, too, right?

 

surely an LLM capable of writing A+ freshman college papers would correctly add two 2-digit numbers?  surely an AI capable of beating grandmasters in chess would be able to tutor a 1000 elo player to a 1500 elo or beyond?  surely an AI capable of answering questions at a university level in diverse subjects such as math, coding, science, law, would be able to recursively improve itself and cause an intelligence explosion?  surely such an AI would at least be ab... (read more)

Reply
5nostalgebraist
The quoted sentence is about what people like Dario Amodei, Miles Brundage, and @Daniel Kokotajlo predict that AI will be able to do by the end of the decade. And although I haven't asked them, I would be pretty surprised if I were wrong here, hence "surely." In the post, I quoted this bit from Amodei: Do you really think that he means "it can do 'any actions, communications, or remote operations enabled by this interface' with a skill exceeding that of the most capable humans in the world – except for writing blog posts or comments"? Do you think he would endorse this caveat if I were to ask him about it? If so, why? Likewise with Brundage, who writes: I mean, he did say "nearly every," so there are some "cognitive domains" in which this thing is still not superhuman.  But do we really think that Brundage thinks "blogging" is likely to be an exception?  Seriously? (Among other things, note that both of these people are talking about AIs that could automate basically any job doable by a remote worker on a computer.  There exist remote jobs which require communication skills + having-interesting-ideas skills such that doing them effectively involves "writing interesting blog posts," just in another venue, e.g. research reports, Slack messages... sometimes these things are even framed as "posts on a company-internal blog" [in my last job I often wrote up my research in posts on a "Confluence blog"]. If you suppose that the AI can do these sorts of jobs, then you either have to infer it's good at blogging too, or you have to invent some very weirdly shaped generalization failure gerrymandered specifically to avoid this otherwise natural conclusion.)
o1 is a bad idea
casensΩ452

In some sense, the Agent Foundations program at MIRI sees the problem as: human values are currently an informal object. We can only get meaningful guarantees for formal systems. So, we need to work on formalizing concepts like human values. Only then will we be able to get formal safety guarantees.

unless i'm misunderstanding you or MIRI, that's not their primary concern at all:

Another way of putting this view is that nearly all of the effort should be going into solving the technical problem, "How would you get an AI system to do some very modest con

... (read more)
Reply
4abramdemski
Good citation. Yeah, I should have flagged harder that my description there was a caricature and not what anyone said at any point. I still need to think more about how to revise the post to be less misleading in this respect. One thing I can say is that the reason that quote flags that particular failure mode is because, according to the MIRI way of thinking about the problem, that is an easy failure mode to fall into. 
Transcript and Brief Response to Twitter Conversation between Yann LeCunn and Eliezer Yudkowsky
casens63

this was posted after your comment, but i think this is close enough:

@ylecun

And the idea that intelligent systems will inevitably want to take over, dominate humans, or just destroy humanity through negligence is preposterous.
They would have to be specifically designed to do so.
Whereas we will obviously design them to not do so.
 

Reply
3paulfchristiano
I'm most convinced by the second sentence: Which definitely seems to be dismissing the possibility of alignment failures. My guess would be that he would back off of this claim if pushed on it explicitly, but I'm not sure. And it is at any rate indicative of his attitude.