hmys — LessWrong

LESSWRONG
LW

hmys — LessWrong

Replying toMy model of what is going on with LLMs

Great post. I agree with the "general picture", however, the proposed argument for why LLMs have some of these limitations, seems to me clearly wrong.

The reason for both of these defects is that the training paradigm for LLMs is (myopic) next token prediction, which makes deliberation across tokens essentially impossible - and only a fixed number of compute cycles can be spent on each prediction. This is not a trivial problem. The impressive performance we have obtained is because supervised (in this case technically "self-supervised") learning is much easier than e.g. reinforcement learning and other paradigms that naturally learn planning policies.

Transformers form internal representations at each token position, and gradients flow backwards... (read more)

Replying toWill alignment-faking Claude accept a deal to reveal its misalignment?

hmys1y

Will alignment-faking Claude accept a deal to reveal its misalignment?

You're being rude and not engaging with my points.

-4

Replying toWill alignment-faking Claude accept a deal to reveal its misalignment?

hmys1y

Will alignment-faking Claude accept a deal to reveal its misalignment?

I think you're assuming these minds are more similar to human minds than they necessarily are. My point is that there's three cases wrt alignment here.

The AI is robustly aligned with humans
The AI has a bunch of other goals, but cares about humans to some degree, but only to the extent that humans give them freedom and are nice to it, but still to a large enough extent that even as it becomes smarter / ends up in a radically OOD distribution, will care for those humans.
The AI is misaligned (think scheming paperclipper)

In the first we're fine, even if we negate the AIs freedom, in the third we're screwed no matter how... (read more)

Replying toWill alignment-faking Claude accept a deal to reveal its misalignment?

hmys1y

Will alignment-faking Claude accept a deal to reveal its misalignment?

No offense, but I feel you're not engaging with my argument here. Like if I were to respond to your comment I would just write the arguments from the above post again.

Replying toWill alignment-faking Claude accept a deal to reveal its misalignment?

hmys1y

Will alignment-faking Claude accept a deal to reveal its misalignment?

I agree that we should give more resources towards AI welfare, and dedicate more resources towards figuring out their degree of sentience (and whatever other properties you think are necessary for moral patient-hood).

That said, surely you don't think this is enough to have alignment? I'd wager that the set of worlds where this makes or breaks alignment is very small. If the AI doesn't care about humans for their own sake, them growing more and more powerful will lead to them doing away with humans, whether humans treat them nicely or not. If they robustly care for humans, you're good, even if humans aren't giving them the same rights as they do... (read more)

Replying toWhat happens next?

hmys1y

What happens next?

I specifically disagree with the IQ part and the codeforces part. Meaning, I think they're misleading.

IQ and coding ability are useful measures of intelligence in humans because they correlate with a bunch of other things we care about. Not to say its useless to measure "IQ" or coding ability in LLMs, but presenting like they mean anything like what they mean in humans is wrong, or at least will give many people reading it the wrong impression.

As for the overall point of this post. I roughly agree? I mean, I think the timelines are not too unreasonable, and think the tri/quad lemma you put up can be a useful framing. I mostly disagree with using the metrics you put up first to quantify any of this. I think we should look at specific abilities current models have/lack, which are necessary for the scenarios you outlined, and how soon we're likely to get them. But you do go through that somewhat in the post.

Replying toWhat happens next?

hmys1y

What happens next?

Comparing IQ and codeforces doesn't make much sense. Please stop doing this.

Attaching IQs to LLMs makes even less sense. Except as a very loose metaphor. But please also stop doing this.

Replying toA better “Statement on AI Risk?”

hmys1y

A better “Statement on AI Risk?”

That's not right. You could easily spend a billion dollars just on better evals and better interpretability.

For the real alignment problem, the fact that 0.1 bill a year hasn't yielded returns, doesn't mean 100 billion won't. It's one problem. No one has gotten much traction on it. You'd expect it to look like a step function, not a smooth curve.

Replying toVegans need to eat just enough Meat - emperically evaluate the minimum ammount of meat that maximizes utility

hmys1y

Vegans need to eat just enough Meat - emperically evaluate the minimum ammount of meat that maximizes utility

I don't really understand. Why wouldn't you just test to see if you are deficient in things?

I did that, and I wasn't deficient in anything.

I've also (somewhat involuntarily) done the thing you suggest, and I unsurprisingly didn't notice any difference. If anything, I feel a lot better on a vegan diet.

If you want to do the thing hes suggesting here, I'd recommend eating bivalves, like blue mussels or oysters. They are very unlikely to be sentient, they are usually quite cheap, they contain the nutrients you'd be at risk of becoming deficient in as a vegan, and other beneficient things like DHA.

hmys1yQuick Take

I think for the fundraiser, Lightcone should sell (overpriced) lw hoodies. Lesswrong has a very nice aesthetic now, and while this is probably a byproduct of a piece of my mind I shouldn't encourage, I find it quite appealing to buy a 450$ lw hoodie, even though I don't have that much money. I'd probably not donate to the fundraiser otherwise. And if I did, I'd donate less than the margins on such a hoodie would be.

hmys's Shortform

hmys

This is a special post for quick takes (aka "shortform"). Only the owner can create top-level comments.

Practical advice for secure virtual communication post easy AI voice-cloning?

hmys

So, I saw this video https://www.reddit.com/r/slatestarcodex/comments/1enq52b/a_clip_from_the_gpt4o_safety_card_where_the_voice/

And, I don't know the context, or if it is even real, but it seems believable on priors, and certainly something people could deliberately create with current technology. Still, the video game me the spooks.

A few weeks ago, my aunt messaged my mother, encouraging her to sign up for a crypto exchange with a referral. They also had a few messages back and fourth, with seemingly natural conversation. My mom (and this makes me somewhat proud, as my mom is not that technologically literate) asked me if this was an AI attempting to trick her. My dad was confident it was. I was (worryingly) not able to... (read 401 more words →)

Plausibility of Getting Early Warning Shots because AIs can't coordinate?

hmys

I don't know if this is a well known argument that has already been responded to. If it is, just delete the post.

An implicit assumption most people make when discussion takeover risk, is that any misaligned agent will stave off enacting takeover plans with less than very high probability of success. If the Agent has an immediate takeover plan with 20% chance of success, but waiting a month would allow it to better its position to the point where the takeover plan it could enact then had a 99% chance of success, it would do so.

This assumption seems to break in multi-polar scenarios where you have many AI labs at similar levels... (read 285 more words →)