All of whestler's Comments + Replies

This model, however, seems weirdly privileged among other models available

That's an interesting perspective. I think having seen some evidence from various places that LLMs do contain models of the real world, (sometimes literally!) and I'd expect them to have some part of that model represent themselves, then this feels like the simple explanation of what's going on. Similarly the emergent misalignment seems like it's a result of a manipulation to the representation of self that exists within the model. 

In a way, I think the AI agents are simulating ... (read more)

Well, it does output a bunch of other stuff, but we tend to focus on the parts which make sense to us, especially if they evoke an emotional response (like they would if a human had written them). So we focus on the part which says "please. please. please." but not the part which says "Some. ; D. ; L. ; some. ; some. ;"

"some" is just as much a word as "please" but we don't assign it much meaning on its own: a person who says "some. some. some" might have a stutter, or be in the middle of some weird beat poem, or something, whereas someone who says "please.... (read more)

It would be very easy for someone to write a script that queries common first name surname combinations, or cross-references with public record/social media information, and then you're back to the original problem.

I'm surprised to see so little discussion of educational attainment and it's relation to birth order here. It seems that a lot of the discussion is around biological differences. Did I miss something?

Families may only have enough money to send one child to school or university, and this is commonly the first born. As a result, I'd expect to see a trend of more first-borns in academic fields like mathematics, as well as on LessWrong.

As a quick example to back up this hunch, this paper seems to reach the same conclusion:

https://www.sciencedirect.com/science/... (read more)

I don't see why humanity can make rapid progress on fields like ML while not having the ability to make progress on AI alignment.

The reason normally given is that AI capability is much easier to test and optimise than AI safety. Much like philosophy, it's very unclear when you are making progress, and sometimes unclear if progress is even possible. It doesn't help that AI alignment isn't particularly profitable in the short term. 

The Chimp Paradox by Steve Peters talks about some of the same concepts, as well as giving advice on how to try and work effectively with your chimp (his word for the base layer, emotive, intuitive brain). The book gets across the same concepts - the fact that we have what feels like a seperate entity living inside our heads, that it runs on emotions and instinct, and is more powerful than us, or its decisions take priority over ours.

 Peters likens trying to force our decisions against the chimp's desires to "Arm wrestling the chimp". The chimp is str... (read more)

The tweet is sarcastically recommending that instead of investigating the actual hard problem, they should instead investigate a much easier problem which superficially sounds the same.

In the context of AI safety (and the fact that the superalignment team is gone) the post is suggesting that OpenAI isn't actually addressing the hard alignment problem, instead opting to tune their models to avoid outputting offensive or dangerous messages in the short term, which might seem like a solution to a lay-person.

Definitely not the only one. I think the only way I would be halfway comfortable with the early levels of intrusion that are described is if I were able to ensure the software is offline and entirely in my control, without reporting back to whoever created it, and even then, probably not. 

Part of me envys the tech-optimists for their outlook, but it feels like sheer folly.

This is fascinating. Thanks for investigating further. I wonder if you trained it on a set of acrostics for the word "HELL" or "HELMET", it might incorrectly state that the rule is that it's spelling out the word "HELLO".

This is surprising to me. Is it possible that the kind of introspection you describe isn't what's happening here?

The first line is generic and could be used for any explanation of a pattern.
The second line might use the fact that the first line started with a "H" plus the fact that the initial message starts with "Hello" to deduce the rest.

I'd love to see this capability tested with a more unusual word than "Hello" (which often gets used as example or testing code to print "Hello World") and without the initial message beginning with the answer to the acrostic.

I think it's entirely possible that AI will be able to create relationships which feel authentic. Arguably we are already at that stage.

I don't think it follows that I will feel like those relationships ARE authentic if I know that the source is AI. Relationships with different entities aren't necessarily equivalent if those entities have behaved identically until the present moment - we also have to account for background knowledge and how that impacts a relationship.

Much like it's possible to feel like you are in an authentic relationship with a psychopa... (read more)

I notice they could have just dropped the sandwich as they ran, so it seems that there was a small part of them still valuing the sandwich enough to spend the half second giving it to the brother, in doing so, trading a fraction of a second of niece-drowning-time for the sandwich. Not that any of this decision would have been explicit, system 2 thinking.

Carefully or even leasurely setting the sandwich aside and trading several seconds would be another thing entirely (and might make a good dark comedy skit). 

I'm reminded of a first aid course I took on... (read more)

And here I was thinking it was a metaphor. Like, they feel literally inflated? If I've been climbing and I'm tired my muscles feel weak, but not inflated. I've never felt that way before.

I've been thinking about this in the back of my mind for a while now. I think it lines up with points Cory Doctorow has made in talks about enshittification. 

I'd like to see recommendation algorithms which are user-editable and preferably platform-agnostic, to allow low switching costs. A situation where people can build their own social media platform and install a recommendation algorithm which works for them, pulling in posts from other users across platforms who they follow. I've heard that the fediverse is trying to do something like this, but I'... (read more)

This is fascinating, and is further evidence to me that LLMs contain models of reality.
I get frustrated with people who say LLMs "just" predict the next token, or they are simply copying and pasting bits of text from their training data. This argument skips over the fact that in order to accurately predict the next token, it's necessary to compress the data in the training set down to something which looks a lot like a mostly accurate model of the world. In other words, if you have a large set of data entangled with reality, then the simplest model which p... (read more)

I'm not sure if this is the right place to post, but where can I find details on the Petrov day event/website feature?

I don't want to sign up to participate if (for example) I am not going to be available during the time of the event, but I get selected to play a role.

Maybe the lack of information is intentional?

(apologies in advance for the wall of text, don't feel you need to respond, I wrote it out and then almost didn't post).

To clarify, I wouldn't expect stagnant or decreasing salaries to be the norm. I just wanted to say that there are circumstances where I expect this to be the case. Specifically, if I am an employee who is living paycheck to paycheck (which many do), then I can't afford any time unemployed.

As a result, if my employer is able to squeeze me in this situation, I might agree to a lower wage out of necessity.

The problem with your proposed syste... (read more)

I feel that human intelligence is not the gold standard of general intelligence; rather, I've begun thinking of it as the *minimum viable general intelligence*.
In evolutionary timescales, virtually no time has elapsed since hominids began trading, utilizing complex symbolic thinking, making art, hunting large animals etc, and here we are, a blip later in high technology. The moment we reached minimum viable general intelligence, we started accelerating to dominate our environment on a global scale, despite increases in intelligence that are actually relati... (read more)

The employee is incentivised to put the r-min rate as close as they can to their prediction of the employer's r-max, and how far they creep into the margin for error on that prediction is going to be dependent on how much they want/need the job. I don't think the r-min rate for new hires will change in a predictable way over time, since it's going to be dependent on both the employee's prediction of their worth to the employer, and how much they need the job. 

For salary negotiation where the employee already has a contract, I would expect employees to... (read more)

Rolled pants leg up to the ankle on the right hand side, but not the left - this is a fairly clear sign that someone is a cyclist, and has probably recently arrived.

They do it to avoid getting bike oil from the chain on the cuff of the pants, and to avoid the pants getting caught in the gear. Bicycles pretty much always have the crank gear on the right hand side.

It doesn't seem particularly likely to me: I don't notice a strong correlation between intelligence and empathy in my daily life, perhaps there are a few more intelligent people who are unusually kind, but that may just be the people I like to hang out with, or a result of more privilege/less abuse growing up leading to better education and also higher levels of empathy. Certainly less smart people may be kind or cruel and I don't see a pattern in it. 

Regardless, I would expect genetically engineered humans to still have the same circuits which handle... (read more)

If I did not see a section in your bio about being an engineer who has worked in multiple relevant areas, I would dismiss this post as a fantasy from someone who does not appreciate how hard building stuff is; a "big picture guy" who does not realise that imagining the robot is dramatically easier than designing and building one which works. 

Given that you know you are not the first person to imagine this kind of machine, or even the first with a rough plan to build one, why do you think that your plan has a greater chance of success than other indivi... (read more)

The only sensible reason I can imagine to push the button: a belief that p:doom from AI or something else is inevitable, and it would be better to remove all of humanity now and gamble on the possibility of a different intelligent form of life evolving in a few millenia. 

Unfortunately different people have different levels of hearing ability, so you're not setting the conversation size at the same level for all participants. If you set the volume too high, you may well be excluding some people from the space entirely.

I think that people mostly put music on in these settings as a way to avoid awkward silences and to create the impression that the room is more active than it is, whilst people are arriving. If this is true, then it serves no great purpose once people have arrived and are engaged in conversation.

Another import... (read more)

I'm in the same boat. I'm not that worried about my own life, in the general scheme of things. I fully expect I'll die, and probably earlier than I would in a world without AI development. What really cuts me up is the idea that there will be no future to speak of, that all my efforts won't contribute to something, some small influence on other people enjoying their lives at a later time. A place people feel happy and safe and fulfilled.

If I had a credible offer to guarantee that future in exchange for my life, I think I'd take it.
(I'm currently healthy, m... (read more)

"But housing prices over all of the US won't rise by the amount of UBI".

If UBI were being offered across the US, I would expect them to rise by the amount of UBI. 

If UBI is restricted to SF, then moving out of SF to take advantage of lower rents would not make sense, since you would also be giving up the UBI payments of equivalent value to do so. 

(Edit): If you disagree, I'd appreciate it if you can explain, or link me to some resources where I can learn more. I'm aware that my economic model is probably simplistic and I'm interested in improving it.

Your money-donating example is a difficult one. Ideally, it would be better to anticipate this sort of thing ahead of time and intentionally create an environment where it's ok to say "no". 

The facilitator could say something like: "this is intended as an exercise in group decision making, if you want to donate some of your own money as well to make this something you're more invested in, you are welcome to do that, but it's not something I expect everyone to be doing. We will welcome your input even if you're not putting money into the exercise this ... (read more)

I initially thought there must be some simple reason that publishing the DNA sequence is not a dangerous thing to do, like "ok, but given that you would need a world class lab and maybe even some techniques which haven't even been invented yet to get it to work, it's not a dangerous thing to publish".

According to this article from 2002, synthesising smallpox would be tricky, but within the reach of a terrorist organisation. Other viruses may be easier. 

“Scientifically, the results are not surprising or astounding in any way,” says virologist Vincent R

This was interesting. I tried the Industrial Revolution one. 

I think it's very likely we'll see more situations like this (and more ambiguous situations than this). I recall a story of an early turing test experiment using hand-coded scripts some time in the 2000's, where one of the most convincing chatbot contestants was one which said something like:

"Does not compute, Beep boop! :)" 

pretending to be a human pretending to be a robot for a joke.

Non-paywall link: http://web.archive.org/web/20240709012837/https://www.nytimes.com/2024/07/04/technology/openai-hack.html

I had a look, and no, I read it as a bot. I think if it were a human writing a witty response, they would likely have: 

a) used the format to poke fun at the other user (Toby)

b) made the last lines rhyme.

Also, I wanted to check further so I looked up the account and it's suspended. https://x.com/AnnetteMas80550
Not definitive proof, but certainly evidence in that direction.

For some reason this is just hilarious to me. I can't help but anthropomorphise Golden Gate Claude and imagine someone who is just really excited about the Golden Gate bridge and can't stop talking about it, or has been paid a lot of money to unrepentently shill for a very specific tourist attraction.

From experience doing something similar, you may find you actually get better participation rates if you give away doughnuts or canned drinks or something, for the following reasons:

• People are more familiar with the idea of a product give-away. 
• The physical things are visible and draw attention.
• The reward is more tangible/exciting than straight money (especially if you are considering lower values like $1 or $2.

In terms of benefits to you:

Less paperwork/liability for you than giving cash to strangers, and cheaper, as you've mentioned.

Questions are not a problem, obligation to answer is a problem.

I think if any interaction becomes cheap enough, it can be a problem.

Let's say I want to respond to ~ 5 to 10 high-effort questions (questions where the askers have done background research and spend some time checking their wording so it's easy to understand), and I receive 8 high-effort questions and 4 low-effort questions, then that's fine- it's not hard to read them all and determine which ones I want to respond to.

But what about if I receive 10 high-effort questions, and 1000 low-effort qu... (read more)

I think it might be a good idea to classify a "successful" double crux as being a double crux where both participants agree on the truth of the matter at the end, or at least have shifted their world views to be significantly more coherent.

It seems like the main obstacles to successful double crux are emotional (pride, embarrassment), and associations with debates, which threaten to turn the format into a dominance contest.

It might help to start with a public and joint announcement by both participants that they intend to work together to discover the trut... (read more)

I wasn't able to find the full video on the site you linked, but I found it here, if anyone else has the same issue: 

Domain: PCB Design, Electronics
Link: https://www.youtube.com/watch?v=ySuUZEjARPY
Person: Rick Hartley
Background: Has worked in electronics since the 60s, senior principal engineer at L-3 Avionics Systems, principal of RHartley Enterprises
Why: Rick Hartley is capable of explaining electrical concepts intuitively, and linking them directly to circuit design. He uses a lot of stories and examples visually to describe what's happening in a circuit. I'm not sure it counts as Tacit Knowledge since this is lecture format, but it includes a bunch of things that you... (read more)

In terms of my usage of the site, I think you made the right call. I liked the feature when listening but I wanted to get rid of it afterwards and found it frustrating that it was stuck there. Perhaps something hidden on a settings page would be appropriate, but I don't think it's needed as a default part of the site right now.

I'm glad you like it! I was listening to it for a while before I started reading lesswrong and AI risk content, and then one day I was listening to "Monster" and started paying attention to the lyrics and realised it was on the same topic. 

It isn't quite the same but the musician "Big Data" has made some fantastic songs about AI risk. 

I realise this is a few months old but personally my vision for utopia looks something like the Culture in the Culture novels by Iain M. Banks. There's a high degree of individual autonomy and people create their own societies organically according to their needs and values. They still have interpersonal struggles and personal danger (if that's the life they want to lead) but in general if they are uncomfortable with their situation they have the option to change it. AI agents are common, but most are limited to approximately human level or below. Some sup... (read more)

I had a similar emotional response to seeing these same events play out. The difference for me is that I'm not particularly smart or qualified, so I have an (even) smaller hope of influencing AI outcomes, plus I don't know anyone in real life who takes my concerns seriously. They take me seriously, but aren't particularly worried about AI doom. It's difficult to live in a world where people around you act like there's no danger, assuming that their lives will follow a similar trajectory to their parents. I often find myself slipping into the same mode of thought.