LESSWRONG
LW

All of Phil Dyer's Comments + Replies

D&D.Sci
Phil Dyer92

Last night I spent a couple of hours obsessively hammering away at Excel to be the first to solve this before noticing firstly that it's three years old, and secondly that I was nowhere near solving it. 

Found it a hugely entertaining concept though, and it was truly time well spent. Before checking the thread's replies, I ended up going for Str+2, Dex+1, Con+2, Cha +5 for a 75% chance.

The most interesting part came today, when I estimated my own stats and wondered how I'd spend the 10 points on myself. 

I feel in 2024 the value lies in Int>Wis>Con>Cha>Dex>Str. In the past and future this looks way different, though.

Reply2
2abstractapplic
I'm glad you liked it! (. . . could you spoiler your strategy and win chance? I know this challenge is three years old, and what you mention here isn't game-breaking info, but I want to keep it possible for people looking/playing through the archives to seek clarifications in the comments without unwittingly/unwillingly learning anything else about the scenario.)
8aphyer
If you enjoyed the concept there have been sequels to this: if you want one that's currently being actively played I have one running now, or there's a D&D.Sci tag with a list of all past scenarios.
Malicious non-state actors and AI safety
Phil Dyer10

yet my probability of success would be absolutely tiny – like 0.01% even if I tried my absolute hardest. That's what I mean when I say that most people would have a near-zero chance. There are maybe a few hundred (??) people in the world who we even need to consider

Could you explain how you come to this conclusion? What do you think your fundamental roadblock would be? Getting the code for AGI or beating everyone else to superintelligence?]

 

My fundamental roadblock would be getting the code to AGI. My hacking skills are non-existent and I wouldn't be ... (read more)

Reply
Malicious non-state actors and AI safety
Phil Dyer10

Hi! Missed your reply for a few days. Sorry, I'm new here.

I'm not sure most people would have a near-zero chance of getting anywhere.  

I think our disagreement may stem from our different starting points. I'm considering literally every person on the planet and saying that maybe 1% of them would act malevolently given AGI. So a sadistic version of me, say, would probably be in the 98% percentile of all sadists in terms of ability to obtain AGI (I know people working in AI, am two connections away from some really key actors, have a university educatio... (read more)

Reply
1keti
Could you explain how you come to this conclusion? What do you think your fundamental roadblock would be? Getting the code for AGI or beating everyone else to superintelligence?]   It's important to remember that there may be quite a few people who would act somewhat maliciously if they took control of AGI, but I best the vast majority of these people would never even consider trying to take control of the world. I think trying to  control AGI would just be far too much work and risk for the vast majority of people who want to cause suffering. However, there still may be a  few people want to harm the world enough to justify trying. They would need to be extremely motivated to cause damage. It's a big world, though, so I wouldn't be surpized if there were a few people like this. I think that a typical, highly motivated malicious actor would have much higher than 1% probability of succeeding. (If mainstream AI research starts taking security against malicious actors super seriously, the probability of the malicious actors' success would be very low, but I'm not sure it will be taken seriously enough.) A person might not know how to hack, building botnets, or eavesdrop, but they could learn. I think a motivated, reasonably capable individual would be able to become proficient in all those things. And they potentially will have decades of training before they would need to use it.
Malicious non-state actors and AI safety
Phil Dyer10

Personally, I think this topic is worth considering since the potential downside of malevolence + AGI is so terrifying. *I have low epistemic confidence in what I’m about to say because serious thinking on the topic is only a few years old, I have no particular expertise and the landscape will probably change radically, in unpredictable ways, between now and AGI. 

For a malicious actor to establish a singleton assuming a hard takeoff, basically three conditions would be necessary: there is at least one malicious actor, at least one such actor can acqui

... (read more)
Reply
1keti
I'm not sure most people would have a near-zero chance of getting anywhere.   If AGI researchers took physical security super seriously, I bet this would make a malicious actors quite unlikely to succeed. But it doesn't seem like they're doing this right now, and I'm not sure they will start.  Theft, extortion, hacking, eavesdropping, and building botnets are things a normal person could do, so I don't see why they wouldn't have a fighting chance. I've been thinking about how someone could currently acquire private code from Google or some other current organization working on AI, and it sounds pretty plausible to me. I'm a little reluctant to go into details here due to informational hazards. What do you think the difficulties would that make most people have a near-zero chance of getting anywhere? Is it from the difficulty in acquiring the code for the AGI? Or getting a mass of hacked computers big enough to compete with AGI researchers? Both seem pretty possible to me for a dedicated individual.
1keti
Has this been discussed in detail elsewhere? I only saw one other article relating to this. I'm not sure if a regular psychopath would do anything particularly horrible if they controlled AGI. Psychopaths tend to be selfish, but I haven't heard of them being malicious. At least, I don't think a horrible torture outcome would occur. I'm more worried about people who are actually sadistic. Could you explain what the 1% chance refers to when talking about a corrupt businessman? Is it the probability that a given businessman could cause a catastrophe? I think the chance would be a lot higher if the businessman tried. Such a person could potentially just hire some criminals to do the theft, extortion, or hacking. Do you think such criminals would also just be very unlikely to succeed? Attackers just need to find a single opening, and a non-malicious organization would need to defend against many?
The Scout Mindset - read-along
Phil Dyer40

TL;DR: Choir agrees preacher’s sermon was very interesting.

So yes, I read this book with no small amount of motivation to enjoy it as I like Julia’s other stuff and am often terrified by the misery that irrationality causes. This is likely not a very impartial comment.

If we assume the goal was to achieve maximum possible swing in total human rationality*, I think it was correct to write the book with a less academic tone than some would have liked. If there had been a load more Bayes’ Theorem in it, people like me would have enjoyed it slightly more, but m... (read more)

Reply