All of AnthonyC's Comments + Replies

This does not imply that the simulation is run entirely in linear time, or at a constant frame rate (or equivalent), or that details are determined a priori instead of post hoc. It is plausible such a system could run a usually-convincing-enough simulation at lower fidelity, back-calculate details as needed, and modify memories to ignore what would have been inconsistencies when doing so is necessary or just more useful/tractable. 'Full detail simulation at all times' is not a prerequisite for never being able to find and notice a flaw, or for getting many... (read more)

That's true, and you're right, the way I wrote my comment overstates the case. Every individual election is complicated, and there's a lot more than one axis of variation differentiation candidates and voters. The whole process of Harris becoming the candidate made this particular election weird in a number of ways. And as a share of the electorate, there are many fewer swing voters than there used to be a few decades ago, and not conveniently sorted into large, coherent blocks.

And yet, it's also true that as few as ~120,000 votes in WI, MI, and PA could h... (read more)

Apparently the new ChatGPT model is obsessed with the immaculate conception of Mary

I mean, "shoggoth" is not that far off from biblically accurate angels... ;-)

I'd say that in most contexts in normal human life, (3) is the thing that makes this less of an issue for (1) and (2). If the thing I'm hearing about it real, I'll probably keep hearing about it, and from more sources. If I come across 100 new crazy-seeming ideas and decide to indulge them 1% of the time, and so do many other people, that's usually, probably enough to amplify the ones that (seem to) pan out. By the time I hear about the thing from 2, 5, or 20 sources, I will start to suspect it's worth thinking about at a higher level.

Exactly. More fundamentally, that is not a probability graph, it's a probability density graph, and we're not shown the line beyond 2032 but just have to assume the integral from 2100-->infinity is >10% of the integral from 0-->infinity. Infinity is far enough away that the decay doesn't even need to be all that slow for the total to be that high.

I second what both @faul_sname and @Noosphere89 said. I'd add: Consider ease and speed of integration. Organizational inertia can be a very big bottleneck, and companies often think in FTEs. Ultimately, no, I don't think it makes sense to have anything like 1:1 replacement of human workers with AI agents. But, as a process occurring in stages over time, if you can do that, then you get a huge up-front payoff, and you can use part of the payoff to do the work of refactoring tasks/jobs/products/companies/industries to better take advantage of what else AI le... (read more)

This was really interesting, thanks! Sorry for the wall of text. TL:DR version: 

I think these examples reflect, not quite exactly willingness to question truly fundamental principles, but an attempt at identification of a long-term vector of moral trends, propagated forward through examples. I also find it some combination of suspicious/comforting/concerning that none of these are likely to be unfamiliar (at least as hypotheses) to anyone who has spent much time on LW or around futurists and transhumanists (who are probably overrepresented in the avai... (read more)

Credit cards are kind of an alternative to small claims court, and there are various reputational and other reasons that allow ordinary business to continue even if it is not in practice enforced by law.

True, but FWIW this essentially puts unintelligible enforcement in the hands of banks instead of the police. Which is probably a net improvement, especially under current conditions. But it does have its own costs. My wife is on the board of a nonprofit that last year got a donation, then the donor's spouse didn't recognize the charge and disputed it. The d... (read more)

As things stand today, if AGI is created (aligned or not) in the US, it won't be by the USG or agents of the USG. I'll be by a private or public company. Depending on the path to get there, there will be more or less USG influence of some sort. But if we're going to assume the AGI is aligned to something deliberate, I wouldn't assume AGI built in the US is aligned to the current administration, or at least significantly less so than the degree to which I'd assume AGI built in China by a Chinese company would be aligned to the current CCP. 

For more con... (read more)

I won't comment on your specific startup, but I wonder in general how an AI Safety startup becomes a successful business. What's the business model? Who is the target customer? Why do they buy? Unless the goal is to get acquired by one of the big labs, in which case, sure, but again, why or when do they buy, and at what price? Especially since they already don't seem to be putting much effort into solving the problem themselves despite having better tools and more money to do so than any new entrant startup.

I really, really hope at some point the Democrats will acknowledge the reason they lost is that they failed to persuade the median voter of their ideas, and/or adopt ideas that appeal to said voters. At least among those I interact with, there seems to be a denial of the idea that this is how you win elections, which is a prerequisite for governing.

That seems very possible to me, and if and when we can show whether something like that is the case, I do think it would represent significant progress. If nothing else, it would help tell us what the thing we need to be examining actually is, in a way we don't currently have an easy way to specify.

If you can strike in a way that prevents retaliation that would, by definition, not be mutually assured destruction.

Correct, which is in part why so much effort went into developing credible second strike capabilities, building up all parts of the nuclear triad, and closing the supposed missile gap. Because both the US and USSR had sufficiently credible second strike capabilities, it made a first strike much less strategically attractive and reduced the likelihood of one occurring. I'm not sure how your comment disagrees with mine? I see them as two sides of the same coin.

If you live in Manhattan or Washington DC today, you basically can assume you will be nuked first, yet people live their lives. Granted people could behave differently under this scenario for non-logical reasons.

My understanding is that in the Cold War, a basic MAD assumption was that if anyone were going to launch a first strike, they'd try to do so with overwhelming force sufficient to prevent a second strike, hitting everything at once.

I agree that consciousness arises from normal physics and biology, there's nothing extra needed, even if I don't yet know how. I expect that we will, in time, be able to figure out the mechanistic explanation for the how. But right now, this model very effectively solves the Easy Problem, while essentially declaring the Hard Problem not important. The question of, "Yes, but why that particular qualia-laden engineered solution?" is still there, unexplained and ignored. I'm not even saying that's a tactical mistake! Sometimes ignoring a problem we're not yet equipped to address is the best way to make progress towards getting the tools to eventually address it. What I am saying is that calling this a "debunking" is misdirection.

I've read this story before, including and originally here on LW, but for some reason this time it got me thinking: I've never seen a discussion about what this tradition meant for early Christianity, before the Christians decided to just declare (supposedly after God sent Peter a vision, an argument that only works by assuming the conclusion) that the old laws no longer applied to them? After all, the Rabbi Yeshua ben Joseph (as the Gospels sometimes called him) explicitly declared the miracles he performed to be a necessary reason for why not believing in him was a sin.

We apply different standards of behavior for different types of choices all the time (in terms of how much effort to put into the decision process), mostly successfully. So I read this reply as something like, "Which category of 'How high a standard should I use?' do you put 'Should I lie right now?' in?"

A good starting point might be: One rank higher than you would for not lying, see how it goes and adjust over time. If I tried to make an effort-ranking of all the kinds of tasks I regularly engage in, I expect there would be natural clusters I can roughly... (read more)

One of the factors to consider, that contrasts with old-fashioned hostage exchanges as described, is that you would never allow your nation's leaders to visit any city that you knew had such an arrangement. Not as a group, and probably not individually. You could never justify doing this kind of agreement for Washington DC or Beijing or Moscow, in the way that you can justify, "We both have missiles that can hit anywhere, including your capital city." The traditional approach is to make yourself vulnerable enough to credibly signal unwillingness to betray ... (read more)

It’s a subtle thing. I don’t know if I can eyeball two inches of height.

Not from a picture, but IRL, if you're 5'11" and they claim 6'0", you can. If you're 5'4", probably not so much. Which is good, in a sense, since the practical impact of this brand of lying on someone who is 5'4" is very small, whereas unusually tall women may care whether their partner is taller or shorter than they are. 

This makes me wonder what the pattern looks like for gay men, and whether their reactions to it and feelings about it are different than straight women.

Lie by default whenever you think it passes an Expected Value Calculation to do so, just as for any other action. 

How do you propose to approximately carry out such a process, and how much effort do you put into pretending to do the calculation?

I'm not as much a stickler/purist/believer in honest-as-always-good as many around here, I think there are many times that deception of some sort is a valid, good, or even morally required choice. I definitely think e.g. Kant was wrong about honesty as a maxim, even within his own framework. But, in practice, I... (read more)

I personally wouldn't want to do a PhD that didn't achieve this!

Agreed. It was somewhere around reason #4 I quit my PhD program as soon as I qualified for a masters in passing.

Any such question has to account for the uncertainty about what US trade policies and tariffs will be tomorrow, let alone by the time anyone currently planning a data center will actually be finished building it.

Also, when you say offshore, do you mean in other countries, or actually in the ocean? Assuming the former, I think that would imply using the data center by anyone in the US would be an import of services. If this starting happening at scale, I would expect the current administration to immediately begin applying tariffs to those services.

@Garrett... (read more)

Do you really expect that the project would then fail at the "getting funded"/"hiring personnel" stages?

Not at all, I'd expect them to get funded and get people. Plausibly quite well, or at least I hope so!

But when I think about paths by which such a company shapes how we reach AGI, I find it hard to see how that happens unless something (regulation, hitting walls in R&D, etc.) either slows the incumbents down or else causes them to adopt the new methods themselves. Both of which are possible! I'd just hope anyone seriously considering pursuing such a ... (read more)

1. You're right, but creating unexpected new knowledge is not a PhD requirement. I expect it's pretty rare that a PhD students achieves that level of research.
2. It wasn't a great explanation, sorry, and there are definitely some leaps, digressions, and hand-wavy bits. But basically: Even if current AI research were all blind mutation and selection, we already know that that can yield general intelligence from animal-level-intelligence because evolution did it. And we already have various examples of how human research can apply much greater random and non-rando

I'm not a technical expert by any means, but given what I've read I'd be surprised if that kind of research were harmful. Curious to hear what others say.

I recently had approximately this conversation with my own employer's HR department. We're steadily refactoring tasks to find what can be automated, and it's a much larger proportion of what our entry-level hires do. Current AI is an infinite army of interns we manage, three years ago they were middle school age interns and now they're college or grad school interns. At some point, we don't know when, actually adding net economic value will require having the kinds of skills that we currently expect people to take years to build. This cuts off the pipeline... (read more)

I also don't have a principled reason to expect that particular linear relationship, except in general in forecasting tech advancements, I find that a lot of such relationships seem to happen and sustain themselves for longer than I'd expect given my lack of principled reasons for them.

I did just post another comment reply that engages with some things you said. 

To the first argument: I agree with @Chris_Leong's point about interest rates constituting essentially zero evidence, especially compared to the number of data points on the METR graph.

To the ... (read more)

Personally I think 2030 is possible but aggressive, and my timeline estimate it more around 2035. Two years ago I would have said 2040 or a bit later, and capabilities gains relevant to my own field and several others I know reasonably well have shortened that, along with the increase in funding for further development.

• The Claude/Pokemon thing is interesting, and overall Pokemon-playing trend across Anthropic's models is clearly positive. I can't say I had any opinion at all about how far along an LLM would get at Pokemon before that result got publicized,

Yes, the reasoning models seem to have accelerated things. ~7 months to ~4 months doubling time on that plot. I'm still not sure I follow why "They found a second way to accelerate progress that we can pursue in parallel to the first" would not cause me to think that progress in total will thereafter be faster. The advent of reasoning models has caused an acceleration of increasing capabilities, not in one or two domains like chess, but across a broad range of domains.

I think @tailcalled hit the main point and it would be a good idea to revisit the entire "Why not just..." series of posts.

But more generally, I'd say to also revisit Inadequate Equilibria for a deeper exploration of the underlying problem. Let's assume you or anyone else really did have a proposed path to AGI/ASI that would be in some important senses safer than our current path. Who is the entity for whom this would or would not be a "viable course?" Who would need to be doing the "considering" of alternative technologies, and what is the process by whic... (read more)

No worries, I appreciate the concept and think some aspects of it are useful. I do worry at a vibes level that if we're not precise about which human-child-rearing methods we expect to be useful for AI training, and why, we're likely to be misled by warm fuzzy feelings.

And yes, that's true about some (maybe many) humans' vengeful and vindictive and otherwise harmful tendencies. A human-like LLM could easily be a source of x-risk, and from humans we already know that human child rearing and training and socializing methods are not universally effective at a... (read more)

We have tools for rearing children that are less smart, less knowledgeable, and in almost all other ways less powerful than ourselves. We do not have tools for specifically raising children that are, in many ways, superhuman, and that lack a human child's level of dependance on their parents or intrinsic emotional drives for learning from their peers and elders. LLMs know they aren't human children, so we shouldn't expect them to act and react like human children.

Agreed with everything in this post, but I would add that (n=4 people, fwiw) there is also a stable state on the other side of Healthy Food. It's still more expensive (though becoming less so, especially if you cook) to buy actually healthy food. But, if you are willing to spend a few months experimenting and exploring, while completely eliminating the hyperpalatable stuff, you can end up in a place where the healthiest foods taste better, and the hyperpalatable stuff makes you feel awful even in the short term. You won't automatically reach a desired weig... (read more)

It's not clear to me that these are more likely, especially if timelines are short. If we developed AI slowly over centuries? Then sure, absolutely likely. If it happens in the next 10 years? Then modifying humans, if it happens, will be a long-delayed afterthought. It's also not at all clear to me that the biological portion is actually adding all that much in these scenarios, and I expect hybridization would be a transitional state.

There's Robin Hanson's The Age of Em.

On this forum, see What Does LessWrong/EA Think of Human Intelligence Augmentation as o... (read more)

I agree that we should be polite and kind to our AIs, both on principle and also because that tends to work better in may cases.

we all labor under the mother's curse and blessing; our children shall be just like us

If I knew that to be true, then a lot of the rest of this post would indeed follow. Among other things, I could then assume away many/most sources of x-risk and s-risk from AGI/ASI. But generative AI is not just like us, it does differ in many ways, and we often don't know which of those ways matter, and how. We need to resolve that confusion and uncertainty before we can afford to let these systems we're creating run loose.

If there are no ✓ at all in the last row and column, what are those connecters for?

It sounds like you're assuming the Copenhagen interpretation of QM, which is not strictly necessary. To the best of my understanding, initially but not solely from the learned hear on LW, QM works just fine if you just don't do that and assume the wave equations are continuous work exactly as written, everywhere, all the time, just like every other law of physics. You need a lot of information processing, but not sophisticated as described here.

There's a semi-famous, possibly apocryphal, story about Feynman when he was a student. Supposedly he learned abou... (read more)

I realize this is in many ways beside the point, but even if your original belief had been correct, "The Men's and Women's teams should play each other to help resolve the pay disparity" is a non-sequitor. Pay is not decided by fairness. It's decided by collective bargaining, under constraints set by market conditions.

You mention them once, but I would love to see a more detailed comparison, not to private industry, but to advocacy and lobbying adoption and usage of AI.

As someone who very much enjoys long showers, a few words of caution.

1. Too-long or too-frequent exposure to hot water (time and temperature thresholds vary per person) can cause skin problems and make body odor worse. Since I started RVing I shower much less (maybe twice a week on average, usually only a few minutes of water flow for each) and smell significantly better, with less dry skin or acne or irritation. Skipping one shower makes you smell worse. Skipping many showers and shortening the remainder can do the opposite.
2. A shower, depending on temperature

In some senses, we have done so many times, with human adults of differing intelligence and/or unequal information access, with adults and children, with humans and animals, and with humans and simpler autonomous systems (like sprites in games, or current robotic systems). Many relationships other than master-slave are possible, but I'm not sure any of the known solutions are desirable, and they're definitely not universally agreed on as desirable. We can be the AI's servants, children, pets, or autonomous-beings-within-strict-bounds-but-the-AI-can-shut-us-down-or-take-us-over-at-will. It's much less clear to me that we can be moral or political or social peers in a way that is not a polite fiction.

So it's quite ironic if there was a version of Jesus that was embracing and retelling some of those 'heretical' ideas.

Sure, but also there are definitely things Jesus is said in the Bible to have taught and done that the church itself later condemned, rejected, or- if I'm feeling generous - creatively reinterpreted. This would be one more example, based on a related but different set of sources and arguments.

Christianity seems to me in general to be much less tolerant of its own inherent ambiguity than many other religions. Not that other faiths don't have... (read more)

Epistemic status: Random thought, not examined too closely.

I was thinking a little while ago about the idea that there are three basic moral frameworks (consequentialism, virtue ethics, deontology) with lots of permutations. It occurred to me that in some sense they form a cycle, rather than one trying to be fundamental. I don't think I've ever considered or encountered that idea before. I highly doubt this is in any way novel, and am curious how common it is or where I can find good sources that explore it or something similar.

Events are judged by their c... (read more)

I can't really evaluate the specific claims made here, I haven't read the texts or done the work to think about them enough, but reading this, The Earth became older, of a sudden. It's the same feeling I had when someone first pointed out that all the moral philosophy I'd been reading amounted to debating the same three basic frameworks (consequentialism, deontology, virtue ethics) since the dawn of writing. Maybe the same is true for the three cranes (chance, evolution, design).

Thanks, "hire"-->"higher" typo fixed.

Indeed. Major quality change from prior models.

Had a nice chat with GPT-4.5 the other day about fat metabolism and related topics. Then I asked it for an optimal nutrition an exercise plan for a hypothetical person matching either I or my wife's age, height, weight, gender, and overall distribution of body fat. It came back detailed plans, very different for each of us, and very different from anything I've seen in a published source, but which extremely closely matches the sets of disparate diets, eating routines, exercise routines, and supplements we'd stumbled upon as "things that seem to make us fe... (read more)

If you do it right, being willing to ask questions of those higher up, like said CEO, is how you get noticed, on their radar, as someone potentially worth watching and investing in and promoting in the future. A secure CEO in a healthy culture is likely to take it as a good sign that employees are aware, intelligent, and paying attention enough to ask clear, well-formed questions.

But if you ask a question in a way that offends that particular individual in whatever way, or makes your direct boss look bad to his direct boss (in either of their perceptions),... (read more)

Without a currently-implausible level of trust in a whole bunch of models, people, and companies to understand how and when to use privileged information and be able to execute it, removing the New Chat button would be a de factor ban on LLM use in some businesses, including mine (consulting). The fact that Chemical Company A asked a question about X last month is very important information that I'm not allowed to use when answering Chemical Company B's new question about the future of X, and also I'm not allowed to tell the model where either question cam... (read more)

There's an important reason to keep some of us around. This is also an important point.