All of Épiphanie Gédéon's Comments + Replies

Remember, while the ARA models are trying to survive, there will be millions of other (potentially misaligned) models being deployed deliberately by humans, including on very sensitive tasks (like recursive self-improvement). These seem much more concerning.

So my main reason for worry personally is that there might be an ARA that is deployed with just the goal of "just clone yourself as much as possible" or a goal similar to this. In this case, the AI does not really have to survive particularly among others as long as it is able to pay for itself and c... (read more)

Thanks a lot for the kind comment!

To scale this approach, one will want to have "structural regularizers" towards modularity, interoperability and parsimony

I am unsure of the formal architecture or requirements for these structural regularizers you mention. I agree with using shared building blocks to speed up development and verification. I am unsure credit assignment would work well for this, maybe in the form of "the more a block is used in a code, the more we can trust it"?

Constraints on the types of admissible model code. We have strongly advoca

Yes, thank you, it's less than 1000 parameters for both

So the first image is based on AI control, which is indeed part of their strategies, and you could see constructability as mainly leading to this kind of strategy applied to plain code for specific subtasks. It's important to note constructability itself is just a different approach to making understandable systems.

The main differences are :

1. Instead of using a single AI, we use many expert-like systems that compose together which we can see the interaction of (for instance, in the case of a go player, you would use KataGo to predict the best move and fla

Thinking more on this and to friends who voiced that non-trees/dag dialog makes it non appealing to them also (especially in contexts where I was inviting them to have dialogue together), would there be interests in making a PR for this feature? I might work on this directly

Came here to comment this. As it is, this seems like just talking on discord/telegram, but with the notion of publishing it later. What I really lack when discussing something is the ability to branch out and backtrack easily and have a view of all conversation topics at once.

That being said, I really like the idea of dialogues, and I think this is a push in the right direction, I have enjoyed the dialogue I read so far greatly. Excited to see where this goes

Reposting (after a slight rewrite) from the telegram group:

This might be a nitpick, but to my (maybe misguided) understanding, alignment is only a very specific subfield of ai safety research, which basically boils down to "how do I give a set of rules/utility function/designs that avoid meta or mesa optimizations that have dramatic unforseen consequences" (This is at least how I understood MIRI's focus pre-2020)

For instance, as I understand it, interpretability research is not directly alignment research. Instead, it is part of the broader "AI safety rese... (read more)

(

Also, I wouldn’t ban orgies, that won’t work

I'm not sure on that point anymore. Monkeypox' spread seems so slow, and the fact that only men gets infected while the proportions of women who have sex with men having sex with men is not that narrow. I wonder if the spread of Monkeypox is not mostly driven by orgies/big events at the moment, and if it's not the best moment to do this. Though it might be a bit too late now, I think it'd have worked two-four weeks ago)

People can totally understand all of this, and also people mostly do understand most of this

On the other hand, the very fact that we say Monkeypox is spreading within the community of Men having Sex with Men is symbolic of the problem, to me. Being MSM and being in this monkeypox-spreading community is very correlated sure, but not synonymous, the cluster we're talking about is more specific: It's the cluster of people participating in orgies, have sex with other strangers several times a week, etc.

Seeing how we're still conflating the two in discussion... (read more)

The more I think about it, the more I wonder if boiling the potatoes infused them with the peels and increased significantly the quantity of solanine I was consuming. An obvious confounder is that whole-boiled potatoes are less fun to eat than in more varied forms, so it doesn't discriminate with the "fun food" theory

Thanks a lot for the estimate, I'll look into recent studies of this to see what I find!

Isn't it common for people who fast for more than 3-5 days not to feel any hunger? I wonder if there's a similar mechanism here

very restrictive diets are very socially costly to follow. If you regularly eat from college dining halls, cafeterias at work, restaurants, other people's homes, etc. you'll have a very hard time following an all-potato diet.

To be fair to SMTM's potato diet, the idea is that it still works even if you cheat a lot.
That was somewhat my experience with the diet though, it makes social interaction a lot more awkward

semaglutide and tirzepatide alone would massively reduce obesity rates if they were more popular

I mean, the idea of a cheap, not very effor... (read more)

"what would the cost/benefit analysis of address every single thing in this class of problems?",

I mean, that is why I included "More research to shrink our unknown unknowns" as a general category. I do not think the research needs to be thorough, 3-4 very broad general area would suffice, but even if that does not fit into the points you mention, a statement along the lines you mentioned could work. In general, I do not think that more than one-two hours should be spent on writing this warning.

As for "If at any point you get sick or begin having side ef... (read more)

it feels unfair judging them on your specific case because they did tell you not to do it

I notice I'm very confused. SMTM said not to do it because of bipolarity:

Yeah, we would actually recommend people with bipolar disorder not enroll, both because there might be interactions with medication (especially if we’re right about the lithium thing) and because the potato diet seems to sometimes trigger hypomania even in people without bipolar

(The reason I chose to do it is that I am not under medication and that I am type 2 bipolar, not type 1)

I did en... (read more)

Right, I must have phrased it poorly.

What I mean is that if the goal was just to explore, and modifying the diet while doing it is not a problem, then it would have made more sense to have access to everyone's data as they were doing it (at least between participants). Also have more ways to communicate between participants to share our experience, tips and tricks, etc...

I feel that, while you went a level of meta up, this article really encapsulates why I am so hesitant about EA. I have several concerns about VNM utilitarianism applied to a global monolithic scope. My experience discussing them in the EA space is people looking at me funny and something along the lines of "How can you be against it though?"

By far the main problem I have, which I feel you pointed here elegantly, is how prone EA is to congratulate itself on being so willing to change its mind, without a broader interrogation of what that even means. (I rem... (read more)

I thought they were pretty upfront about how this was exploratory (trying to generate hypotheses and see if the effect is plausible, not confirm a hypothesis).

Also, yes, that was my impression as well and what I signed up for. It's just that a lot of features of the study directly clash with this exploration.

(I also disagree that their results are that remarkable.)

Edit: By remarkable, I mean "shows that it is specific to potatoes"

Typo, I meant chips/crisp. Thanks!

Then shouldn't it be weirder? Like, having full open data, not trying a month specifically and varying this, etc?

I think doing this kind of thing is really important because trying to make everything safe means we frequently don't do useful science

That is a very good point, and I agree.

Two things on this point:

• It did not seem to me that SMTM marked this as clearly unsafe, but I am not neutral enough to know. Just: There is a difference between "quit if you feel unwell" and "This might have drastic negative impact on you you will not even notice or fe