All of Andaro's Comments + Replies

I agree. I certainly didn't mean to imply that the Trump administration is trustworthy.

My point was that the analogy of AIs merging their utility functions doesn't apply to negotiations with the NK regime.

It's not a question of timeframes, but of how likely you are to lose the war, how big the concessions would have to be to prevent the war, and how much the war would cost you even if you win (costs can have flow-through effects into the far future).

Not that any of this matters to the NK discussion.

The idea is that isolationism and destruction aren't cheaper than compromise. Of course this doesn't work if there's no mechanism of verification between the entities, or no mechanism to credibly change the utility functions. It also doesn't work if the utility functions are exactly inverse, i.e. neither side can concede priorities that are less important to them but more important to the other side.

A human analogy, although an imperfect one, would be to design a law that fulfills the most important priorities of a parliamentary majorit... (read more)

>The symmetric system is in favor of action.

This post made me think how much I value the actions of others, rather than just their omissions. And I have to conclude that the actions I value most in others are the ones that *thwart* actions of yet other people. When police and military take action to establish security against entities who would enslave or torture me, I value it. But on net, the activities of other humans are mostly bad for me. If I could snap my fingers and all other humans dropped dead (became inactive), I would instrumentally be bette... (read more)

>as the world branches, my total measure should decline many orders of magnitude every second

I'm not sure why you think that. From any moment in time, it's consistent to count all future forks toward my personal identity without having to count all other copies that don't causally branch from my current self. Perhaps this depends on how we define personal identity.

>but it doesn't affect my decision making.

Perhaps it should - tempered by the possibilities that your assumptions are incorrect, of course.

Another accounting trick: Coun... (read more)

That's a clever accounting trick, but I only care what happens in my actual future(s), not elsewhere in the universe that I can't causally affect.

>Thus, by not signing for cryonics she increases the share of her futures where she will be hostily resurrected in total share of her futures.

But she decreases the share of her futures where she will be resurrected at all, some of which contain hostile resurrection, and therefore she really decreases the share of her futures where she will be hostilely resurrected. She just won't consciously experience those where she doesn't exist, which is better than suffering from the perspective of those who consider suffering negative utility.

>It is even possible (in fact, due to resource constraints, it is likely) that they’re at odds with one another.

They're almost certainly extremely at odds with each other. Saving humanity from destroying itself points in the other direction from reducing suffering, not by 180 degrees, but at a very sharp angle. This is not just because of resource constraints, but even more so because humanity is a species of torturers and it will try to spread life to places where it doesn't naturally occur. And that life obviously will contain large amounts ... (read more)

>Our life could be eternal and thus have meaning forever.

Or you could be tortured forever without consent and without even being allowed to die. You know, the thing organized religion has spent millennia moralizing through endless spin efforts, which is now a part of common culture, including popular culture.

Let's just look at our culture, as well as contemporary and historical global cultures. Do we have:

• a consensus of consensualism (life and suffering should be voluntary)? Nope, we don't.
• a consensus of anti-torture (torturing people being il

I’m confused about OpenAI’s agenda.

Ostensibly, their funding is aimed at reducing the risk of AI dystopia. Correct? But how does this research prevent AI dystopia? It seems more likely to speed up its arrival, as would any general AI research that’s not specifically aimed at safety.

If we have an optimization goal like “Let’s not get kept alive against our will and tortured in the most horrible way for millions of years on end”, then it seems to me that this funding is actually harmful rather than helpful, because it increases the probability that AI dystopia arrives while we are still alive.

Not all proposed solutions to x-risk fit this pattern: If government spends taxes to build survival shelters that will shelter only a chose few who will then go on to perpetuate humanity in case of a cataclysm, most tax payers receive no personal benefit.

Similarly, if government-funded programs solve AI value loading problems and the ultimate values don't reflect my personal self-regarding preferences, I don't benefit from the forced funding and may in fact be harmed by it. This is also true for any scientific research whose effect can be harmful to me personally even if it reduces x-risk overall.

I observe that you are communicating in bad faith and with hostility, so I will use my right to exit for any further communication with you.

What? Why? No sane person would classify "he will murder me if I leave" as "the right to exit isn't blocked". I don't expect much steelmanning from the downvote-bots here, but if you're strawmanning on a rationalist board, good-faith communication becomes disincentivized. It's not like I have skin in the game; all my relationships are nonviolent and I neither give a shit about feminism nor anti-feminism.

Still, if "she's such a nice person but sometimes she explodes" isn't compatible with revealed ... (read more)

I didn't read the whole post, but most of that is just the right to exit being blocked by various mechanisms, including socioeconomic pressure and violence. And the socioeconomic ones aren't even necessarily incompatible with revealed preference; if the alternative is homelessness, this may suck, but the partner still has no obligation to continue the relationship and the socioeconomic advantages are obviously a part of the package.

if we are able to wirehead in an effective manner it might be morally obligatory to force them into wireheading to maximize utility.

Not interested in this kind of "moral obligation". If you want to be a hedonistic utilitarian, use your own capacity and consent-based cooperation for it.

I think it's worth making the distinction between reward hacking, pleasure wireheading, and addiction more clearly. There's some overlap, but these are different concepts with different implications for our utility.

The whole ideological subtext reeks with puritan moralism. You imply that we exist to make humanity's future bigger, rather than to do whatever the hell we actually prefer.

As long as pleasure wireheading is consensual, you longtermists can simply forgo your own pleasure wireheading and instead work very hard on the whole growth an... (read more)

The demand for sexual violence in fiction is easy to explain. It allows us to fantasize about behavior that would be prohibitively disadvantageous in practice, and it allows us to reflect on hypothetical situations that are relevant to our interests, such as how to deal with violent people.

My default model for abusive relationships *where the right to exit is not blocked* is indeed revealed preference. Not necessarily revealed preference for the abuse, but for the total package of goods and bads in the relationship.

The sex and romance market is a market af... (read more)

Indeed, as mentioned, without altruism, voting behaviour is fairly inexplicable.

I vote to reward or penalize politicians based on their previous choices, rather than to create better outcomes. That is, I look back, not forward.

There are some exceptions, e.g. when a candidate before assuming office is sending unusually credible signals, e.g. glorifying torture or some such. Other than that, I mostly ignore promises, and instead implement reciprocity for past decisions.

Edited after more reflection:

Whereas the expected benefit of voting to you alone is the Br

I agree with other commenters that the slavery framing is unhelpful. However, I mostly do agree with Jordan Peterson otherwise.

Human rights set expectations how we treat each other. From my perspective, respect for them is conditional on reciprocity. I will not respect the rights of an individual who doesn't respect mine. Their function is to set standards of behavior that make everybody better off.

A benefit of human rights, rather than mammal rights or just smaller-identity rights is that they benefit everyone who can understand the concept, so they&... (read more)

I have no idea what toonalfrink's goals for the conversation are. But when someone writes something like,

>So you find yourself in this volunteering opportunity with some EA's and they tell you some stuff you can do, and you do it, and you're left in the dark again. Is this going to steer you into safe waters? Should you do more? Impress more? Maybe spend more time on that Master's degree to get grades that set you apart, maybe that'll get you invited with the cool kids?

then the only sensible option from my perspective is to take... (read more)

If the required kind of multiverse exists, this leads to all kinds of contradictions.

For example, in some universes, Personal Identity X may have given consent to digital resurrection, while in others, the same identity may have explicitly forbidden it. In some universes, their relatives and relationships may have positive prefrences regarding X's resurrection, in others, they may have negative preferences.

Given your assumed model of personal identity and the multiverse, you will always find that shared identities have contradicting preferences. They ... (read more)

To be precise, this seems like a cost to Alice of Bob having a wide circle, if Alice and Bob are close. If they aren't, and especially if we bring in a veil of ignorance, then Alice is likely to benefit somewhat from Bob having a wide circle.

Yes, but Alice doesn't benefit from Bob's having a circle so wide it contains nonhuman animals, far future entities or ecosystems/biodiversity for their own sake.

and my reaction is that none of that stops children from dying of malaria, which is really actually a thing I care about and don't wan

But an expanding circle of moral concern increases value differences. If I have to pay for a welfare system, or else pay for a welfare system and also biodiversity maintainance and also animal protection and also development aid and also a Mars mission without a business model and also far-future climate change prevention, I'd rather just pay for the welfare system. Other ideological conflicts would also go away, such as the conflict between preventing animal suffering and maintaining pristine nature, ethical natalism vs. ethical anti-natalism, and so on.

Yes, it certainly cuts both ways. Of course, your country's welfare system is also available to you and your family if you ever need it, and you benefit more directly from social peace and democracy in your country, which is helped by these transfers. It is hard to see how you could have a functioning democracy without poor people voting for some transfers, so unless you think democracy has no useful function for you, that's a cost in your best interest to pay.

The moral circle is not ever expanding, and I consider that a good thing.

A very wide moral circle is actually very costly to a person. Not only can it cause a lot of stress to think of the suffering of beings in the far future or nonhuman animals in farming or in the wild, but it also requires a lot of self-sacrifice to actually live up to this expanded circle.

In addition, it can put you at odds with other well-meaning people who care about the same beings, but in a different way. For example, when I still cared about future generations, I mostly cared ab... (read more)