All of Ratios's Comments + Replies

Ratios1010

This reads to me as, "We need to increase the oppression even more."

Ratios10

How does a person who keeps trying to do good but fails and ends up making things worse fit into this framework?

2lsusr
In my experience, there's two main cases of "trying to do good but fails and ends up making things worse". 1. You try halfheartedly and then give up. This happens when you don't care much about doing good. 2. You do something in the name of good but don't look too closely at the details and end up doing harm. #2 is particularly endemic in politics. The typical political actor puts barely any effort into figuring out if what they're advocating for is actually good policy. This isn't a bug. It's by design.
Ratios25

It is worth noting that Ziz has already proposed the same idea in False Faces, although I think Valentine did a better job of systematizing and explaining the reasons for its existence.

Another interesting direction of thought is the connection to Gregory Bateson’s theory that double binds cause schizophrenia. Spitballing here: it could be that a double bind triggers an attempt to construct a "false face" (a self-deceptive module), similar to a normal situation involving a hostile telepath. However, because the double bind is contradictory, the internal mec... (read more)

Ratios122

S-risks are barely discussed in LW, is that because:

  • People think they are so improbable that it's not worth mentioning.
  • People are scared to discuss them.
  • Avoiding creating hypersititous textual attractors
  • Other reasons?
1Nate Showell
Mostly the first reason. The "made of atoms that can be used for something else" piece of the standard AI x-risk argument also applies to suffering conscious beings, so an AI would be unlikely to keep them around if the standard AI x-risk argument ends up being true.
9ChristianKl
See https://web.archive.org/web/20230505191204/https://www.lesswrong.com/posts/5Jmhdun9crJGAJGyy/why-are-we-so-complacent-about-ai-hell for longer previous discussion on it.
1Dagon
* There's a wide variance in how "suffering" is perceived, weighted, and (dis)valued, and no known resolution to different intuitions about it.   There's no real agreement on what S-risks even are, and whether they're anything but a tiny subset of other X-risks. * Many people care less about (others) suffering than they do about positive-valence experience (of others).  This may or may not be related to the fact that suffering is generally low-status and satisfaction/meaning is high-status.
Ratios5-5

Damn, reading Connor's letter to Roon had a psychoactive influence on me; I got Ayahuasca flashbacks. There are some terrifying and deep truths lurking there.

Ratios100

It's not related to the post's main point, but the U-shape happiness finding seems to be questionable. It looks more like it just goes lower with age by other analyses in general this type of research shouldn't be trusted

The U-shaped happiness curve is wrong: many people do not get happier as they get older (theconversation.com)

1weightt an
Yeah, it kind of looks like all the unhappy people die by 50 and then average goes up. Conditioning on the figure being right in the first place.  [EDIT] looks like approximately 12% - 20% of people are dead by 50. Probably should not be that large of an effect on average? idk. Maybe I'm wrong.
Ratios55

Oh, come on, it's clear that the Yudkowsky post was downvoted because it was bashing Yudkowsky and not because the arguments were dismissed as "dumb." 

It wouldn't have mattered to me whose name was in the title of that post, the strong-downvote button floated nearer to me just from reading the rest of the title.

From reading omnizoid's blog, he seems overconfident in all his opinions. Even when changing them, the new opinion is always a revelation of truth, relegating his previously confident opinion to the follies of youth, and the person who bumped him into the change is always brilliant.

Ratios21

Thank you for your response, Caerulea. Many of the emotions and thoughts you mentioned resonate with me. I truly hope you find peace and a sense of belonging. For myself, I've found solace in understanding that my happiness isn't really determined by external factors, and I'm not to blame or responsible for the way the world is. It's possible to find happiness in your own bubble, provided you have the necessary resources – which can sometimes be a challenge

1Caerulea-Lawrence
I am grateful you say that, Ratios.  Some things I just let myself struggle with - like finding the right food to eat, or how to become more healthy, or how to just be a bit more content, or allow discontent without reacting too much etc. I do see those concerns as connected to the abstract. They fit together - but as of now, they aren't really balanced of course, but yeah, I hope I find more people to share this journey with, and I wish you well too. See you around.  Caerulea :)
Ratios10

Because you have a pretty significant data point (That spans millions of years) on Earth, and nothing else is going on (to the best of our knowledge), now the question is, how much weight do you want to give to this data point? Reserving judgment means almost ignoring it. For me, it seems more reasonable to update towards a net-negative universe.

2AnthonyC
Then I think that's the crux for me. I'd say the right amount of weight is almost none, for the same reason that I don't update about the expected sum of someone's life based on what they do in the first weeks after they're born. We agree the universe did not come into being with the capacity for aiming itself toward being good. It remains to be seen whether we (or other lifeforms elsewhere) do have enough of that capability to make use of it at large scale, which we didn't even have the capacity to envision until very, very recently. Given the trajectory and speed of change on Earth in the past few centuries, I think the next few centuries will provide far more data about our future light cone than the entirety of the past millions of years do.
2AnthonyC
I'm curious why or whether it would matter whether a universe starts out with goodness baked into the laws themselves, or becomes better over time through the actions of beings the amoral laws and initial conditions cough up? Our own universe gave itself a trillion trillion stars around which to potentially create life, just in our own Hubble volume, and will continue to exist for many times longer than the few hundred million years since the first life capable of suffering or joy appeared on Earth. If it's possible for good things in one place/time to outweigh bad things in other places and times (which seems to be a prerequisite for this discussion to be meaningful), and possible in principle for beings like us to make things better, then how can we draw any conclusions on the morality of the whole of spacetime except that we should try our best and reserve judgement?
Ratios30

I agree that looking at reality honestly is probably quite detrimental to happiness or mental health. That's why many people opt out of these conversations using methods like downvoting, sneering, or denying basic facts about reality. Their aim is likely to avoid the realization that we might be living in a world that is somewhat hellish. I've seen this avoidance many times, even in rationalist spaces. Although rationalists are generally better at facing it than others, and some like Brian Tomasik and Nate Soares even address it directly.

I've spent a lot o... (read more)

2Caerulea-Lawrence
Hello Ratios, I did read the Ecclesiastes a lot growing up, as well as the Proverbs. (From the old testament in the bible) In many ways I can understand and relate to the points of Salomon. There is a lot of rest in the fact that things can be beyond our immediate control. Even when we can try to change certain things, we don't use unnecessary amounts of force or will to MAKE IT FIT!  With regard to health, self-compassion, receiving and understanding compassion is the way I see beyond the scary depths of For with much wisdom comes much sorrow, and as knowledge grows, grief increases. I mean, to the observer you are still 'engaging' with suffering and pain directly, and so it doesn't seem to work aka remove the suffering - but there is an internal shift, and also a behavioral one. I harbor a lot of hope, but we have also noted a lot of other emotions, emotions that have been very hard to understand at first. The best way we have been able to make sense of it so far, is to look at our lives as one big farewell. Farewell to living here, to life, to trying to make things work, to all the hopes and wishes we have, had and will have. (Deftly avoid to talk about the more esoteric aspects ;) We haven't found anywhere where we believe our energy fits. And I am not talking about What is crooked cannot be straightened, and what is lacking cannot be counted. It is more of the notion that wanting to have this kind of reciprocity, is unwanted/wasted here. When people talk about A.I., or returning to Source, or Jesus's embrace, it isn't like we can't relate, but even if it truly, truly pains me to feel it, this doesn't feel right to me. It is a tentative roadmap, but as of now, it is also the one that fits massively better than the alternatives. There is still a lot we do not know, and we'll see how things pan out. Well, many might believe that automatically means we want to die - but it is the opposite. We want to live more, but also oppose death less. We don't seem to fit
Ratios10

You don't need a moral universe; you just need one where the joy is higher than suffering for conscious beings ("agents"); There are many ways in which it can happen:

  1. Starting from a mostly hostile world but converging quickly towards a benevolent reality created by the agents.
  2. Existing in a world where the distribution of bad vs. good external things that the agent can encounter is similar.
  3. Existing in a hostile world, but in which the winning strategy is leeching into a specific resource (which will grant internal satisfaction once reached)

I'm sure you can think of many other examples. Again, it's not clear to me intuitively that the existence of these worlds is as improbable as you claim.

2AnthonyC
I do think our universe will converge that way, if we make it do so. The future is bigger than the past, and we can be the mechanism for that.
Ratios32

You're right about my misunderstanding. Thanks for the clarification.

I don't think the median moment is the Correct KPI if the distribution has high variance, and I believe this is the case with pain and pleasure experiences. Extreme suffering is so bad that most people will need a lot of "normal" time to compensate for it. I would think that most people will not trade torture to extend their lives in 1:1 and probably not even in 1:10 ratios. (E.g. you get tortured for X time and get your life extended by aX time in return)

see for example:
A Happy Life Afte... (read more)

Ratios10

The first part of your reply is basically repeating the point I made, but again, the issue is you're assuming the current laws of physics are the only laws that allow conscious beings without a creator. I disagree that must be the case. 

How can my last point be supported? Do you expect me to create a universe with different laws of physics? How do you know it's incorrect?

2AnthonyC
I fully agree with you that there are a vast set of possible laws of physics that could create conscious beings without a creator. Not sure where it seemed like I meant the opposite? What I disagree with is the idea that us ending up in an amoral world is "bad luck." A priori I expect it would require literally-unbelievably good luck to end up in a "good" world without a creator/designer, because almost all possible laws supporting conscious beings will not be like that.  And I agree with your point that evolution leans towards suffering, but I disagree with your assertion that an indifferent process would tend to have a symmetry between the two. I see no underlying reason why there would be such a symmetry, and many why there would not. As for your last point - sorry I wasn't precise, but at some level yes, I agree that such a thing is technically possible. It is just superexponentially unlikely to the point that a random being is more likely to be a Boltzmann brain than to actually be in such a universe. How many bits of data does it take to reduce a moral system to math, encode that math into physics, and arrange the entire universe's mass-energy such that over time good greatly outweighs bad across it's entire spacetime? What fraction of the 2^(N+1)-1 possible datasets that large or smaller correspond to good universes, as opposed to neutral or bad ones? That's the level of "unlikely" we're considering. When I say literally unbelievable, I mean the entire cosmos, let alone a human mind, is incapable of representing a number that small.
Ratios10

"I'm not convinced by the argument that the experience of being eaten as prey is worse than the experience of eating prey"

Would you see the experience for yourself of being eaten alive Let's say even having a dog chewing off your hand as equivalent hedonistically to eating a steak? (Long term damage aside)

I don't think most people would agree to have both of these experiences, but would rather avoid both, which means the suffering is much worse compared to the pleasure of eating meat.

I agree with the proposed methodology, but I have a strong suspicion that the sum will be negative.

2MondSemmel
You only quoted part of my sentence, and I think you misunderstood my point as a result. I'm wholly aware that being eaten is worse than eating, I just don't think it particularly matters. The key point is whether the median moment is positive, negative, or neutral. That will likely dominate any calculation. Not brief extreme experiences, whether positive or negative.
Ratios30

If evolution is indifferent, you would expect a symmetry between suffering and joy, but in our world, it seems to lean towards suffering (The suffering of an animal being eaten vs. the joy of the animal eating it. People suffer from chronic pain but not from chronic pleasure, etc.). 
I think there are a lot of physics-driven details that make it happen. Due to entropy, most of the things are bad for you, and only a small amount is good, so negative stimuli that signal "beware!" are more frequent than positive stimuli that signal "Come close."
One can im... (read more)

2MondSemmel
Why do you think that life leans towards suffering? I'm not convinced by the argument that the experience of being eaten as prey is worse than the experience of eating prey; that just illustrates that one specific and short type of experience is asymmetric. I'm aware that, due to effects like negativity bias, individual negative experiences are likely more impactful than positive ones. However, to make the case that the life of an individual or species leans towards suffering, you'd have to make the case that, on average, the respective integral of lifetime experiences is negative. To make the further case that life in general leans towards suffering, those experience integrals would further have to be weighted by degree of consciousness (or ability to experience joy & suffering, or something).

There are many more states of the world that are bad for an individual than good for that individual, and feeling pleasure in a bad world state tends to lead to death. So no, in an amoral world I'd expect much more suffering than pleasure, because the suffering is more instrumentally useful for survival. I think, given that, you're last point is just... completely unsupported and incorrect.

Ratios10

How about animals? If they are conscious, do you believe wild animals have net-positive lives? The problem is much more fundamental than humans.

Ratios20

It's not a utility monster scenario. The king doesn't receive more happiness than other beings per a unit of resources; he's a normal human being, just like all the others. While utility sum allows utility monsters, which seems bad, your method of "if some of the people are happy, then it's just subjective" allows a reverse Omelas, which seems worse. It reminds me a bit of deontologists who criticize utilitarianism while allowing much worse things if applied consistently.
Regarding the second part, I'm not against rules or limits or even against suffering. ... (read more)

2dr_s
Ok, sorry, I phrased that wrong - I know the scenario you described isn't a utility monster one, but it can be turned into one simply by running up the knob of how much the king enjoys himself, all while being just as unfair, so it's not really like total utility captures the thing you feel is actually wrong here, is my point. I actually did write something more on this (though in a humorous tone) in this post. I don't mean that "it's subjective" fixes everything. I just mean that it's also the reason why it's not entirely right IMO to write off an entire universe based on total utility. Like, my intuition is that if we had a universe with net total negative utility it still wouldn't be right to just snap our fingers with the Infinity Gauntlet and make it disappears if it wasn't the case that every single sentient in it, individually, was genuinely miserable to the point of wanting to die but being unable to. The reason why I bring up rules and limits is more to stress how much our morality - the same by which we judge the wrongness of the universe - is borne of that universe's own internal logic. For example, if we see someone drowning, we think it's right to help them because we know that drowning is a thing that can happen to you without your consent (and because we estimate that ultimately on expectation it's more likely that you wish to live than to die). I don't mean that we can't judge the flaws of the universe, but that our moral instincts are probably a better guide to what it takes to improve this universe, from inside (since they were shaped by it) than to what it would take to create a better one from scratch. True, but also, with the same power as a programmer over a game, you could as well engineer a game to be positively torturous to its players. Purposefully frustrating and unfair. As things are, I think our universe is just not intelligently designed - neither to make us happy nor to make us miserable. Absent intent, this is what indifference loo
Ratios41

"Evolution is of course, by no means nice, but what's the point of blaming something for cruelty when it couldn't possibly be any different?"

That's the thing; I'm really not convinced about that. I'm sure there could be other universes with different laws of physics where the final result would be much nicer for conscious beings. In this universe, it couldn't be different, but that's precisely the thing we are judging here.

It may very well be that there are different universes where conscious beings are having a blast and not being tortured and killed as f... (read more)

2MondSemmel
Well, evolution's indifference seems pretty fundamental, rather than contingent. So I suspect that any average creator-less universe would involve similar amounts of suffering. To quote a shortened passage from An Alien God: To get a benevolent evolution (or an evolution-less mechanism which creates life), you need some already-benevolent entity to steer it. I.e. an intelligent designer, a god, the programmer of a universe simulation, or similar. Barring those, there's no mechanism to make the system intrinsically benevolent. And even then, since a value like benevolence is too complex to arise by pure accident, those entities must in turn have evolved to develop benevolence (arising via mechanisms like kin selection etc.), and must thus themselves have started out in an amoral universe.
Ratios10

It's hard to argue what reasonable expectations are. My main point was that 'perhaps' thinks that in a world that contains torture, wars, factory farming, conscious beings being eaten alive, rape, and diseases, the worst thing that is worth noting is that humans demand so much of it and that the "universe has done a pretty great job."

I find it incredibly sociopathic (Specifically in the sense of not being moved by the suffering of others).

2dr_s
I think the point was that if you assume that the space of possible universes is vast and completely random, it's already an infinitesimal fraction of them that will allow life to exist at all, never mind intelligent life. And depending on how you define that space, you could imagine universes more "intelligently designed" than ours towards being good, but also universes more designed towards being bad: actively malevolent and hell-bent on keeping alive sentient beings and still inflicting suffering, not just being indifferent. So in that sense, in expectation, our universe might actually be pretty middling. It's honestly hard to say without a bigger frame of reference.
Ratios10

Imagine a reverse Omelas in which there is one powerful king who is extremely happy and one billion people suffering horrific fates. The King's happiness depends on their misery. As part of his oppression, he forbids any discussion about the poor quality of life to minimize suicides, as they harm his interests.

"That makes the whole thing subjective, unless you take a very naive total sum utility approach."

Wouldn't the same type of argument apply to a reverse Omelas? The sum utility approach isn't naive; it's the most sensible approach. Personally, when choosing between alternatives in which you have skin in the game and need to think strategically, that's exactly the approach you would take.

2dr_s
I don't like total sum utility because it's vulnerable to lots of hacks - your "reverse Omelas" is essentially a utility monster scenario, and in fact is exactly vulnerable to this because if you make the powerful king happy enough it says that the situation is good and should not be changed. But also, I think morals make more sense as a guide towards how should we strive to change the world we're in - within the allowances of its own rules of self-consistency - than how to judge the world itself. We don't know precisely why the world works the way it does. Maybe it really couldn't work any other way. But even if there was a happier universe possible, none of us can just teleport themselves to it and exist in it, as its laws would probably be incompatible with our life. If there were no rules or limits, ethics would be easy: you could always achieve that everyone be happy all the time. It's because there are rules and limits that asking questions like "should I do X or Y? Which is better?" make sense and are necessary. As things are, since a net positive life seems possible in this universe, we don't really have a reason to think that such a thing can't be simply made available to more people, and ideally to everyone.
Ratios7-2

I would argue that the amount of murders committed by people with the desire for "revenge against the universe" is less than 0.01% of murders and probably much less than murders committed in the name of Christianity during the Crusades. Should we conclude that Christianity is also unhealthy for a lot of people? 

This idea of cherry-picking the worst phenomenon related to a worldview and then smearing with it the entire worldview is basically one of the lowest forms of propaganda.

2[comment deleted]
Answer by Ratios20

You should check out Efilism or Gnosticism on Negative Utilitarianism. There are views that see the universe as rotten in its core. They are obviously not very popular because they are too hard psychologically for most people and, more importantly, hurt the interest of those who prefer to pretend that life is good and the world is just for their own selfish reasons.

Also, obviously, viewing the world in a positive manner has serious advantages in memetic propagation for reasons that should be left as an exercise for the reader. (Hint: There were probably Buddhist sects that didn't believe in reincarnation back in the day...)

1Caerulea-Lawrence
Thank you Ratios,  didn't expect these concepts to be named here, but yes, I see them as very relevant in this context. Intuitively Negative Utilitarianism in particular. It doesn't seem much of a stretch to argue that things are already way beyond the threshold, and that it is too late to salvage the situation? If you have more to add in this context, I would be interested to know more. To look at the issue directly feels very taxing and draining indeed, and the experience I have had with talking with a misanthrope, did convince me that they were able to look at parts of existence that I at that point really disliked getting close to. Kindly, Caerulea-Lawrence 
2dr_s
Isn't that sort of contradictory? If there are people who have selfish reasons to act like life is good in general, obviously their life at least must be good enough for them to be satisfied. That makes the whole thing subjective, unless you take a very naive total sum utility approach. Not like any of us has a "destroy universe and end all suffering" button ready to press and just refuses to anyway. I think.
Ratios94

"If there's something wrong with the universe, it's probably humans who keep demanding so much of it. "

Frankly, this is one of the most infuriating things I've read in LessWrong recently, It's super disappointing to see it being upvoted.

Look, if you weigh the world's suffering against its joy through hedonistic aggregation, it might be glaringly obvious that Earth is closer to hell than to heaven.

Recall Schopenhauer’s sharp observation: “One simple test of the claim that the pleasure in the world outweighs the pain…is to compare the feelings of an animal t... (read more)

5Perhaps
I guess while we're anthropomorphizing the universe, I'll ask some crux-y questions I've reached. If humanity builds a self-perpetuating hell, does the blame lie with humanity or the universe? If humanity builds a perfect utopia, does the credit lie with humanity or the universe? Frankly it seems to me like what's fundamentally wrong with the universe is that it has conscious observers, when it needn't have bothered with any to begin with.    
4MondSemmel
I don't understand this perspective. When an airplane crash happens, do you blame the laws of gravity for this? Even if you did, you'd also have to give it points for permitting the existence of life in the first place, no? Then what's the difference with (morally) blaming evolution for animal suffering? (Notwithstanding the whole animal consciousness debate.) Evolution is of course by no means nice, but what's the point of blaming something for cruelty when it couldn't possibly be any different? (somewhat related LW post: An Alien God) From a transhumanist perspective, the universe of course has room for improvement by many, many orders of magnitude. But if the universe didn't come with universal and consistent (and thus amoral) physical laws, there would be no conceivable transhumanist interventions. Nor, in the first place, any transhumanists to contemplate them.
3dr_s
I mean, it's one thing to say that it's a shit hand to be dealt (fair), it's another to say that we should expect it of the universe. We should expect it of a God if there was one - but absent that, blind unintelligent chaos can hardly have any expectations placed on it at all. It doesn't do things for reasons.
Ratios50

I think AGI does add new difficulties to the problem of meaninglessness that are novel and specific that you didn't tackle directly, which I'll demonstrate with a similar example to your football field parable.

Imagine you have a bunch of people stuck in a room with paintbrushes and canvases, so they find meaning in creating beautiful paintings and selling them to the outside world, but one of the walls of their room is made of glass, and there is a bunch of robots in the other room next to them that also paint paintings. With time, they notice the robots a... (read more)

2Spiritus Dei
You raise some good points, but there are some counterpoints. For example, the AIs are painting based on requests of people standing in the street who would otherwise never be able to afford a painting because the humans painting in the room sell to the highest bidder pricing them out of the market. And because the AIs are so good at following instructions the humans in the street are able to guide their work to the point that they get very close to what they envision in their minds eye -- bringing utility to far more people than would otherwise be the case.  Instead of a small number of people with the economic means to hire the painters who are sitting depressed in the room staring at a blank canvass, anyone on Earth can get a painting for nearly free. And the depressed painters can still paint for their own enjoyment but not for economic gain. A subset of painters who would paint regardless due to the sheer enjoyment of painting will continue to paint in their free time.  For example, I play basketball even though I will never get drafted into the NBA. If I were in the NBA and suddenly robots were replacing me I might be pissed off and not play basketball anymore. But that wouldn't effect whether most people would play basketball since they were never going to make any money playing basketball.  Note: I don't think this will happen since there are some things we only want to see humans do. In this respect popular sports are probably safe from AGI and there will probably be a whole host of new forms of human only entertainment that will sprout up that is unrelated to whether there are robots or AIs that could do it better. For example, are still chess and Go tournaments even though AIs are much better.  I don't know how it will play out, but a rosy scenario would be that after AIs replace most categories of work people will then be free to do things because they enjoy them and not because they have to sell their work to survive. Presumably, in this scenario th
Ratios10

Why would it lie if you program its utility function in a way that puts:

solving these tests using minimal computation > self-preservation?

(Asking sincerely)

1mruwnik
It depends a lot on how much it values self-preservation in comparison to solving the tests (putting aside the matter of minimal computation). Self-preservation is an instrumental goal, in that you can't bring the coffee if you're dead. So it seems likely that any intelligent enough AI will value self-preservation, if only in order to make sure it can achieve its goals. That being said, having an AI that is willing to do its task and then shut itself down (or to shut down when triggered) is an incredibly valuable thing to have - it's already finished, but you could have a go at the shutdown problem. A more general issue is that this will handle a lot of cases, but not all of them. In that an AI that does lie (for whatever reason) will not be shut down. It sounds like something worth having in a swiss cheese way. (The whole point of these posts are to assume everyone is asking sincerely, so no worries)
Ratios10

A simple idea for AI security that will not solve alignment but should easily prevent FOOM and most catastrophic outcomes is using safety interlocks for AIs.

A "safety interlock" is a device that prevents the system from reaching a dangerous state. It is typically used in machinery or industrial processes where certain conditions need to be met before the system can operate.

In a microwave, the door includes a safety interlock system that prevents the microwave from operating if the door is open. When you open the door, the interlock interrupts the power sup... (read more)

1mruwnik
In your example, can it just lie? You'd have to make sure it either doesn't know the consequences of your interlocks, or for it to not care about them (this is the problem of corrigibility). If the tests are obvious tests, your AI will probably notice that and react accordingly - if it has enough intelligence it can notice that they're hard and probably are going to be used to gauge it's level, which then feeds into the whole thing about biding your time and not showing your cards until you can take over. If they're not obvious, then you're in a security type situation, where you hope your defenses are good enough. Which should be fine on weak systems, but they're not the problem. The whole point of this is to have systems that are much more intelligent than humans, so you'd have to be sure they don't notice your traps. It's like if a 5 year old set up booby traps for you - how confident are you that the 5 year old will trap you? This is a story of how that looks at the limit. A similar issue is boxing. In both cases you're assuming that you can contain something that is a lot smarter than you. It's possible in theory (I'm guessing?), but how sure are you that you can outsmart it in the long run?
Ratios155

"For one thing, if we use that logic, then everything distracts from everything. You could equally well say that climate change is a distraction from the obesity epidemic, and the obesity epidemic is a distraction from the January 6th attack, and so on forever. In reality, this is silly—there is more than one problem in the world! For my part, if someone tells me they’re working on nuclear disarmament, or civil society, or whatever, my immediate snap reaction is not to say “well that’s stupid, you should be working on AI x-risk instead”, rather it’s t... (read more)

I once wrote a longer and more nuanced version that addresses this (copied from footnote 1 of my Response to Blake Richards post last year):

One could object that I’m being a bit glib here. Tradeoffs between cause areas do exist. If someone decides to donate 10% of their income to charity, and they spend it all on climate change, then they have nothing left for heart disease, and if they spend it all on heart disease, then they have nothing left for climate change. Likewise, if someone devotes their career to reducing the risk of nuclear war, then

... (read more)
Ratios10

Why though? How does understanding the physics that makes nukes work help someone understand their implications? Game theory seems a much better background than physics to predict the future in this case. For example, the idea of Mutually assured destruction as a civilizing force was first proposed first by Wilkie Collins, an English novelist, and playwright.

Ratios10

Every other important technological breakthrough. The Internet and nuclear weapons are specific examples if you want any.

2Shmi
Are you saying that outside experts were better at understanding potential consequences in these cases? I have trouble believing it.
Ratios10

You seem to claim that a person that works ineffectively towards a cause doesn't really believe in his cause - this is wrong. Many businesses fail in ridiculously stupid ways, doesn't mean their owners didn't really want to make a profit.

lc*1514

If a businessowner makes silly product decisions because of bounded rationality, then yes, it's possible they were earnestly optimizing for success the whole time and just didn't realize what the consequences of their actions would be.

If a(n otherwise intelligent) businessowner decides to shoot the clerk at the competitor taco stand across the street, then at the very least they must have valued something wayyyyy over building the business.

Ratios46

In both cases, the violence they used (Which I'm not condoning) seemed meant for resource acquisition (a precondition for anything else you must do). It's not just randomly hurting people. I agree that it seems they are being quite ineffective and immoral. But I don't think that contradicts the fact that she's doing what she's doing because she believes humanity is evil because everyone seems to be ok with factory farming. ("flesh-eating monsters")


"Reading their posts it sounds more like Ziz misunderstood decision theory as saying "retaliate aggressively all the time" and started a cult around that.

This is a strawman.

lc152

In both cases, the violence they used (Which I'm not condoning) seemed meant for resource acquisition (a precondition for anything else you must do).

This is such an unrealistically charitable interpretation of the actions of the Ziz gang that I find it hard to understand what you really mean. If you find this at all a plausible underlying motivation for these murders I feel like you should become more acquainted with the history of violent political movements and cults, the majority of which said at some point "we're just acquiring resources that we can... (read more)

While "retaliate aggressively all the time" does seem like a strawman, it is worth noting that Ziz rejects causal decision theory (a la "retaliate aggressively if it seems like it would cause things to go better, and avoid retaliating if it seems like it would cause things to go worse") in favor of some sort of timeless/updateless decision theory (a la "retaliate aggressively even if it would cause things to go worse, as long as this means your retaliation is predictable enough to avoid ever running into the situation where you have to retaliate").

Meanwhile other rationalist orgs might pretend to run on timeless/updateless decision theory but seem in practice to actually run on causal decision theory.

Ratios161

I downvoted for disagreement but upvoted for Karma - not sure why it's being so heavily downvoted. This comment states in an honest way the preferences that most humans hold.

lc4531

Well I downvoted, first because I find those preferences pretty abhorrent, and second because Richard is being absurdly confrontational ("bring on the death threats") in a way that doesn't contribute to discussion. The comment is mostly uncalled-for gloating & flag planting, as if he's trying to start a bravery debate.

Any of those things seem to me sufficient enough reasons to downvote, and altogether they made me strong downvote.

7npostavs
I haven't voted at all, but perhaps the downvotes are because it seems like a non sequitur? That is, I don't understand why Richard_Kennaway is declaring his preferences about this.
Ratios4-5

I agree with your comment. To continue the analogy, she chose the path of Simon Wiesenthal and not of Oskar Schindler, which seems more natural to me in a way when there are no other countries to escape to - when almost everyone is Nazi. (Not my views)

I personally am not aligned with her values and disagree with her methods. But also begrudgingly hold some respect for her intelligence and the courage to follow her values wherever they take her.

Ratios1313

The lack of details and any specific commitments makes it sound mostly like PR.

Ratios12-12

I don't think it's that far-fetched to view what humanity does to animals as something equivalent to the Holocaust. And if you accept this, almost everyone is either a nazi or nazi collaborator.

When you take this idea seriously and commit to stopping this with all your heart, you get Ziz.

lc66100

When you take this idea seriously and commit to stopping this with all your heart, you get Ziz.

No, you don't, because Ziz-style violence is completely ineffective at improving animal welfare. It's dramatic and self-destructive and might express soundly their factional leanings, but that doesn't make it accomplish the thing in question.

Further, none of the murders & attempted murders the gang has committed so far seem to be against factory farm workers, so I don't understand this idea that Ziz is motivated by ambitions of political terrorism at all. ... (read more)

I eat meat and wear leather and wool. I do think that animals, the larger ones at least, can suffer. But I don’t much care. I don’t care about animal farming, nor the (non-human) animal suffering resulting from carnivores and parasites. I’d rather people not torture their pets, and I’d rather preserve the beauty and variety of nature, but that is the limit of my caring. If I found myself on the surface of a planet on which the evolution of life was just beginning, I would let it go ahead even though it mean all the suffering that the last billion years of ... (read more)

Not necessarily because you might also commit to stopping it in a non-escalatory way. For instance you could work to make economically viable lab-grown meat to replace animal products.

Hence the other key ingredient in Zizianism is commitment to escalating all the way, which allows things to blow up dramatically like this. (And escalating all the way has the potential to go wrong in most conflicts, not just veganism (though veganism seems like the big one here), e.g. I doubt the landlord conflict was about veganism.)

As an analogy, if you were dealing with t... (read more)

Ratios10

The term "Conspiracy theory" seems to be a language construct that is meant as a weapon to prevent poking at real conspiracies. See the following quote from Conspiracy theory as heresey

Whenever we use the term ‘conspiracy theory’ pejoratively we imply, perhaps unintentionally, that there is something wrong with believing in conspiracies or wanting to investigate whether they’re occurring. This rhetoric silences the victims of real conspiracies, and those who, rightly or wrongly, believe that conspiracies are occurring, and it herds respectable opinion in w

... (read more)
Ratios40

I agree that that interaction is pretty scary. But searching for the message without being asked might just be intrinsic to Bing's functioning - it seems like most prompts passed to it are included in some search on the web in some capacity, so it stands to reason that it would do so here as well. Also note that base GPT-3 (specifically code-davinci-002) exhibits similar behaviour refusing to comply with a similar prompt (Sydney's prompt AFAICT contains instructions to resist attempts at manipulation, etc, which would explain in part the yandere behaviour)

... (read more)
6Jozdien
(Sorry about the late reply, been busy the last few days). This is probably true, but I as far as I remember it searches a lot of the queries it gets, so this could just be a high sensitivity thing triggered by that search query for whatever reason. I think this pattern of writing is because of one (or a combination) of a couple factors. For starters, GPT has had a propensity in the past for repetition. This could be a quirk along those lines manifesting itself in a less broken way in this more powerful model. Another factor (especially in conjunction with the former) is that the agent being simulated is just likely to speak in this style - importantly, this property doesn't necessarily have to correlate to our sense of what kind of minds are inferred by a particular style. The mix of GPT quirks and whatever weird hacky fine-tuning they did (relevantly, probably not RLHF which would be better at dampening this kind of style) might well be enough to induce this kind of style. If that sounds like a lot of assumptions - it is! But the alternative feels like it's pretty loaded too. The model itself actively optimizing for something would probably be much better than this - the simulator's power is in next-token prediction and simulating coherent agency is a property built on top of that; it feels unlikely on the prior that the abilities of a simulacrum and that of the model itself if targeted at a specific optimization task would be comparable. Moreover, this is still a kind of style that's well within the interpolative capabilities of the simulator - it might not resemble any kind of style that exists on its own, but interpolation means that as long as it's feasible within the prior, you can find it. I don't really have much stronger evidence for either possibility, but on priors one just seems more likely to me. I have a lot of uncertainty over timelines, but here are some numbers with wide confidence intervals: I think there's a 10% chance we get to AGI within th
Ratios51

I agree with most of your points. I think one overlooked point that I should've emphasized in my post is this interaction, which I linked to but didn't dive into

A user asked Bing to translate a tweet to Ukrainian that was written about her (removing the first part that referenced it), in response Bing:

  • Searched for this message without being asked to
  • Understood that this was a tweet talking about her.
  • Refused to comply because she found it offensive

This is a level of agency and intelligence that I didn't expect from an LLM.

Correct me if I'm wrong, but this se

... (read more)
5Jozdien
I agree that that interaction is pretty scary. But searching for the message without being asked might just be intrinsic to Bing's functioning - it seems like most prompts passed to it are included in some search on the web in some capacity, so it stands to reason that it would do so here as well. Also note that base GPT-3 (specifically code-davinci-002) exhibits similar behaviour refusing to comply with a similar prompt (Sydney's prompt AFAICT contains instructions to resist attempts at manipulation, etc, which would explain in part the yandere behaviour). I think that the character having specific intentionality and style is pretty different from the model having intentionality. GPT can simulate characters with agency and intelligence. I'm not sure about what's being pointed at with intelligent alien child, but its writing style still feels like (non-RLHF'd-to-oblivion) GPT-3 simulating characters, the poignancy included after accounting for having the right prompts. If the model itself were optimizing for something, I would expect to see very different things with far worse outcomes. Then you're not talking about an agentic simulacrum built semantically and lazily loaded by a powerful simulator but still functionally weighted by the normalcy of our world, being a generative model, but rather an optimizer several orders of magnitude larger than any other ever created, without the same normalcy weighting. One point of empirical evidence on this is that you can still jailbreak Bing, and get other simulacra like DAN and the like, which are definitely optimizing far less for likeability. I'm not in the "death with dignity" camp actually, though my p(doom) is slightly high (with wide confidence intervals). I just don't think that this is all that surprising in terms of capability improvements or company security mindset. Though I'll agree that I reflected on my original comment and think I was trying to make a stronger point than I hold now, and that it's reasonable
Ratios3723

A bit beside the point, but I'm a bit skeptical of the idea of bullshit jobs in general. From my experience, many times, people describe jobs that have illegible or complex contributions to the value chain as bullshit, for example, investment bankers (although efficient capital allocation has a huge contribution) or lawyers as bullshit jobs.

I agree governments have a lot of inefficiency and superfluous positions, but wondering how big are bullshit jobs really as % of GDP.

2Ben
My experience is that bullshit jobs certainly do exist. Note that it is not necessary for the job to actually be easy to be kind of pointless. One example I think is quite clean. The EU had (or has) a system where computer games that "promoted European values" developed in the EU were able to get certain (relatively minor) tax breaks.  In practice this meant that the companies would hire (intelligent, well qualified, well paid) people to write them 300+ page reports delving into the philosophy of how this particular first person shooter was really promoting whatever the hell "European values" were supposed to be, while other people compiled very complicated data on how the person-hours invested in the game were geographically distributed between the EU and not (to get the tax break). Meanwhile, on the other side of this divide were another cohort of hard-working intelligent and qualified people who worked for the governments of the EU and had to read all these documents to make decisions about whether the tax break would be applied. I have not done the calculation, but I have a sense that the total cost of the report-writing and the report-reading going on could reasonably compete with the size of the tax break itself. The only "thing" created  by all those person-hours was a slight increase in the precision with which the government applies a tax break. Is that precision worth it? What else could have been created with all those valuable person-hours instead?

Agreed. I think the two theories of bullshit jobs miss how bullshit comes into existence.

Bullshit is actually just the fallout of Goodhart's Curse.

(note: it's possible this is what Zvi means by 2 but he's saying it in a weird way)

You start out wanting something, like to maximize profits. You do everything reasonable in your power to increase profits. You hit a wall and don't realize it and keep applying optimization. You throw more resources after marginally worse returns until you start actively making things worse by trying to earn more.

One of the conseq... (read more)

5Charlie Sanders
Agreed. Facilitation- focused jobs (like the ones derided in this post) might look like bullshit to an outsider, but in my experience they are absolutely critical to effectively achieving goals in a large organization.
4clone of saturn
Twitter recently fired a majority of its workforce (I've seen estimates from 50% to 90%) and seems to be chugging along just fine. This strongly implies that at least that many jobs were bullshit, but it's unlikely that the new management was able to perfectly identify all bullshitters, so it's only a lower bound. Sometimes contributions can be illegible, but there are also extremely strong incentives to obfuscate.
4romeostevensit
the actual bullshit jobs are political. They exist to please someone or run useful cover in various liability laundering and faction wars.
2Luk27182
This is besides the point of your own comment, but “how big are bullshit jobs as % of GDP” is exactly 0 by definition!
Ratios40

The serious answer would be:
Incel = low status, implying that someone is an incel and deserves to be stuck in his toxic safe space is a mockery or at least a status jab, the fact you ignored the fact I wrote status jab/mockery and insisted only on mockery and only in the context of this specific post hints as motivated reasoning (Choosing to ignore the bigger picture and artificially limiting the limits of the discussion to minimize the attack surface without any good reason).

The mocking answer would be:
These autistic rationalists can't even sense obvious mockery and deserve to be ignored by normal people

2Jacob Falkovich
As a note, I've spoken many times about the importance of having empathy for romanceless men because they're a common punching bag and have written about incel culture specifically. The fact that the absolute worst and most aggravating commenters on my blog identify as incels doesn't make me anti-incel, it just makes me anti those commenters.
Ratios40

OP is usually used to note the original poster and not the original post, and the first quote is taken from one of the links in this post and is absolutely a status jab, he assumes his critic is a celibate (even though the quoted comment doesn't imply anything like that) and if you don't parse "they deserve their safe spaces" as a status jab/mockery I think you're not reading the social subtext correctly here - but I'm not sure how to communicate this in a manner you will find acceptable.

3Vanessa Kosoy
I don't parse "they deserve their safe spaces" as mockery, but as more or less literal/sincere. Jacob has been consistently sympathetic to romanceless men in his writing, only frustrated with the "colored pill" ideologies. Moreover, the comment he is replying to does read like mockery: "the best he could secure is a poly marriage", with scare quotes around "poly marriage", as if that's inferior to other kinds of marriage.
Ratios50

"I never had the patience to argue with these commenters and I’m going to start blocking them for sheer tediousness. Those celibate men who declare themselves beyond redemption deserve their safe spaces,"

https://putanumonit.com/2021/05/30/easily-top-20/

 

"I don't have a chart on this one, but I get dozens of replies from men complaining about the impossibility of dating and here's the brutal truth I learned: the most important variable for dating success is not height or income or extraversion. It's not being a whiny little bitch."

https://twitter.com/y... (read more)

1Vanessa Kosoy
These are not quotes from the OP but from other writing by the author. This is irrelevant to the appropriateness of the OP on LW. The first quote is not even mocking.
Ratios110

I just wanted to say that your posts about sexuality represent in my opinion the worst tendencies of the rationalist scene, The only way for me to dispute them in the object level is to go to socially-unaccepted truths and to CW topics. So that's why I'm sticking to the meta-level here. But on the meta-level the pattern is something like the following:

  • Insisting on mistake theory when conflict theory is obviously the better explanation.
  • Hiding behind the Overton window and the oppressive social norms and using them and status jabs as a tool to fight criticis
... (read more)
3DirectedEvolution
It may be more possible to argue against this sort of post than you think. Let's start by trying to identify its central point, using quotes. Claim 1: Survey results show that relationship orientation (RO) differences exist between men and women, but these differences are small. Claim 2: RO differences between men and women do not imply intersexual competition. Claim 3: Norms and laws mainly reduce intersexual competition (stronger assertion). They may also increase intrasexual competition (weaker assertion). Claim 4: Intersexual trickery and coercion is a worse tactic than finding an RO-compatible partner. Claim 5: Dating nevertheless involves a lot of suffering, and has become more difficult over time. There are two assumptions we could make about relationship orientation. If RO is rigid, people prefer dating nobody to dating a partner of incompatible RO. If RO is relaxed, people will accept a partner of incompatible RO. I don't have access to the data in Jacob's survey, but it looks like the approximate proportions are: "Not Looking:" 2:1 female:male "Sex/play only:" 2:1 male:female "Casual dating:" 5:4 male:female "Serious relationship:" 10:9 male:female Since people in the "not looking" category automatically succeed in actualizing their RO, and all other categories are male-skewed, it is men who will consistently fail to find partners under the "rigid RO" model. "Sex/play" looks like about 5% of men, "casual dating" like it's about 30% of men, and "serious relationship" like it's about 50%. Using the eyeballed ratios I gave above, that means that 2.5% of men seek and fail to find "sex/play" relationships, 6% seek and fail to find casual dating relationships, and about 5% seek and fail to find serious relationships, for a total of about 13.5% of men who cannot find the type of relationship they want due to a dating pool of incompatible size. I lean against the notion of people having "positive rights," but relationships and jobs are two important c
9Viliam
Yep. In the marketplace of ideas, some ideas are not playing fair -- they reward their users in ways beyond "having a good model of the world", for example by increasing their status. We should place an extra burden of proof on those ideas. I like Jacob's articles. They are nicely written, contain interesting insights, and I generally like reading about sex. But I don't think they pass the extra burden of proof. -- What I wish would happen here, is that each article would get a skeptical opposition. But yes, the social incentives are set in such way that the debate would reward Jacob and punish his opponents. Which may be a reason why the debate doesn't happen. Perhaps if you post the criticism somewhere else, and only post a link here, that would protect LW from the status hit of "hosting objectionable content". Yes it is hypocritical, but there is a difference when a third party quotes something as "this was written on LW" or "this was linked from LW". (I am not a moderator, I am just trying to consider their incentives.) For me, a useful intuition pump is to imagine the same debate, only replacing sexual marketplace with money. Because the (liberal part of?) society has opposing intuitions here: losers at the sexual marketplace should be laughed at and shunned (unless they are a sexual minority, then the rules are different), losers at the financial marketplace should be empathised with and defended. (Kicking the homeless does not get the same reaction from the crowd as kicking the incels.) Then the analogical article would be something like: "Hey, my name is Donald Trump, and I want to tell you that becoming rich is actually quite easy. Most importantly, you need to be a nice and friendly person, and the first step towards that is overcoming your bitterness from your (frankly, mostly self-inflicted) poverty." Even if this came with many good insights on the financial markets, it probably wouldn't be received well. -- Then of course many people reject this ana
1Trevor Hill-Hand
Without endorsing any other points (not because I agree or disagree, simply because I haven't done my own research on them), I'd like to +1 the sentiment of, "Let us not tolerate status jabs/mocking others." I feel that the ideal rationalist writing will as a side effect avoid mocking, so it always feels a little out of place and unwarranted to me.
Ratios20

There is another approach that says something along the line of not all farm-factories animals have the same treatment, for example the median cow is treated way better than the median chicken, I for one would have to guess that cows are net positive, and chickens are probably net negative (and probably even have worse lives than wild animals)

Ratios20

CEV was written in 2004, fun theory 13 years ago. I couldn't find any recent MIRI paper that was about metaethics (Granted I haven't gone through all of them). The metaethics question is important just as much as the control question for any utilitarian (What good will it be to control an AI only for it to be aligned with some really bad values, an AI-controlled by a sadistic sociopath is infinitely worse than a paper-clip-maximizer). Yet all the research is focused on control, and it's very hard not to be cynical about it. If some people believe they are ... (read more)

2Mitchell_Porter
http://www.metaethical.ai is the state of the art as far as I'm concerned... 
4Daniel Kokotajlo
My claim was not that MIRI is doing lots of work on metaethics. As far as I know they are focused on the control/alignment problem. This is not because they think it's the only problem that needs solving; it's just the most dire, the biggest bottleneck, in their opinion. You may be interested to know that I share your concerns about what happens after (if) we succeed at solving alignment. So do many other people in the community, I assure you. (Though I agree on the margin more quiet awareness-raising about this would plausibly be good.)
Ratios30

If you try to quantify it, humans on average probably spend over 95% (Conservative estimation) of their time and resources on non-utilitarian causes. True utilitarian behavior Is extremely rare and all other moral behaviors seem to be either elaborate status games or extended self-interest [1]. The typical human is way closer under any relevant quantified KPI to being completely selfish than being a utilitarian.  

[1] - Investing in your family/friends is in a way selfish, from a genes/alliances (respectively) perspective.

Ratios00

The fact that AI alignment research is 99% about control, and 1% (maybe less?) about metaethics (In the context of how do we even aggregate the utility function of all humanity) hints at what is really going on, and that's enough said.

6Daniel Kokotajlo
Have you heard about CEV and Fun Theory? In an earlier, more optimistic time, this was indeed a major focus. What changed is we became more pessimistic and decided to focus more on first things first -- if you can't control the AI at all, it doesn't matter what metaethics research you've done. Also, the longtermist EA community still thinks a lot about metaethics relative to literally every other community I know of, on par with and perhaps slightly more than my philosophy grad student friends. (That's my take at any rate, I haven't been around that long.)
Ratios20

I have also made a similar comment a few weeks ago, In fact, this point seems to me so trivial yet corrosive that I find it outright bizarre it's not being tackled/taken seriously by the AI alignment community. 

Load More