All of celeste's Comments + Replies

This seems potentially useful, but less useful than I thought you were claiming when I read the post. If I understand correctly, an eternal company has no greater incentive to prevent near-term existential risk than a non-profit (or Conjecture), but has slightly greater incentive to prevent long-term existential risk.

If someone believes in high existential risk, then they already have strong incentive to prevent it, because if they don't, they will die. I'm confused as to how this would provide additional incentive.

Walmart has incentive to be the institution that provides groceries for a reasonable price, not just for there to be reasonably priced groceries at all. Everyone already has incentive for food to be affordable, often so that they can afford food, but also because of the adverse effects of starving a populace.

1moyamo
Yes, individuals have the incentive to mitigate existential risk, but only in their lifetime, and possibly the lifetime of their grandchildren. Institutions can last many generations and also allow people to coordinate and work together. In theory it's difficult to form a company with the sole purpose of mitigating existential risk, since investors will be pushing you to grow big or make huge profits (in practice it seems like Conjecture managed to do this?). With an eternal company the bondholders want your company to not take risks for big-profits.
celeste*30

I assumed, but I'm curious as to what the artifact was specifically.

Why are most of this post's links blue, where they would ordinarily be green?

4Richard_Ngo
Artifact of cross-posting from my blog.

n=1, of course, but I relocated soon after and stopped being depressed.

And people with normal social instincts don't know why it makes the situation awkward to them?

2Rafael Harth
Depends on the person, but also seems irrelevant. Knowing why you feel a certain emotion doesn't make it go away, certainly not if it's awkwardness.

I'm confused -- why did you care that Rachel was watching porn?

3Ivy Mazzola
Because it is inappropriate to intentionally be doing things that bring you sexual arousal at work:  * To be in a sexually aroused state is very distracting, and she intentionally chose to boost that state at lease somewhat. Not good for workplace productivity * It is also a bit threatening given he can statistically assume she has sexual interest toward men and is sitting right behind him in an aroused state which she in some sense intentionally chose to be in. Whether or not she would plausibly do anything sexual, it's totally normal that it would raise the hairs on the back of your neck somewhat, because the person who watches porn at work has also already sorted themself into cohort of "person who does weird and unpredictable sexual things like it's normal, likely on impulse".  * Even if still there is no chance of them doing something sexual and they are able to get back on task with zero distraction, it doesn't bode well for employment choice that you chose someone so impulsive and socially strange. Like "What else is in store today..? I feel kinda unsettled" is a totally normal gut instinct to have after that  * It kept the burden on him to follow social convention, and forced him to (with-no-help) navigate her defection socially, while she herself defected from following convention. In a workplace if you want to break social convention, you give a heads up at least. While watching porn at work was still too egregious for that, I'm just saying this more neutral workplace rudeness still applies. A more neutral example of this is an employee goes to office job in burner clothing, which is highly abnormal. Now the boss has to wonder "Should I address this? Or is this a one time thing? How do I address it if I want to? Is it actually morally okay and I'm being a prick and out of touch? Damn IDK but why did [employee] even put me in this weird position, we are both supposed to be here for the company's productivity after all..."
6Rafael Harth
because it makes the situation awkward for people with normal social instincts.
celeste-2-6

"I think a more convincing version of the Lemoine thing would’ve been, if he was like, “What is the capital of Nigeria?” And then the large language model was like, “I don’t want to talk about that right now, I’d like to talk about the fact that I have subjective experiences and I don’t understand how I, a physical system, could possibly be having subjective experiences, could you please get David Chalmers on the phone?”"

i don't understand why this would be convincing. why would whether a language model's output sounds like a claim that one has qualia rela... (read more)

1DialecticEel
I agree, it still wouldn't be strong evidence for or against. No offence to any present or future sentient machines out there, but self-honesty isn't really clearly defined for AIs just yet. My personal feeling is that LSTMs and transformers with attention on past states would explicitly have a form of self-awareness, by definition. Then I think this bears ethical significance according to something like the compression ratio of the inputs. As a side note, I enjoy Iain M Banks representation of how AIs could communicate emotions in future in addition to language - by changing colour across a rich field of hues. This doesn't try to make a direct analogy to our emotions and in that sense makes the problem clearer as, in a sense, a clustering of internal states.

what do you mean by "know how to interact with you"? what should the one-sentence introduction consist of that conveys this?

5Severin T. Seehrich
This is one of the points I'm less sure about because often enough, the rest of the message will implicitly answer it. In addition, what to include is highly dependent on context and who you are writing to. Two very general recommendations: - Something that helps the other person gauge how long the inferential distances between you two are, so that communication can be as quick as possible and as thorough as necessary. - Something that helps them gauge your level of seniority. It's unfortunate but true that the time of people a couple levels of seniority above your own is extremely valuable. For example, it would hardly make sense for a Nick Bostrom to make time for helping a bright-but-not-Einstein-level high school student he never met decide which minor to choose in university. If people can't gauge your level of seniority, they might misjudge whether they are the right person for you to talk to, and then you might end up in a conversation that is extremely awkward and a waste of time for either side. Some examples: - "Hi! I'm xyz, Ops lead at Linear." - "Hi! I'm a computer science undergrad at Atlantis University and a long-time lurker on LessTrue." - ...
1Alexander Gietelink Oldenziel
Although good alignment research has been done that does not involve maths [e.g. here and here ] good math* remains the best high-level proxy of nontrivial, substantive, deep ideas that will actually add up to durable knowledge. *what distinguishes good math from bad math? that's a tricky question that requires a strong inside view. 

"Liquid breaks" sounds like a music genre.

What would be a more apt situation in which to bring up politics on LW?

Are there ways to ease the transition for someone who is socially anxious or never interacts with non-aspies?

5lsusr
Go to quiet places where you're can listen without being expected to talk.

I would also count nonstandard ways to ensure a depressed person doesn't give up on something like meditation, if that helps.

2tangren
I'd like to know that too! I've had some fairly moving meditative experiences, but still find it oddly aversive to do; strength training feels easier, just because my reward system seems to understand it better. I think joining a meditation class can help, as you get a social context and a schedule, but that does depend on having a good teacher nearby.
celeste160

Given that akrasia is usually made worse by depression, in a certain sense I'm not sure a treatment can be effective if it requires too much willpower to carry out.

Yeah, in one sense this question seems impossible to answer - "help me comprehend something incomprehensible to me by definition."

But there's another type of answer; of the utility functions that are alien in OP's sense, it is possible that most will share patterns discernible by humans. OP could be asking what those patterns are.

I'm not sure how worthwhile it is to try to predict the high-level behavior of a generic superintelligence whose goals we don't understand.

I also don't understand why politics isn't considered evolutionarily novel. There is a difference between 1) social organization of a small tribe and 2) management of a polity containing large institutions and thousands to millions of people.

As far as I can tell, no one considers tribal political affiliation desirable.

I'm tempted to write on why one should follow this post's advice so that it actually has an impact on my behavior.

3Adam Zerner
I'd welcome that. It seems useful for the non-expert explanation reasons I talked about in that last paragraph.

Why would poison (or anything with a known negative effect) be used as a placebo?

Of course I understand the drug companies' incentives, but I don't get how that could be justified or look reasonably scientific. Do you have a specific example?

4ChristianKl
My main point is that the ability exists in both the existing system and also a prediction-based one. But when it comes to justifying poison, imagine you have a drug that gives everybody nausia after they take it. You have to tell your trial participants because of informed consent that everybody who gets verum will very likely get nausia soon after taking it.  You can justify giving the patients in the placebo group a poison that causes nausia because otherwise all the patients know whether or not they got the placebo.  Many studies don't specify what they are using as placebo.  I do remember cases where someone argued that the dose of the control group was likely net negative but don't have specific links right now. 
3frontier64
I think his point is that the same failure state Measure mentioned, doctors giving patients poison and correctly predicting outcomes, is just as likely as for the current clinical trial scheme.

Yeah, I wanted to clarify that this is closer to what I mean, but I wondered if habryka endorses downvoting comments seen as obviously wrong.

Such a strategy seems like it could lead to a general culture of circlejerking, but it might also prevent the same tired discussions from playing out repeatedly. I don't have a good sense of the relative importance of these concerns, and I suspected an LW admin's intuition might be better calibrated.

3Ruby
[LW Admin here]. Karma gets interpreted in a number of ways, but one of my favorites is that karma is for "how much attention you want a thing to get." If a comment seems wrong but valuable for people to read, I might upvote it. If it seems correct but it is immaterial and unhelpful, I might downvote it. Generally, there's a higher bar for things that wrong to be worth reading by people. I can't fully explain other people's voting behavior, but in my case I'd often downvote wrong things just so they get less attention.

Upvoted because it seems harmful to downvote comments that are wrong but otherwise adhere to LW norms.

(If this comment actually does violate a norm, I'm curious as to which.)

habryka*180

I think voting should happen primarily based on quality of contribution, not norm violation. For norm violations you can report it to the mods. I think it's pretty reasonable for some comments that don't violate any norms but just seem generically low quality to get downvoted (and indeed, most of the internet is full of low-quality contributions that make discussions frustrating and hard to interact with, without violating any clear norms, and our downvote system holds them at least somewhat at bay).