All of pseud's Comments + Replies

I agree there's nothing about consciousness specifically, but it's quite different to the hidden prompt used for GPT-4 Turbo in ways which are relevant. Claude is told to act like a person, GPT is told that it's a large language model. But I do now agree that there's more to it than that (i.e., RLHF).

It's possibly just matter of how it's prompted (the hidden system prompt). I've seen similar responses from GPT-4 based chatbots. 

The cited markets often don't support the associated claim. 

"This question will resolve in the negative to the dollar amount awarded"

This is a clear, unambiguous statement.

If we can't agree even on that, we have little hope of reaching any kind of satisfying conclusion here.

Further, if you're going to accuse me of making things up (I think this is, in this case, a violation of the sensible frontpage commenting guideline "If you disagree, try getting curious about what your partner is thinking") then I doubt it's worth it to continue this conversation.

Metaculus questions have a good track record of being resolved in a fair matter.

Do they? My experience has been the opposite. E.g. admins resolved "[Short Fuse] How much money will be awarded to Johnny Depp in his defamation suit against his ex-wife Amber Heard?" in an absurd manner* and refused to correct it when I followed up on it.

*they resolved it to something other than the amount awarded to Depp despite thatamount being the answer to the question and the correct resolution according to the resolution criteria

My comment wasn't well written, I shouldn't have used the word "complaining" in reference to what Said was doing. To clarify:

As I see it, there are two separate claims:

1. That the complaints prove that Said has misbehaved (at least a little bit)
2. That the complaints increase the probability that Said has misbehaved 

Said was just asking questions - but baked into his questions is the idea of the significance of the complaints, and this significance seems to be tied to claim 1. 

Jefftk seems to be speaking about claim 2. So, his comment doesn't seem like a direct response to Said's comment, although the point is still a relevant one. 

It didn't seem like Said was complaining about the reports being seen as evidence that it is worth figuring out whether thing could be better. Rather, he was complaining about them being used as evidence that things could be better.

It's probably worth noting that Yudkowsky did not really make the argument for AI risk in his article. He says that AI will literally kill everyone on Earth, and he gives an example of how it might do so, but he doesn't present a compelling argument for why it would.[0] He does not even mention orthogonality or instrumental convergence. I find it hard to blame these various internet figures who were unconvinced about AI risk upon reading the article.

[0] He does quote “the AI does not love you, nor does it hate you, and you are made of atoms it can use for something else.”

I'd prefer my comments to be judged simply by their content rather than have people's interpretation coloured by some badge. Presumably, the change is a part of trying to avoid death-by-pacifism, during an influx of users post-ChatGPT. I don't disagree with the motivation behind the change, I just dislike the change itself. I don't like being a second-class citizen. It's unfun. Karma is fun, "this user is below an arbitrary karma threshold" badges are not. 

A badge placed on all new users for a set time would be fair. A badge placed on users with more than a certain amount of Karma could be fun. Current badge seems unfun - but perhaps I'm alone in thinking this. 

Anybody else think it's dumb to have new user leaves beside users who have been here for years? I'm not a new user. It doesn't feel so nice to have a "this guy might not know what he's talking about" badge by my name.

Like, there's a good chance I'll never pass 100 karma, or whatever the threshold is. So I'll just have these leaves by my name forever?

To be clear, that it more-likely-than-not would want to kill everyone is the article's central assertion. "[Most likely] literally everyone on Earth will die" is the key point. Yes, he doesn't present a convincing argument for it, and that is my point. 

The point isn't that I'm unaware of the orthogonality thesis, it's that Yudkowsky doesn't present it in his recent popular articles and podcast appearances[0]. So, he asserts that the creation of superhuman AGI will almost certainly lead to human extinction (until massive amounts of alignment research has been successfully carried out), but he doesn't present an argument for why that is the case. Why doesn't he? Is it because he thinks normies cannot comprehend the argument? Is this not a black pill? IIRC he did assert that superhuman AGI would likely deci... (read more)

Yud keeps asserting the near-certainty of human extinction if superhuman AGI is developed before we do a massive amount of work on alignment. But he never provides anything close to a justification for this belief. That makes his podcast appearances and articles unconvincing - a most surprising, and crucial part of his argument is left unsupported. Why has he made the decision to present his argument this way? Does he think there is no normie-friendly argument for the near-certainty of extinction? If so, it's kind of a black pill with regard to his argumen... (read more)

Why not ask him for his reasoning, then evaluate it? If a person thinks there's 10% x-risk over the next 100 years if we don't develop superhuman AGI, and only a 1% x-risk if we do, then he'd suggest that anybody in favour of pausing AI progress was taking "unacceptable risks for the whole of himanity".

I don't like it. "The problem of creating AI that is superhuman at chess" isn't encapsulated in the word "chess", so you shouldn't say you "solved chess" if what you mean is that you created an AI that is superhuman at chess. What it means for a game to be solved is widely-known and well-developed[0]. Using the exact same word, in extremely similar context, to mean something else seems unnecessarily confusing. 

[0] See https://en.wikipedia.org/wiki/Solved_game

Nit: that's not what "solved" means. Superhuman ability =/= solved.

My thoughts: 

There is no reason to live in fear of the Christian God or any other traditional gods. However, there is perhaps a reason to live in fear of some identical things: 

• We live in a simulation run by a Christian, Muslim, Jew, etc., and he has decided to make his religion true in his simulation. There are a lot of religious people - if people or organisations gain the ability to run such simulations, there's a good chance that many of these organisations will be religious, and their simulations influenced by this fact. 

And the followi... (read more)

How do we know there is no afterlife? I think there's a chance there is. 

Some examples of situations in which there is an afterlife:

• We live in a simulation and whatever is running the simulation decided to set up an afterlife of some kind. Could be a collection of its favourite agents, or a reward for its best behaved ones, or etc.
•  We do not live in a simulation, but after the technological singularity an AI is able to reconstruct humans and decides to place them in a simulated world or to re-embody them
• Various possibilities far beyond our curren

I think that's where these companies' AI safety budgets go: make sure the AI doesn't state obvious truths about the wrong things / represent the actually popular opinions on the wrong things. 

Why are people disagreeing with this statement?

I would gladly suffer a hundred years of pain if it was the only way for me to live one more good day. I think a world where a thousand suffer but one lives a good life is vastly superior to a world in which only ten suffer but none live a good life. Good is a positive quality. But suffering is a zero quality. The absence of a thing, rather than the negative form. So, no matter how much suffering there is, it never offsets even the smallest amount of good.

This is a view that came naturally to me, but it isn't a view I've noticed others share.

The experience... (read more)

I read your post but I thought it was more about aesthetics than technology. 

Horizon Worlds is a program where users can make their own environments and can make aesthetic decisions for themselves.

Yes, I think the graphics are quite simple. I think your explanation relating to  current limitations of VR is enough to address a lot of the OP's confusion/questioning. It's not that Meta is purposefully trying to look bad; they're just sharing the state of the art honestly. It's also worth noting that this seen by Zuckerberg as a temporary state, and his goal very much seems to be photorealism. If you listen to his recent interview with Lex Fridman you'll hear him bring up photorealism again and again. 

Why is Meta sharing their work now i... (read more)

I don't think the screenshot looks that bad. Netizens love to be irrationally extremely negative about Zuck, and it's possible you have been swept up in this.

Yes, I can think of several reasons why someone might downvote the OP. What I should have said is "I'm not sure why you'd think this post would be downvoted on account of the stance you take on the dangers of AGI."

Not sure why you'd think this post would be downvoted. I suspect most people are more than welcoming of dissenting views on this topic. I have seen comments with normal upvotes as well as agree/disagree votes, I'm not sure if there's a way for you to enable them on your post.

 I think a quick web-search is useful. Having read something is an improvement over having no knowledge, and it's ridiculous that people don't do a quick web-search more often. I'm not disagreeing with your point that Googling is better than doing nothing to learn at all. 

My first comment just pointed out that what you learn may be quite inaccurate or out-of-date. 

Now, I'll go further and suggest that what you learn may be purposefully misleading. When it comes to politically or financially sensitive topics (and a searcher won't always reali... (read more)

I assume you mean by "80/20 answer" that betting between half and full pot will be the correct sizing approximately 80% of the time one bets. I think the actual percentage is significantly lower than 80%. 

In general, bet sizing is an incredibly complex topic, but all you have to know is to size your bet between 1/2 and the full size of the pot.

This isn't correct. There are frequently occurring situations in NLHE where betting much less than half the pot or much greater than the full pot is the correct move. This is true both by theoretically-optimal strategy[1] and by practically-optimal strategy[2]. 

This can be used as a data-point when considering the epistemic status of things learned while doing a "quick Google". 

[1] i.e. the Nash equilibrium strategy

[2] i.e. the strategy that makes the most money against real, human opponents