Rafael Harth

I'm an independent researcher currently working on a sequence of posts about consciousness. You can send me anonymous feedback here: https://www.admonymous.co/rafaelharth.

Sequences

Consciousness Discourse
Litereature Summaries
Factored Cognition
Understanding Machine Learning

Wikitag Contributions

Comments

Sorted by

I think the basic mechanism here is that people actually judge most things by their pleasantness to a much larger degree than they tend to admit, and general bad mood decreases pleasantness.

Directionally I agree with lc saying it sounds like you're depressed, but I don't think it actually has to be anywhere near clinical depression. I think "I'm generally sadder so things seem less exciting" is a very commonly true description. You reporting it could have more to do with you being more introspective than with how extreme the condition is.

The general remedy is just to improve your well-being, which of course is very difficult. But I don't think there's any conceptual move you can make that will help here; the core issue very much seems to be a lack of fulfillment.

On the contrary, if your AGI definition includes most humans, it sucks.

All the interesting stuff that humanity does is done by thing that most humans can't do. What you call baby AGI is by itself not very relevant for any of the dangers about AGI discussed in e.g. superintelligence. You could quibble with the literal meaning of "general" or whatever but the historical associations with the term seem much more important to me. If people read years of how AGI will kill everyone and then you use the term AGI, obviously people will think you mean the thing with the properties they've read about.

Bottom line is, current AI is not the thing we were talking about under the label AGI for the last 15 years before LLMs, so we probably shouldn't call it AGI.

But not that unpleasant, I guess. I really wonder what people think when they see a benchmark on which LLMs get 30%, and then confidently say that 80% is "years away". Obviously if LLMs already get 30%, it proves they're fundamentally capable of solving that task[1], so the benchmark will be saturated once AI researchers do more of the same. Hell, Gemini 2.5 Pro apparently got 5/7 (71%) on one of the problems, so clearly outputting 5/7-tier answers to IMO problems was a solved problem, so an LLM model getting at least 6*5 = 30 out of 42 in short order should have been expected. How was this not priced in...?

Agreed, I don't really get how this could be all that much of an update. I think the cynical explanation here is probably correct, which is that most pessimism is just vibes based (as well as most optimism).

If a child plays too many videogames you might take away their switch, and while that might decrease their utility, I'd hardly describe it as suffering in any meaningful sense.

Not sure this is important to discuss, but I definitely would. If I remember correctly, this kind of thing had a pretty strong effect on me when I was small, probably worse than getting a moderate injury as an adult. I feel like it's very easy to make a small kid suffer because they're so emotionally defenseless and get so easily invested in random things.

I still think you have rose-colored glasses about how discussion works -- not how it should work, but how it does work -- and that this is causing you to make errors. E.g., the habryka quote sounds insane until you reflect on what discourse is actually like. E.g., in the past debate we've had you initially said that the moderation debates weren't about tone, before we got to the harder normative stuff.

It's probably possible to have your normative views while having a realistic model of how discussions work, but err I guess I think you haven't reconciled them yet, at least not in, e.g., this post.

My reactions

  • It's probably good if companies are optimizing for IMO since I doubt this generalizes to anything dangerous.
  • This tweet thread is trying super hard to sound casual & cheerful but it is entirely an artificial persona, none of this is natural. It is extremely cringe.

Didn't even know that! (Which kind of makes my point.)

Even though all of that is true, it just seems like such a strange way to frame the problem. Yes, the commenter's catalogue of top-level posts does provide some information about how productive a back and fourth would be. But if Said had two curated posts but commented in exactly the same style, the number of complaints about him would be no different; the only thing that would change is that people wouldn't use the 'he doesn't write top level posts' as a justification. (Do you actually doubt this?) Conversely if everyone thought his comments were insightful and friendly, no one would care about his top-level posts. The whole point about top level posts is at least 80% a rationalization/red herring.

Yes, if you assume that the probability of seeing an observation was 100% under your favorite model then seeing it doesn't update you away from that model, but that assumption is obviously not true. (And I already conceded that the update is marginal!)

Load More