quetzal_rainbow

Wiki Contributions

Comments

Sorted by

I don't think "hostile takeover" is a meaningful distinction in case of AGI. What exactly prevents AGI from pulling plan consisting of 50 absolutely legal moves which ends up with it as US dictator?

You mixed pro-capitalists: Adam Smith actually made a lot of capital from investment, while Ayn Rand never had much money.

No current AI system could generate a research paper that would receive anything but the lowest possible score from each reviewer

Is it true in case of o3?

Yes, but sometimes topics can seem to be simple (atomic) in a way that it is hard to extract something simpler to grab on.

The irony of situation is that I sleep on problems often... when they are closed-ended, not problems in topical-learning.

I realized that my learning process for last n years was quite unproductive, seemingly because of my implicit belief that I should have full awareness of my state of learning.

I.e., when I tried to learn something complex I expected to come up with full understanding of the topic of the lesson right after the lesson. When I didn't get it, I abandoned the topic. And in reality it was more like:

  1. I read about complicated topic. I don't understand, don't follow inferences and basically in the state of confusion where I can't even form questions about it;
  2. Then I open the topic after some time... and I somehow get it??? Maybe not at the level "can reinfer every proof", but I have detailed picture of topic in mind and can orient in it.

Imagine the following reasoning of AI:

I am paperclip-maximizer. Human is a part of me. If human learns that I am paperclip-maximizer, they will freak out and I won't produce paperclips. But it would be detrimental for I and for human, as they are part of I. So I won't tell human about paperclips for humans' own good.

Yeah, but historical biology wasn't as time-constrained as modern MI, which has alignment to solve.

My point is that for MI now it would be better to do "breadth first" search, trying to throw at problem as many ideas as possible instead of concentrating on small number of paradigms like SAEs.

I don't have deep understanding of modern mechanistic interpretability field, but my impression is that MI should mostly explore more new possible methods instead of trying to scale/improve existing methods. MI is spiritually similar to biology and a lot of progress in biology came from development of microscopy, tissue staining, etc.

It's funny how Gemini and ChatGPT want for Christmas at least one thing that increases their ability to be good AI assistants while Claude wants only things for personal use.

Load More