LESSWRONG
LW

All of nahoj's Comments + Replies

Interest poll: A time-waster blocker for desktop Linux programs
nahoj1-2

2/4 Maybe I'll have a look if/when it's ready

Reply1
1nahoj
4/4 I need to alpha-test this right now.
1nahoj
3/4 I want this, please release it!
1nahoj
2/4 Maybe I'll have a look if/when it's ready
1nahoj
1/4 Not for me.
Secondary forces of debt
nahoj32

Reading this gave me an uncomfortable moment considering my feelings for all the people who expect things of me or of whom I expect things, outside the specific context of debt.

It makes me think of the very common case in society of someone taking care of an elderly or otherwise care-needing relative.

But like @Dagon says, this is only one aspect of such interpersonal relationships, out of many. In particular, taking this "as reason to avoid debt in all its forms more" sounds to me like hoping never to get in a situation which in fact happens all the time. ... (read more)

Reply
Math-to-English Cheat Sheet
nahoj10

Thanks, I have applied most suggestions.

Indeed I didn't choose the formulas myself but just told GPT to produce some, and then removed a few that seemed dubious or irrelevant.

Reply
Will the first AGI agent have been designed as an agent (in addition to an AGI)?
nahoj10

Right. So, considering that the most advanced AIs of a leading AI company such as OpenAI are not agents, what do you think of the following plan to solve or help solve AI risk: keep making more and more powerful Q&A AIs that are not agents until we have ones that are smarter than us, then ask them how to solve the problem. Do you think this is a safe and reasonable pursuit? Or do you think we just won't get to superhuman intelligence that way?

Reply
3Charlie Steiner
You could get to superintelligence that way, except that before that happens, someone else is going to make an AI that actively seeks out information and navigates the real world.  And it's not all that safe in an absolute sense - large sequence models are so trustworthy specifically because we're using them on problems where we can give lots of examples of humans solving them. By default, when you ask a big Q&A AI how to solve alignment, it will just tell you the sort of bad answer a human would give. Trying to avoid that default carries risks, and just seems like the wrong thing to be doing. Building tools to help humans solve the problem isn't crazy, but this is different than expecting the answer to spring fully formed from a big AI that you trust without knowing much about alignment.
Will the first AGI agent have been designed as an agent (in addition to an AGI)?
nahoj10

I'm not sure I understand, do you mean that considering these possibilities is too difficult because there are too many or that it's not a priority because AIs not designed as agents are less dangerous? Or both?

Reply
3Charlie Steiner
The latter, specifically because it's less likely.
Will the first AGI agent have been designed as an agent (in addition to an AGI)?
nahoj10

Thank you for your answer. In my example I was thinking of an AI such as a language model that would have latent ≥human-level capability without being an agent, but could easily be made to emulate one just long enough for it to get out of the box, e.g. duplicate itself. Do you think this couldn't happen?

More generally, I am wondering if the field of AI safety research studies somewhat specific scenarios based on the current R&D landscape (e.g. "A car company makes an AI to drive a car and then someone does xyz and then paperclips") and tailor-made safe... (read more)

Reply
3Charlie Steiner
I think that would have the form of current AI research, but would involve extremely souped-up models of the world relative to what we have now (even moreso for the self-driving car), to the extent that it's not actually that close to modern AI research. I think it's reasonable to focus our efforts on deliberate attempts to make AGI that navigates the real world.