All of osten's Comments + Replies

Orange peel is a standard ingredient in Chinese cooking. Just be careful with pesticides.

Why wouldn't AI agents or corporations led by AIs develop their own needs and wants, the way corporations do that as well currently? An economy has no need for humans, it only needs needs and scarce potential to meet those needs.

Agree, but not sure what you are implying. Is it, Sam is not as concerned about risks because the expected capabilities are lower than he publicly lets on, timelines are longer than indicated and hence we should be less concerned as well? 

On the one hand this is consistent with Sam's family planning. On the other hand, other OpenAI employees that are less publicly involved and perhaps have less marginal utility from hype messaging have consistent stories (e.g. roon, https://nitter.poast.org/McaleerStephen/status/1875380842157178994#m).

The implication is that you absolutely can't take Altman at his bare word, especially when it comes to any statement he makes that, if true, would result in OpenAI getting more resources. Thus you need to a) apply some interpretative filter to everything Altman says, and b) listen to other people instead who don't have a public track record of manipulation like Altman.

the only way to appropriately address [long term risk from AI systems of incredible capabilities] is to ship product and learn.

4MondSemmel
The frustrating thing is that in some ways this is exactly right (humanity is okay at resolving problems iff we get frequent feedback) and in other ways exactly wrong (one major argument for AI doom is that you can't learn from the feedback of having destroyed the world).

True, but nitpicking about the memorability: The long-term value may not be in the short-term value of the conversation itself. It may be in the introduction to someone by someone you briefly got to know in an itself low-value conversion, by the email for a job getting forwarded to you etc. You wouldn't necessarily say the conversation was memorable, but the value likely wouldn't have been realized without it.

It doesn't need to be a singular high-value conversation. I'd say the long-term value of conversations is heavy tailed and so it may pay to have lots of conversations of low expected value. https://www.sciencedirect.com/science/article/abs/pii/B9780124424500500250

6Garrett Baker
If conversations are heavy tailed then we should in fact expect people to have singular & likely memorable high-value conversations.

The rationalist term is ‘front running steel man, for German Claude suggests Replikationsmangeleinsichtsanerkennung (‘acknowledgement of the insight despite lack of replication’):

Tess: There should be a German word that means “I see where you’re going with this, and while I agree with the point you will eventually get to, the scientific study you are about to cite doesn’t replicate.”

I would interpret Replikationsmangeleinsichtsanerkennung as 'positive recognition that sb. changed his mind and accepts that sth. doesn't replicate'. Perhaps replikationsmangelunbeschadete Zustimmung would be better. Yes, adjectives compound with compound nouns.

Both require an appropriate Overton Window.

6Seth Herd
Yeah but it seems likely we'll get that as soon as we can say "yeah this thing is renting its own services, on purpose. Experts agree that's what's happening." Something that can do that is obviously alive. Current attitudes make sense in the face of real systems that are obviously not independently agentic.i expect dramatic updates when we have real evidence of real competent agents.
Answer by osten10

Two complications perhaps?: Earth surface curvature + hills, does it work at night?

Relatedly, I schedule all my todos and my todo list contains only the ones scheduled to within the last week. If a todo is at risk of lapsing from the list, I will either have to actively reschedule it, in which case it is probably important to me, or it just drops into a list of stale todos. Occasionally I remember stale todos and can reschedule them or when I have enough time I browse the list of stale todos to see if there is anything still interesting.

Edit: This method helps because I get overwhelmed and anxious from overly long todo lists.

This gave me an idea: You could have a website where bidders upload a problem description, public and private data, an optimization goal (in the form of a solution-evaluation algorithm), and a bid like 'for a solution at least x good I pay y'. Takers can submit algorithms that produce solutions. They get run against the data with time and memory limit as specified by the bidder and if they match the solution quality the taker gets paid.

Is there something like this around?

1StrivingForLegibility
It seems straightforward! Kaggle is the closest example I've been able to think of. But yes that's totally the sort of thing that I think would constitute an optimization market!

Thanks, I appreciate it. It must be very difficult. Please don't lose your patience.

So will the maximally curious AI be curious about what would happen if you genetically modified all humans to become unicorns?

4ChristianKl
If curiosity is driving then it's more likely that you will genetically modify some into becoming unicorns and others into becoming dinosaurs.  In general, you would expect that AI to do a lot of different things and not keep all humans the same. 

But it also provides incredibly easy interpretability, because these systems think in English.

I'm not sure this point will stand because it might be cheaper to have them think in their own language: https://www.lesswrong.com/posts/bNCDexejSZpkuu3yz/you-can-use-gpt-4-to-create-prompt-injections-against-gpt-4

1IC Rainbow
I doubt that any language less represented than English (or JS/Python) would be better since the amount of good data to ingest would be much less for them.
7Seth Herd
I think that could become a problem. I think that type of compression is useful but not really dramatic, but I'm not sure. So there would still be an alignment tax. But it might be small enough to not prevent people from paying it. Keeping internal processes in natural language will also make for easier debugging just to get the system to work well and do things you want. People are still going have to want to make aligned systems. This approach might just make it a whole lot easier.

Mr ChatGPT, does my bum look too big in that dress?

2Dzoldzaya
Not sure what your point is here...
2Optimization Process
Thanks for the link! Respectable Person: check. Arguing against AI doomerism: check. Me subsequently thinking, "yeah, that seemed reasonable": no check, so no bounty. Sorry!   It seems weaselly to refuse a bounty based on that very subjective criterion, so, to keep myself honest, I'll post my reasoning publicly. If I had to point at parts that seemed unreasonable, I'd choose (a) the comparison of [X-risk from superintelligent AIs] to [X-risk from bacteria] (intelligent adversaries seem obviously vastly more worrisome to me!) and (b) "why would I... want to have a system that wants to reproduce? ...Those are bad things, don't do that... regulate those." (Everyone will not just!) (I post these points not in order to argue about them, just as a costly signal of my having actually engaged intellectually.) (Though, I guess if you do want to argue about them, and you convince me that I was being unfairly dismissive, I'll pay you, I dunno, triple?)

Are you sure about the imcreased sustained attention among cocaine users? The abstract in your link seems to suggest the opposite.

1Xodarap
I very well could be wrong, but I believe that slower response time indicates less impulsivity, which is "better" sustained attention. Here's how the review article I'm summarizing describes the findings:
4Eric Neyman
Thanks for mentioning Asch's conformity experiment -- it's a great example of this sort of thing! I might come back and revise it a bit to mention the experiment. (Though here, interestingly, a participant's action isn't exactly based on the percentage of people giving the wrong answer. It sounds like having one person give the right answer was enough to make people give the right answer, almost regardless of how many people gave the wrong answer. Nevertheless, it illustrates the point that other people's behavior totally does influence most people's behavior to quite a large degree, even in pretty unexpected settings.)