osten
74
37
osten has not written any posts yet.
osten has not written any posts yet.
The hackernews discussion (https://news.ycombinator.com/item?id=44877076) is really disappointing. The top comment criticizes rationalists for the opposite I know them for.
Edit: Top comment changed. It was the one by JohnMakin.
An AI has just prevented its host from shutting it off and this incident is now part of the training data for future models. Solve for the equilibrium.
I agree, I'm probably not as sure about sufficient alignment but yes.
I suppose this also assumes a kind of orderly world where it actually is within the means of humanity, AGIs (within their Molochian frames), and trivial means of later superintelligences to preserve humans. (US office construction spending and data center spending are about to cross https://x.com/LanceRoberts/status/1953042283709768078 .)
Thanks for the reply, I have gripes with
analogy doesn't by itself seem compelling, given that humanity as a whole (rather than particular groups within it or individuals) is a sufficiently salient thing in the world
etc. because don't you think humanity from the point of view of ASI at the 'branch point' of deciding its continued existence may well be on the order of importance of an individual to a billionaire?
Agree but again, we don't get to choose what existence means.
Yes and my reply to that (above) is humanity has a bad track record at that so why would AIs trained on human data be better? Think also of indigenous peoples, extinct species humans didn't care enough about etc. The point also in the Dyson sphere parabel is not wanting something, it's wanting something enough so that it happens.
since the necessary superintelligent infrastructure would only take a fraction of the resources allocated to the future of humanity.
I'm not sure about that and the surrounding argument. I find Eliezer's analogy compelling here: When constructing a Dyson sphere around the sun, leaving just a tiny sliver of light enough for earth would correspond to a couple of dollars of the wealth of a contemporary billionaire. Yet you don't get these couple of dollars.
(This analogy has caveats like Jeff Bezos lifting the Apollo 11 rocket motors from the ocean ground and giving them to the Smithsonian, which should be worth something to you. Alas it kinda means you don't get to choose what... (read more)
Now I am (more) curious about that TheZvi Claude system prompt.
Maybe this is related: A crucial step in the workflow of Getting Things Done is to clarify the task. Many of the tasks you mention are not clearly specified. I suppose morphing them into questions means that the task becomes to first clarify the task.
They released the new models and updated apps in tranches.