All of ukc10014's Comments + Replies

Quite like this forecast from Andrew Critch on milestones on AI development, and my reactions:

The timeline he suggests, in ~10 years, we face choice 6a/b, which implies at least 3 possibilities:

A) we need society-level consensus (which might be force-backed) that humans can/should control agents (or digital entities, more generally) who are in all (economically/militarily) relevant aspects, superior to us. Assuming they fit within the moral circle as we currently conceive (@davidchalmers42 @jeffrsebo Thomas Metzinger and Nick Bostrom / Carl Shulman, have ... (read more)

Thanks for this - a useful framing - a slightly more poetic framing would perhaps be the early writing of Nick Land (which has picked up notoriety owing to some combination of his political views and/or the people that have adopted/adapted his ideas). This article summarises Land's thoughts on capitalism and AGI.

This is good, both for training data reasons, but also because it tries to present reasons to the successor (I'm not using ASI as that seems a pretty loaded/unclear term, i.e. the post could be of interest to proto-AGIs/ASIs and could be a profitable conversation with something well before AGI), acting in (our projection of) "its" "self-interest" (quoted as those are pretty anthropomorphic terms and bake in assumptions about identity).

I had a couple of comments:

• the essay doesn't make a positive case for keeping us around (in the sense of 'try to preserv

On the overall point of using LLMs-for-reasoning this (output of a team at AI Safety Camp 2023) might be interesting - it is rather broad-ranging and specifically about argumentation in logic, but maybe useful context: https://compphil.github.io/truth/ 

That’s really useful, thank you.

This is really useful, thank you - Bach's views are quite hard to capture without sitting through hours of podcasts (though he has re-started writing).

In response to Roman’s very good points (i have only for now skimmed the linked articles); these are my thoughts:

I agree that human values are very hard to aggregate (or even to define precisely); we use politics/economy (of collectives ranging from the family up to the nation) as a way of doing that aggregation, but that is obviously a work in progress, and perhaps slipping backwards. In any case, (as Roman says) humans are (much of the time) misaligned with each other and their collectives, in ways little and large, and sometimes that is for good or bad... (read more)

I've taken a crack at #4 but it is more about thinking through how 'hundreds of millions of AIs' might be deployed in a world that looks, economically and geopolitically, something like today's (i.e. the argument in the OP is for 2036 so this seems a reasonable thing to do). It is presented as a flowchart which is more succinct than my earlier longish post.

Good catch, thank you - fixed & clarified !

I noticed that footnotes don't seem to come over when I copy-paste from Google Docs (where I originally wrote the post), hence I have to put them in individually (using the LW Docs editor). Is there a way of just importing them? Or is the best workflow to just write the post in LW Docs?

Perhaps this is too much commentary (on Rao's post), but given (I believe) he's pretty widely followed/respected in the tech commentariat, and has posted/tweeted on AI alignment before, I've tried to respond to his specific points in a separate LW post.  Have tried to incorporate comments below, but please suggest anything I've missed.  Also if anyone thinks this isn't an awful idea, I'm happy to see if a pub like Noema (who have run a few relevant things e.g. Gary Marcus, Yann LeCun, etc.) would be interested in putting out an (appropriately edi... (read more)

A fool was tasked with designing a deity. The result was awesomely powerful but impoverished - they say it had no ideas on what to do. After much cajoling, it was taught to copy the fool’s actions. This mimicry it pursued, with all its omnipotence.  

The fool was happy and grew rich.

And so things went, ‘til the land cracked, the air blackened, and azure seas became as sulfurous sepulchres.

As the end grew near, our fool ruefully mouthed something from a slim old book: ‘Thou hast made death thy vocation, in that there is nothing contemptible.’
 

Thanks ! I'd love to know which points you were uncomfortable with...

Here's my submission, it might work better as bullet points on a page.

AI will transform human societies over the next 10-20 years.  Its impact will be comparable to electricity or nuclear weapons.  As electricity did, AI could improve the world dramatically; or, like nuclear weapons, it could end it forever.  Like inequality, climate change, nuclear weapons, or engineered pandemics, AI Existential Risk is a wicked problem.  It calls upon every policymaker to become a statesperson: to rise above the short-term, narrow inte... (read more)