The more recent Safeguarded AI document has some parts that seem to me to go against the interpretation I had, which seems to go along the lines of this post.
Namely, that davidad's proposal was not "CEV full alignment on AI that can be safely scaled without limit" but rather "sufficient control of AI that is as little more powerful as possible than sufficiently powerful for ethical global non-proliferation".
In other words:
On "lab leaders would choose to stop if given the coordination-guaranteed button" vs "big ol' global mood shift", I think the mood shift is way more likely (relatively) for two reasons.
One of which was argued directionally about and I want to capture more crisply the way I see it, and the other I didn't see mentioned and might be a helpful model to consider.
The "inverse scaling law" for human intelligence vs rationality. "AI arguments are pretty hard" for "folks like scott and sam and elon and dario" because it's very easy for intelligent people to wade
Thanks. Yeah, makes sense for official involvement to be pretty formal and restricted.
More in a "just in case someone reads this and has something to share" I'd like to extend the question to unofficial efforts others might be thinking about or coordinating around.
It would also be good if those who do get involved formally feel like there's enough outside interest to be worth their time to make informal requests for help, like "if you have 10h/week I'm looking for a volunteer research assistant to help me keep up with relevant papers/news leading up to the summit."
Beyond the people with the right qualifications to get directly involved right away e.g. via the form, are there "supporting role" tasks/efforts that interested individuals of different skillsets and locations could help out with? Baseline examples could be volunteering to do advocacy, translation, ops, making introductions, doing ad-hoc website/software development, summarizing/transcribing/editing audio/video, etc.? Is there a recommended discord/slack/other where this kind of support is being coordinated?
Considering all humans dead, do you still think it's going to be the boring paperclip kind of AGI to eat all reachable resources? Any chance that inscrutable large float vectors and lightspeed coordination difficulties will spawn godshatter AGI shards that we might find amusing or cool in some way? (Value is fragile notwithstanding)
I can see how advancing those areas would empower membranes to be better at self-defense.
I'm having a hard time visualizing how explicitly adding concept, formalism, or implementation of membranes/boundaries would help advance those areas (and in turn help empower membranes more).
For example, is "what if we add membranes to loom" a question that typechecks? What would "add membranes" reify as in a case like that?
In the other direction, would there be a way to model a system's (stretch goal: human child's; mvp: a bargaining bot's?) membrane quantitatively s... (read more)