If you wanna talk about the humanity(ies), well I looked up Chief Vision Officer of AISI Adam Russel, and he has an interesting profile.

Russell completed a Bachelor of Arts in Cultural Anthropology from Duke University, and an M.Phil. and a D.Phil. in Social Anthropology from University of Oxford, where he was a Rhodes Scholar.^[2] He played with the Oxford University RFC for four varsity matches and also worked with the United States national rugby union team, and worked as High Performance director for the United States women's national rugby union team in the 2014 and 2017 Women's Rugby World Cups.^[3]

Russell was in the industry, where he was a senior scientist and principal investigator on a wide range of human performance and social science research projects and provided strategic assessments for a number of different government organizations.^[2][4] Russell joined Intelligence Advanced Research Projects Activity (IARPA) as a program manager.^[2][4] He developed and managed a number of high-risk, high-payoff research projects for the Office of the Director of National Intelligence.^[2] Russell joined DARPA as a program manager in July 2015.^[2][4] His work there focused on new experimental platforms and tools to facilitate discovery, quantification and "big validation" of fundamental measures in social science, behavioral science and human performance.^[2]

In 2022, secretary Xavier Becerra selected Russell to serve as the acting deputy director for the Advanced Research Projects Agency for Health (ARPA-H), effective June 6. In this role, Russell leads the process to stand up ARPA-H.^[5]

Hmm he's done a lot of macho human-enhancement-adjacent stuff. I wonder if there were some centaurists involved here.

• I previously noted a lot of research projects in neurotech research in DoD funding awards. I'm making a connection between this and a joke I heard recently on a navy seals podcast. "The guys often ask what they can do to deal with drones. So you start showing them how to work the jammer devices, or net guns, and their eyes glaze over, it's not what they wanted, they're disappointed. They're thinking like, 'no... how can I deal with it. Myself.' "
• So even though alignment-by-merger is kinda obviously not going to work (you'd have to reverse-engineer two vats of inscrutable matrices, instead of one. And the fleshy pink one wasn't designed to be read from and can only be read on a neuron-by-neuron level after being plastinated (which also kills it). AGI alignment is something that a neuralink cannot solve.), it's conceivable that it's an especially popular line of thought among military/sports types.

Otherwise, this kinda lines up with my confessions on manhattan projects for AGI. You arguably need an anthropologist to make decisions about what 'aligned' means. I don't know if you really need one (a philosophically inclined decision theorist, likely to already be involved already, would be enough for me) but I wouldn't be surprised to see an anthropologist appointed in the most serious projects.

Feel like there's a decent chance they already changed their minds as a result of meeting him or engaging with their coworkers about the issue. EAs are good at conflict resolution.

Wouldn't really need reward modelling for narrow optimizers. Weak general real-world optimizers, I find difficult to imagine, and I'd expect them to be continuous with strong ones, the projects to make weak ones wouldn't be easily distinguishable from the projects to make strong ones.

Oh, are you thinking of applying it to say, simulation training.

Cool then.

Are you aware that prepotence is the default for strong optimizers though?

Are you proposing applying this to something potentially prepotent? Or does this come with corrigibility guarantees? If you applied it to a prepotence, I'm pretty sure this would be an extremely bad idea. The actual human utility function (the rules of the game as intended) supports important glitch-like behavior, where cheap tricks can extract enormous amounts of utility, which means that applying this to general alignment has the potential of foreclosing most value that could have existed.

Example 1: Virtual worlds are a weird out-of-distribution part of the human utility function that allows the AI to "cheat" and create impossibly good experiences by cutting the human's senses off from the real world and showing them an illusion. As far as I'm concerned, creating non-deceptive virtual worlds (like, very good video games) is correct behavior and the future would be immeasurably devalued if it were disallowed.

Example 2: I am not a hedonist, but I can't say conclusively that I wouldn't become one (turn out to be one) if I had full knowledge of my preferences, and the ability to self-modify, as well as lots of time and safety to reflect, settle my affairs in the world, set aside my pride, and then wirehead. This is a glitchy looking behavior that allows the AI to extract a much higher yield of utility from each subject by gradually warping them into a shape where they lose touch with most of what we currently call "values", where one value dominates all of the others. If it is incorrect behavior, then sure, it shouldn't be allowed to do that, but humans don't have the kind of self-reflection that is required to tell whether it's incorrect behavior or not, today, and if it's correct behavior, forever forbidding it is actually a far more horrifying outcome, what you'd be doing is, in some sense of 'suffering', forever prolonging some amount of suffering. That's fine if humans tolerate and prefer some amount of suffering, but we aren't sure of that yet.

(instutitional reform take, not important due to short timelines, please ignore)

The kinds of people who do whataboutism, stuff like "this is a dangerous distraction because it takes funding away from other initiatives", tend also to concentrate in low-bandwidth institutions, the legislature, the committee, economies righteously withering, the global discourse of the current thing, the new york times, the ivy league. These institutions recognize no alternatives to them, while, by their nature, they can never grow to the stature required to adequately perform the task assigned to them.
I don't think this is a coincidence, and it makes it much easier for me to sympathize with these people: They actually believe that we can't deal with more than one thing at a time.

They generally have no hope for decentralized decisionmaking, and when you examine them closely you find that they don't really seem to believe in democracy, they've given up on it, they don't talk about reforming it, they don't want third parties, they've generally never heard of decentralized public funding mechanisms, certainly not futarchy. So it's kind of as simple as that. They're not being willfully ignorant. We just have to show them the alternatives, and properly, we basically haven't done it yet. The minarchists never offered a solution to negative externalities or public goods provision. There are proposals but the designs are still vague and poorly communicated. There has never been an articulation of enlightened technocracy, which is essentially just succeeding at specialization or parallelization in executive decisionmaking. I'm not sure enlightened technocracy was ever possible until the proposal of futarchy, a mechanism by which non-experts can hold claimed experts accountable.

Yeah I'm pretty sure you would need to violate heisenberg uncertainty in order to make this and then you'd have to keep it in a 0 kelvin cleanroom forever.

A practical locked battery with tamperproofing would mostly just look like a battery.

I don't recognize wikipedia's theories as predictive. Mine has some predictions, but I hope it's obvious why I would not be interested in making this a debate or engaging much in the conceptual dismantling of subcultures at all.

I didn't read RS's claim as the claim that all subcultures persist through failure, but now that you ask, no, yeah, ime a really surprising number of these subcultures actually persist through failure.

• I know of a fairly influential subculture of optics-oriented politics technologists who've committed to a hostile relationship towards transhumanism. Transhumanism (the claim that people want to change in deep ways and that technology will fairly soon permit it) suggests that racial distinctions will become almost entirely irrelvant, so in order to maintain their version of afrofuturism where black and white futurism remain importantly distinct projects, they have to find some way to deny transhumanism. But rejecting transhumanism means they are never allowed to actually do high quality futurism because they can't ask transhumanist questions and get a basic sense of what the future is going to be like. Or like, as soon as any of them do start asking those questions, those people wake up and drop out of that subculture. I've also met black transhumanists who identified as afrofuturists though. I can totally imagine articulations of afrofuturism that work with transhumanism. So I don't know how the entire thing's going to turn out.
• Anarcho-punks fight only for the underdogs. That means they're attached to the identity of being underdogs, as soon as any of them start really winning, they'd no longer be recognised as punk, and they know this, so they're uninterested in — and in many cases, actively opposed to — succeeding in any of their goals. There are no influential anarcho-punks, and as far as I could gather, no living heroes.
• BDSM: My model of fetishes is that they represent hedonic refuges for currently unmeetable needs, like, deep human needs that for one reason or another a person can't pursue or even recognise the real version of the thing they need in the world as they understand it, I think it's a protective mechanism to keep the basic drive roughly in tact and wired up by having the subject pursue symbolic fantasy versions of it. This means that getting the real thing (EG, for submissives, a committed relationship with someone you absolutely trust. For doms... probably a sense of safety?) would obsolete the kink, and it would wither away. I think they mostly don't know this, but the mindset in which the kink is seen as the objective requires that the real thing is never recognised or attained, so these communities reproduce best by circulating memes that make it harder to recognise the real thing.

I guess this is largely about how you define the movements' goals. If the goal of punk is to have loud parties with lots of drugs, it's perfect at that. If the goal is to bring about anarchosocialism or thrive under a plural geopolitical order, it's a sworn loser.