Yes, civilian robot can acquire a gun, but it still makes it safer than a military robot that already has a whole arsenal of military gadgets and weapons right away. It would have to do additional work to acquire it, and it is still better to have it do more work, have more roadblocks than less.

I think we are mainly speculating on what the military might want. It might want to have a button that will instantly kill all their enemies with one push, but they might not get that (or they might, who knows now). I personally do not think they will put more efficient AI (efficient in murdering humans) below... (read more)

Replying toHow does the ever-increasing use of AI in the military for the direct purpose of murdering people affect your p(doom)?

How does the ever-increasing use of AI in the military for the direct purpose of murdering people affect your p(doom)?

I'm being heavily downvoted here, but what exactly did I say wrong? In fact I believe i said nothing wrong.

It does worsen situation with Israel military forces mass murdering Palestinian civilians due to AIs decisions with operators just rubber stamping the actions.

Here is the +972 Mag Report: https://www.972mag.com/lavender-ai-israeli-army-gaza/

I highly advise you to read as it goes into higher details as to how it exactly internally works.

Replying toHow does the ever-increasing use of AI in the military for the direct purpose of murdering people affect your p(doom)?

Justausername2y

How does the ever-increasing use of AI in the military for the direct purpose of murdering people affect your p(doom)?

This AI powered weaponry can always be hacked/modified, even talked to perhaps, this all gives way for them to be used in more than a single way. You can't hack a bullet, you can hack an AI powered ship. So singularly they might not be dangerous, but they don't exist in isolation.

Also, militarisation of AI might create systems that are designed to be dangerous, amoral and without any proper oversight. This opens us a to a flood of potential dangers, some that are even hard to predict now.

-2

Replying toHow does the ever-increasing use of AI in the military for the direct purpose of murdering people affect your p(doom)?

Justausername2y

How does the ever-increasing use of AI in the military for the direct purpose of murdering people affect your p(doom)?

How does militarisation of AI and so-called slaughterbots don't affect your p-doom at all? Plus, I mean, we are clearly teaching AI how to kill, giving it more power and direct access to important systems, weapons and information.

-4

Replying toHow does the ever-increasing use of AI in the military for the direct purpose of murdering people affect your p(doom)?

Justausername2y

How does the ever-increasing use of AI in the military for the direct purpose of murdering people affect your p(doom)?

I'm sorry but the presented example of Israel's Lavender system shows the exact opposite: it exacerbates an already prevalent mass murder of innocent civilians to an even bigger degree, where operators just rubber stamp the decisions. I'm afraid in that example it does absolutely nothing to reduce the indiscriminate large-scale killing, but directly facilitates it. Its right there in the attached +972 Mag report.

And I'm sorry, but did you mean to call the currently targeted Palestinian civilians an unpeaceful "victims" (in quotes)? Because to me that just sounds barbaric and utterly insane and simply immoral, especially during the time of the ongoing Gazan genocide.

-5

How does the ever-increasing use of AI in the military for the direct purpose of murdering people affect your p(doom)?

Justausername

Justausername, johnswentworth

I haven't personally heard a lot of recent discussions about it, which is strange considering that both startups like Andruil and Palantir are developing systems for military use, OpenAI recently deleted a clause prohibiting the use of its products in the military sector, and the government sector is also working on making AI-piloted drones, rockets, information systems (hello, Skynet and AM), etc.

And the most recent and perhaps chilling use of it comes from the Israel's invasion of Gaza, where Israeli army has marked tens of thousands of Gazans as suspects for assassination, using Lavender AI targeting system with little human oversight and a permissive policy for casualties.

So how does all of it affect your p(doom) and what are your general thoughts on this and how do we counter that?

Relevant links:

https://www.972mag.com/lavender-ai-israeli-army-gaza/

https://www.wired.com/story/anduril-roadrunner-drone/

https://www.bloomberg.com/news/articles/2024-01-10/palantir-supplying-israel-with-new-tools-since-hamas-war-started

Replying toSam Altman’s Chip Ambitions Undercut OpenAI’s Safety Strategy

Justausername2y

Sam Altman’s Chip Ambitions Undercut OpenAI’s Safety Strategy

The board should have finished the job.

•••

Enhancing Corrigibility in AI Systems through Robust Feedback Loops

Justausername

Abstract

This article explores the concept of corrigibility in artificial intelligence and proposes a detailed framework for a robust feedback loop to enhance corrigibility. The ability to continuously learn and correct errors is critical for safe and beneficial AI, but developing corrigible systems comes with significant technical and ethical challenges. The feedback loop outlined involves gathering user input, interpreting feedback contextually, enabling AI actions and learning, confirming changes, and iterative improvement. The article analyzes potential limitations of this approach and provides detailed examples of implementation methods using advanced natural language processing, reinforcement learning, and adversarial training techniques. It emphasizes the need for thoughtful design and testing to mitigate risks and biases. Fostering corrigibility... (read 1766 more words →)

Replying toEmbedding Ethical Priors into AI Systems: A Bayesian Approach

Justausername3y

Embedding Ethical Priors into AI Systems: A Bayesian Approach

It should matter very little who I am, what should matter more is what I have.
Why have I written it? I think AI Alignment is necessary and I think what have been proposed here is a good idea, at least in theory and if not wholly then at least partly, and I think it can help with AI alignment.

We could use a combination of knowledge graphs, neural nets, logic modules and clarification through discussion to let AIs make nuanced deductions about ethical situations as they evolve. And while quantifying ethics is challenging, we quantitatively model other complex concepts like emotions and intelligence so difficulty alone is not insurmountable. And it might be... (read more)

Embedding Ethical Priors into AI Systems: A Bayesian Approach

Justausername

Abstract

Artificial Intelligence (AI) systems have significant potential to affect the lives of individuals and societies. As these systems are being increasingly used in decision-making processes, it has become crucial to ensure that they make ethically sound judgments. This paper proposes a novel framework for embedding ethical priors into AI, inspired by the Bayesian approach to machine learning. We propose that ethical assumptions and beliefs can be incorporated as Bayesian priors, shaping the AI’s learning and reasoning process in a similar way to humans’ inborn moral intuitions. This approach, while complex, provides a promising avenue for advancing ethically aligned AI systems.

Introduction

Artificial Intelligence has permeated almost every aspect of our lives, often making decisions... (read 6151 more words →)

-5

Replying toAll AGI Safety questions welcome (especially basic ones) [July 2023]

Justausername3y

All AGI Safety questions welcome (especially basic ones) [July 2023]

Hi! I have a certain proposal that i wanted to make and get some feedback, one of the moderators have directed me here.

The name of the propsal is:
Supplementary Alignment Insights Through a Highly Controlled Shutdown Incentive

My proposal entails constructing a tightly restricted AI subsystem with the sole capability of attempting to safely shut itself down in order to probe, in an isolated manner, potential vulnerabilities in alignment techniques and then improve them.

Introduction:

Safely aligning powerful AI systems is an important challenge. Most alignment research appropriately focuses on techniques like reinforcement learning from human feedback that try to directly optimize AI for human-compatible goals. But analyzing how AI subsystems attempt to circumvent safety constraints... (read 709 more words →)

Autonomous Alignment Oversight Framework (AAOF)

Justausername

Abstract:

To align advanced AIs, an ensemble of diverse, transparent Overseer AIs will independently monitor the target AI and provide granular assessments on its alignment with constitution, human values, ethics, and safety. Overseer interventions will be incremental and subject to human oversight. The system will be implemented cautiously, with extensive testing to validate capabilities. Alignment will be treated as an ongoing collaborative process between humans, Overseers, and the target AI, leveraging complementary strengths through open dialog. Continuous vigilance, updating of definitions, and contingency planning will be required to address inevitable uncertainties and risks.

Introduction:

As advanced AI systems grow in capability and autonomy, ensuring their alignment with human values becomes increasingly critical. The proposed... (read 1106 more words →)

-9

Replying toSupplementary Alignment Insights Through a Highly Controlled Shutdown Incentive

Justausername3y

Supplementary Alignment Insights Through a Highly Controlled Shutdown Incentive

I think such a system where an AI in a sandboxed and airgapped environment is tasked with achieving a state where it is shut down at least once, while trying to overcome the guardrails and restrictions might prove quite usefil in finding out the weakspots in our barriers and then improving them.

Supplementary Alignment Insights Through a Highly Controlled Shutdown Incentive

Justausername

My proposal entails constructing a tightly restricted AI subsystem with the sole capability of attempting to safely shut itself down in order to probe, in an isolated manner, potential vulnerabilities in alignment techniques and then improve them.

Introduction:

The core idea is to construct a tightly constrained shutdown module within a broader... (read 659 more words →)

LESSWRONG
LW

LESSWRONG
LW

Justausername

How does the ever-increasing use of AI in the military for the direct purpose of murdering people affect your p(doom)?

Supplementary Alignment Insights Through a Highly Controlled Shutdown Incentive

Enhancing Corrigibility in AI Systems through Robust Feedback Loops

Embedding Ethical Priors into AI Systems: A Bayesian Approach

Justausername

Justausername

How does the ever-increasing use of AI in the military for the direct purpose of murdering people affect your p(doom)?

Enhancing Corrigibility in AI Systems through Robust Feedback Loops

Embedding Ethical Priors into AI Systems: A Bayesian Approach

Autonomous Alignment Oversight Framework (AAOF)

Supplementary Alignment Insights Through a Highly Controlled Shutdown Incentive

Justausername

How does the ever-increasing use of AI in the military for the direct purpose of murdering people affect your p(doom)?

Supplementary Alignment Insights Through a Highly Controlled Shutdown Incentive

Enhancing Corrigibility in AI Systems through Robust Feedback Loops

Embedding Ethical Priors into AI Systems: A Bayesian Approach

Justausername

Justausername

How does the ever-increasing use of AI in the military for the direct purpose of murdering people affect your p(doom)?

Enhancing Corrigibility in AI Systems through Robust Feedback Loops

Embedding Ethical Priors into AI Systems: A Bayesian Approach

Autonomous Alignment Oversight Framework (AAOF)

Supplementary Alignment Insights Through a Highly Controlled Shutdown Incentive

Abstract

Abstract

Introduction

Abstract:

Introduction: