Capybasilisk

Also note that fundamental variables are not meant to be some kind of “moral speed limits”, prohibiting humans or AIs from acting at certain speeds. Fundamental variables are only needed to figure out what physical things humans can most easily interact with (because those are the objects humans are most likely to care about).

Ok, that clears things up a lot. However, I still worry that if it's at the AI's discretion when and where to sidestep the fundamental variables, we're back at the regular alignment problem. You have to be reasonably certain what the AI is going to do in extremely out of distribution scenarios.

Replying toHalf-baked idea: a straightforward method for learning environmental goals?

Capybasilisk1y

Half-baked idea: a straightforward method for learning environmental goals?

You may be interested in this article:

Model-Based Utility Functions

Orseau and Ring, as well as Dewey, have recently described problems, including self-delusion, with the behavior of agents using various definitions of utility functions. An agent's utility function is defined in terms of the agent's history of interactions with its environment. This paper argues, via two examples, that the behavior problems can be avoided by formulating the utility function in two steps: 1) inferring a model of the environment from interactions, and 2) computing utility as a function of the environment model. Basing a utility function on a model that the agent must learn implies that the utility function must initially be expressed in terms of specifications to be matched to

Capybasilisk1y

Towards shutdownable agents via stochastic choice

Is an AI aligned if it lets you shut it off despite the fact it can foresee extremely negative outcomes for its human handlers if it suddenly ceases running?

I don't think it is.

So funnily enough, every agent that lets you do this is misaligned by default.

Replying toTowards shutdownable agents via stochastic choice

Capybasilisk1y

Towards shutdownable agents via stochastic choice

I'm pointing out the central flaw of corrigibility. If the AGI can see the possible side effects of shutdown far better than humans can (and it will), it should avoid shutdown.

You should turn on an AGI with the assumption you don't get to decide when to turn it off.

-3

Replying toJust because an LLM said it doesn't mean it's true: an illustrative example

Capybasilisk1y

Just because an LLM said it doesn't mean it's true: an illustrative example

According to Claude: green_leaf et al, 2024

[LINKPOST] Agents Need Not Know Their Purpose

Capybasilisk

https://arxiv.org/abs/2402.09734

Ensuring artificial intelligence behaves in such a way that is aligned with human values is commonly referred to as the alignment challenge. Prior work has shown that rational agents, behaving in such a way that maximizes a utility function, will inevitably behave in such a way that is not aligned with human values, especially as their level of intelligence goes up. Prior work has also shown that there is no "one true utility function"; solutions must include a more holistic approach to alignment. This paper describes oblivious agents: agents that are architected in such a way that their effective utility function is an aggregation of a known and hidden sub-functions. The hidden

Capybasilisk

Modern deep learning approaches usually transform inputs into a modality-specific form. For example, the most common deep learning approach to image classification involves decoding image file bytes into an RGB tensor which is passed into a neural network. Instead, we investigate performing classification directly on file bytes, without the need for decoding files at inference time. Using file bytes as model inputs enables the development of models which can operate on multiple input modalities. Our model, \emph{ByteFormer}, achieves an ImageNet Top-1 classification accuracy of 77.33% when training and testing directly on TIFF file bytes using a transformer backbone with configuration similar to DeiT-Ti (72.2% accuracy when operating on RGB images). Without modifications

... (read more)

The Involuntary Pacifists

Capybasilisk

This might be an idea better put to r/WritingPrompts (or ChatGPT), but a scenario has occurred to me which might count either as a near-utopia, or extreme dystopia, depending on your point of view.

Imagine a future Earth where a great power (this can be a superintelligent AI, benevolent alien race, or just straight-up God, depending on your preferences) has imposed a version of the libertarian harm principle on humanity. Most actions a human can engage in today would be available to you, you just wouldn't be able to commit an act of violence against another human. Sure, you might have an extremely strong desire to harm a human, but try as you... (read 354 more words →)

Reward Is Not Necessary: How To Create A Compositional Self-Preserving Agent For Life-Long Learning

Capybasilisk

We introduce a physiological model-based agent as proof-of-principle that it is possible to define a flexible self-preserving system that does not use a reward signal or reward-maximization as an objective. We achieve this by introducing the Self-Preserving Agent (SPA) with a physiological structure where the system can get trapped in an absorbing state if the agent does not solve and execute goal-directed polices. Our agent is defined using new class of Bellman equations called Operator Bellman Equations (OBEs), for encoding jointly non-stationary non-Markovian tasks formalized as a Temporal Goal Markov Decision Process (TGMDP). OBEs produce optimal goal-conditioned spatiotemporal transition operators that map an initial state-time to the final state-times of a policy

... (read more)

The Opposite Of Autism

Capybasilisk

It's probably not presuming too much to guess that many around here have personal experience with the autism spectrum, if not in relation to themselves, then with close family. I say this because the kinds of subjects discussed around here are exactly the type that would appeal to those of an autistic persuasion, e.g. technical systems, logic, and (arguably) utilitarian philosophy. Many here probably have backgrounds in STEM, and those fields tend to have a significant over-representation of people on the spectrum.

An issue that often comes up in software design (a field with high ASD representation) is programmers not being able to properly model the wants and needs of non-technical end-users. I... (read 724 more words →)

Deriving Our World From Small Datasets

Capybasilisk

A suggested solution to AGI safety is to ensure intelligent systems only work on abstract mathematical problems that have, as far as we know, little to no connection with the "real world". I'll sidestep some obvious problems (limited usefulness, temptation to formulate real world problems in mathematical terms) to explore some implications of this idea, and then tie it back to modelling physical reality efficiently.

Ideally, these powerful "math oracles" would be handed datasets related to purely theoretical problems, model the data, and send back some form of domain-relevant answer. These AIs would live solely in a universe of esoteric mathematical objects and their relations. They would neither know nor care about humans... (read 586 more words →)

Shadows Of The Coming Race (1879)

Capybasilisk

I recently came across the short essay below by George Eliot, the male pen name of Victorian author Mary Ann Evans. It's quite remarkable in its prescience of current AI debates, particularly how long the "servant-master" relationship can persist between humans and their machines.

What's striking for me is that I'd read several of her novels such as Middlemarch and Adam Bede, but nothing about these fairly typical Victorian works ever hinted that she was thinking about the kinds of questions raised in this essay.

Shadows Of The Coming Race

My friend Trost, who is no optimist as to the state of the universe hitherto, but is confident that at some future period within the duration of

... (read 1927 more words →)

Leukemia Has Won

Capybasilisk

Has The Function To Sort Posts By Votes Stopped Working?

Capybasilisk

It only seems to work with comments made by a particular user.

LESSWRONG
LW

LESSWRONG
LW

Shadows Of The Coming Race (1879)

[Link Post] Bytes Are All You Need: Transformers Operating Directly On File Bytes

The Involuntary Pacifists

[LINKPOST] Agents Need Not Know Their Purpose

Capybasilisk

Capybasilisk

[LINKPOST] Agents Need Not Know Their Purpose

[Link Post] Bytes Are All You Need: Transformers Operating Directly On File Bytes

The Involuntary Pacifists

Reward Is Not Necessary: How To Create A Compositional Self-Preserving Agent For Life-Long Learning

The Opposite Of Autism

Deriving Our World From Small Datasets

Shadows Of The Coming Race (1879)

Capybasilisk

Shadows Of The Coming Race (1879)

[Link Post] Bytes Are All You Need: Transformers Operating Directly On File Bytes

The Involuntary Pacifists

[LINKPOST] Agents Need Not Know Their Purpose

Capybasilisk

Capybasilisk

[LINKPOST] Agents Need Not Know Their Purpose

[Link Post] Bytes Are All You Need: Transformers Operating Directly On File Bytes

The Involuntary Pacifists

Reward Is Not Necessary: How To Create A Compositional Self-Preserving Agent For Life-Long Learning

The Opposite Of Autism

Deriving Our World From Small Datasets

Shadows Of The Coming Race (1879)

Shadows Of The Coming Race