x

LESSWRONG
LW

christosi

Message

51

2y

Towards shutdownable agents via stochastic choice

We[1] have a new paper testing the POST-Agents Proposal (PAP). The abstract and main-text is below. Appendices are in the linked PDF. Abstract * Some worry that advanced artificial agents may resist being shut down. * The POST-Agents Proposal (PAP) is an idea for ensuring that doesn’t happen. * A...

Jul 8, 202459

christosi

Subscribe

Message

51

2y

Towards shutdownable agents via stochastic choice

We[1] have a new paper testing the POST-Agents Proposal (PAP). The abstract and main-text is below. Appendices are in the linked PDF. Abstract * Some worry that advanced artificial agents may resist being shut down. * The POST-Agents Proposal (PAP) is an idea for ensuring that doesn’t happen. * A...

Jul 8, 202459

Towards shutdownable agents via stochastic choice

Elliott Thornley (EJT)

Elliott Thornley (EJT), alexr, christosi, LAThomson

2y

We^[1] have a new paper testing the POST-Agents Proposal (PAP). The abstract and main-text is below. Appendices are in the linked PDF.

Abstract

Some worry that advanced artificial agents may resist being shut down.
The POST-Agents Proposal (PAP) is an idea for ensuring that doesn’t happen.
A key part of the PAP is using a novel ‘Discounted Reward for Same-Length Trajectories (DReST)’ reward function to train agents to:
1. pursue goals effectively conditional on each trajectory-length (be ‘USEFUL’)
2. choose stochastically between different trajectory-lengths (be ‘NEUTRAL’ about trajectory-lengths).
In this paper, we propose evaluation metrics for USEFULNESS and NEUTRALITY.
We use a DReST reward function to train simple agents to navigate gridworlds, and we find that these agents learn to be USEFUL and

... (read 6827 more words →)

11

59