Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

A toy model of the treacherous turn

13 Stuart_Armstrong 08 January 2016 12:58PM

Jaan Tallinn has suggested creating a toy model of the various common AI arguments, so that they can be analysed without loaded concepts like "autonomy", "consciousness", or "intentionality". Here a simple attempt for the "treacherous turn"; posted here for comments and suggestions.

Meet agent L. This agent is a reinforcement-based agent, rewarded/motivated by hearts (and some small time penalty each turn it doesn't get a heart):

continue reading »