10 bits!!! That's not much of a message!
10 bits short of the needed message, not a 10-bit message. I mean that e.g. an approximation gives 100 bits when full accuracy would be 110 bits (and 10 bits is an upper bound).
The idea of a shared environment arises because the proposed machine - in which the human-like values are to be implemented - is to live in the same world as the human. So, one does not need to specify all the details of the environment - since these are shared naturally between the agents in question.
That still doesn't answer my point; it just shows how once you have one agent, adding others is easy. It doesn't show how getting the first, or the "general" agent is easy.
Re: "That still doesn't answer my point; it just shows how once you have one agent, adding others is easy. It doesn't show how getting the first, or the "general" agent is easy."
To specify the environment, choose the universe, galaxy, star, planet, lattiude, longitude and time. I am not pretending that information is simple, just that it is already there, if your project is building an intelligent agent.
This thread is for the discussion of Less Wrong topics that have not appeared in recent posts. If a discussion gets unwieldy, celebrate by turning it into a top-level post.