private_messaging comments on Reply to Holden on 'Tool AI' - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Loading…
Subscribe to RSS Feed
Powered by Reddit
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (320)
I don't see why the 'oracle' has to work from some real world goal in the first place. The oracle may have as it's terminal goal the output of the relevant information on the screen with the level of clutter compatible with human visual cortex, and that's it. Up to you to ask it to represent it in particular way.
Or not even that; the terminal goal of the mathematical system is to make some variables represent such output; an implementation of such system has those variables be computed and copied to the screen as pixels. The resulting system does not even self preserve; the abstract computation making abstract variables have certain abstract values is attained in the relevant sense even if the implementation is physically destroyed. (this is how software currently works)
The screen is a part of the real world.
Well, in the software you simply don't implement the correspondence between mathematical abstractions that you rely on to build software (the equations, real valued numbers) and implementation (electrons in the memory, pixels on display, etc). There's no point in that. If you do you encounter other issues, like wireheading.