= 27d567bc2d48d9f40e9ccc20ea370b15)
Peter_de_Blanc comments on Complexity of Value ≠ Complexity of Outcome - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (198)
I think your term "God's utility function" is a bit confusing - as if it's just one utility function. If you value your genes, and I value my genes, and our genes are different, then we have different utility functions.
Also, the vast majority of possible minds don't have genes.
Maybe. Though if you look at:
http://originoflife.net/gods_utility_function/
...then first of all the term is borrowed/inherited from:
http://en.wikipedia.org/wiki/God%27s_utility_function
...and also, I do mean it in a broader sense where (hopefully) it makes a bit more sense.
The concept is also referred to as "Goal system zero" - which I don't like much.
My latest name for the idea is "Shiva's goals" / "Shiva's values" - a reference to the Hindu god of destruction, creation and transformation.
By "gene" I mean:
"Small chunk of heritable information"
http://alife.co.uk/essays/informational_genetics/
Any sufficiently long-term persistent structure persists via a copying process - and so has "genes" in this sense.
What we mean by preference. Except that preference, being a specification of a computation, has a lot of forms of expression, so it doesn't "persist" by a copying process, it "persists" as a nontrivial computational process.
A superintelligence that persists in copying a given piece of information is running a preference (computational process) that specifies copying as the preferable form of expression, over all the other things it could be doing.
No, no! Genes is just intended to refer to any heritable information. Preferences are something else entirely. Agents can have preferences which aren't inherited - and not everything that gets inherited is a preference.
Anything information that persists over long periods of time persists via copying.
"Copying" just means there's Shannon-mutual information between the source and the destination which originated in the source. Complex computations are absolutely included - provided that they share this property.
Then preference still qualifies. This holds as a factual claim provided we are talking about reflectively consistent agents (i.e. those that succeed in not losing their preference), and as a normative claim regardless.
I would appreciate it if you avoid redefining words into highly qualified meanings, like "gene" for "anything that gets copied", and then "copying" for "any computation process that preserves mutual information".
Re: Then preference still qualifies. This holds as a factual claim provided [bunch of conditions]
Yes, there are some circumstances under which preferences are coded genetically and reliably inherited. However, your claim was stronger. You said what meant by genes was what "we" would call preferences. That implies that genes are preferences and preferences are genes.
You have just argued that a subset of preferences can be genetically coded - and I would agree with that. However, you have yet to argue that everything that is inherited is a preference.
I think you are barking up the wrong tree here - the concepts of preferences and genes are just too different. For example, clippy likes paperclips, in addition to the propagation of paperclip-construction instructions. The physical paperclips are best seen as phenotype - not genotype.
Re: "I would appreciate it if you avoid redefining words into highly qualified meanings [...]"
I am just saying what I mean - so as to be clear.
If you don't want me to use the words "copy" and "gene" for those concepts - then you are out of luck - unless you have a compelling case to make for better terminology. My choice of words in both cases is pretty carefully considered.
Not "bunch of conditions". Reflective consistency is the same concept as "correctly copying preference", if I read your sense of "copying" correctly, and given that preference is not just "thing to be copied", but also plays the appropriate role in decision-making (wording in the grandparent comment improved). And reflectively consistent agents are taken as a natural and desirable (from the point of view of those agents) attractor where all agents tend to end up, so it's not just an arbitrary category of agents.
But there are many different preferences for different agents, just as there are different genes. Using the word "genes" in the context where both human preference and evolution are salient is misleading, because human genes, even if we take them as corresponding to a certain preference, don't reflect human preference, and are not copied in the same sense human preference is copied. Human genes are exactly the thing that currently persists by vanilla "copying", not by any reversible (mutual information-preserving) process.
Confusing terminology is still bad even if you failed to think up a better alternative.