= 27d567bc2d48d9f40e9ccc20ea370b15)
timtyler comments on Intelligence Explosion vs. Co-operative Explosion - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (61)
I can't meet your "complex value extraction" challenge. I never meant to imply "complete" extraction - just that we can extract value information (like this) and then copy it around with high fidelity. Revealed preferences can be good, but I wouldn't like to get into quantifying their accuracy here.
OK.
I certainly agree that any information we know how to digitally encode in the first place, we can copy around with high fidelity.
But we don't know how to digitally encode our values in the first place, so we don't know how to copy them. That's not because value is some kind of mysterious abstract ethereal "whatness of the if"... we can define it concretely as the stuff that informs, and in principle allows an observer to predict, our revealed preferences... but because it's complicated.
I'm inclined to agree with Wei_Dai that high-fidelity value sharing would represent a significant breakthrough in our understanding of and our ability to engineer human psychology, and would likely be a game-changer.
Well, we do have the idea of revealed preference. Also, if you want to know what people value, you can often try asking them. Between them, these ideas work quite well.
What we can't do is build a machine that optimises them - so there is something missing, but it's mostly not value information. We can't automatically perform inductive inference very well, for one thing.
I suspect I agree with you about what information we can encode today, and you seem to agree with me that there's additional information in our brains (for example, information about berries) that we use to make those judgments which revealed preferences (and to a lesser extent explicitly articulated preferences) report on, which we don't yet know how to encode.
I don't really care whether we call that additional information "value information" or not; I thought initially you were claiming that we could in practice encode it. Thank you for clarifying.
Also agreed that there are operations our brains perform that we don't know how to automate.