= 27d567bc2d48d9f40e9ccc20ea370b15)
Dmytry comments on AI Risk and Opportunity: Humanity's Efforts So Far - Less Wrong Discussion
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (47)
If the dog chews something really expensive up, there is no point punishing the dog proportionally more for that. That would be wrong; some level of punishment is optimal for training; beyond this is just letting anger out.
It's not mere pruning. You need a person to be able to feed your pets, you need them to get through the door, you need a key, you can get a key at key duplicating place, you go to key duplicating place you know of to make a duplicate.
That stops them from being usefully modelled as 'choose action that gives maximum utility'. You can't assume that it makes action that results in maximum utility. You can say that it makes action which results in as much utility as this agent with its limitations could get out of that situation, but that's almost tautological at this point. Also, see
http://en.wikipedia.org/wiki/Intelligent_agent
for terminology.
Well, the utility agent as per wiki article, is clearly stupid because it won't reason backwards. And the utility maximizers discussed by purely theoretical AI researchers, likewise.
Needs something better than trying all models and seeing what fits, though. One should ideally be able to use the normal reasoning to improve models. It feels that a better model has bigger utility.
You would probably want to let the dog know that some of your chewable things are really expensive. You might also want to tell it about the variance in the value of your chewable items. I'm sure there are some cases where the owner might want to manipulate the dog by giving it misleading reward signals - but honest signals are often best.
These are the researchers who presume no computational resource limitation? They have no need to use optimisation heuristics - such as the ones you are proposing - they assume unlimited computing resources.
Needs something better than trying all models and seeing what fits, though. One should ideally be able to use the normal reasoning to improve models. [...]
Sure. Humans use "normal reasoning" to improve their world models.
Well, i don't think that quite works, dogs aren't terribly clever. Back to humans, e.g. significant injuries hurt a lot less than you'd think they would, my guess is that small self inflicted ones hurt so much for effective conditioning.
The ugly is when some go on and talk about the AGIs certainly killing everyone unless designed in some way that isn't going to work. And otherwise paint wrong pictures of AGI.
Ya. Sometimes even resulting in breakage, when they modify world models to fit with some pre-existing guess.
The idea and its explanation both seem pretty speculative to me.
Pavlovian conditioning is settled science; the pain being negative utility value for intelligence etc, not so much.
The "idea" was:
...and its explanation was:
I'm inclined towards scepticism - significant injuries often hurt a considerable amount - and small ones do not hurt by disproportionally large amounts - at least as far as I know.
There do seem to be some ceiling-llike effects - to try and prevent people passing out and generally going wrong. I don't think that is to do with your hypothesis.
The very fact that you can pass out from pain and otherwise the pain interfering with thought and actions, implies that the pain doesn't work remotely like utility should. Of course one does factor in pain into the utility, but that is potentially dangerous for survival (as you may e.g. have to cut your hand off when its stuck under boulder and you already determined that cutting the hand off is the best means of survival). You can expect interference along the lines of passing out from the network training process. You can't expect interference from utility values being calculated.
edit:
Okay for the middle ground: would you agree that pain has Pavlovian conditioning role? The brain also assigns it negative utility, but the pain itself isn't utility, it evolved long before brains could think very well. And in principle you'd be better off assigning utility to lasting damage rather than to pain (and most people do at least try).
edit: that is to say, removing your own appendix got to be easy (for surgeons) if pain was just utility, properly summed with other utilities, making you overall happy that you got the knife for the procedure and can save yourself, through the entire process. It'd be like giving up an item worth $10 for $10 000 000 . There the values are properly summed first, not making you feel the loss and feel the gain separately.
You don't think consciousness should be sacrificed - no matter what the degree of damage - in an intelligently designed machine? Nature sacrifices consciousness under a variety of circumstances. Can you defend your intuition about this issue? Why is nature wrong to permit fainting and passing out from excessive pain?
Of course pain should really hurt. It is supposed to distract you and encourage you to deal with it. Creatures in which pain didn't really, really hurt are likely to have have left fewer descendants.
Well, in so much as the intelligence is not distracted and can opt to sit still play dead, there doesn't seem to be a point in fainting. Any time i have somewhat notable injury (falling off bike, ripping the chin, and getting nasty case of road rash), the pain is less than pain of minor injuries.
Contrast the anticipation of pain with actual pain. Those feel very different. Maybe it is fair to say that the pain is instrumental in creating anticipation of pain, which acts more like utility for intelligent agent. It also serves as a warning signal, and for conditioning, and generally as something that stops you from eating yourself. (and perhaps for telling the intelligence what is and isn't your body). The pain is supposed to encourage you to deal with the damage, but not to distract you from dealing with the damage.
I don't pretend to know exactly why nature does it - but I expect there's a reason. It mat be that sometimes being conscious is actively bad. This is one of the reasons for administering anaesthetics - there are cases where a conscious individual in a lot of pain willl ineffectually flail around and get themselves into worse trouble - where they would be better off being quiet and still - "playing dead".
As to why not "play dead" while remaining conscious - that's a bit like having two "off" switches. There's already an off switch. Building a second one that bypasses all the usual responses of the conscious mind while remaining conscious could be expensive. Perhaps not ideal for a rarely-used feature.