= 27d567bc2d48d9f40e9ccc20ea370b15)
linkhyrule5 comments on The Empty White Room: Surreal Utilities - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (124)
*shrug* I did list that as a separate tier. Surreal Utilities are meant to be a way to formalize tiers; the actual result of the utility-computation depends on where you put your tiers.
The point of this post is to show that humans really do have tiers, and surreals do a good job of representing tiers; the question of how to assign utilities is an open one.
How do you know humans have tiers? The situation has never come up before. We've never had the infinite coincidence where the value at the highest tier is zero.
Also, why does it matter? It's never going to come up either. If you program an AI to have tiers, it will quickly optimize that out. Why waste processing power on lower tiers if it has a chance of helping with the higher ones?
See: gedankenexperiment. I can guess what I'd choose given a blank white room.
And that is a flaw in the system. But it's one that real-valued utility systems have as well. See: Pascal's Mugging. An AI vulnerable to Pascal's Mugging will just spend all its time breaking free of a hypothetical Matrix.
I did mention this under Flaws, you know...