What you are observing is part of the phenomenon of meta-contrarianism. Like everything Yvain writes, the aforementioned post is well worth a read.
Hmm. To me it seemed intuitively clear that the function would be monotonic.
In retrospect, this monotonicity assumption may have been unjustified. I'll have to think more about what sort of curve this function follows.
or they could even restrict options to typical government spending.
JoshuaFox noted that the government might tack on such restrictions
That said, it's not so clear where the borders of such restrictions would be. Obviously you could choose to allocate the money to the big budget items, like healthcare or the military. But there are many smaller things that the government also pays for.
For example, the government maintains parks. Under this scheme, could I use my tax money to pay for the improvement of the park next to my house? After all, it's one of the many things that tax money often works towards. But if you answer affirmatively, then what if I... (read more)
Even formalisms like AIXI have mechanisms for long-term planning, and it is doubtful that any AI built will be merely a local optimiser that ignores what will happen in the future.
As soon as it cares about the future, the future is a part of the AI's goal system, and the AI will want to optimize over it as well. You can make many guesses about how future AI's will behave, but I see no reason to suspect it would be small-minded and short-sighted.
You call this trait of planning for the future "consciousness", but this isn't anywhere near the definition most people use. Call it by any other name, and it becomes clear that it is a property that any well designed AI (or any arbitrary AI with a reasonable goal system, even one as simple as AIXI) will have.
No, no, no: He didn't say that you don't have permission if you don't steal it, only that you do have permission if you do.
What you said is true: If you take it without permission, that's stealing, so you have permission, which means that you didn't steal it.
However, your argument falls apart at the next step, the one you dismissed with a simple "etc." The fact that you didn't steal it in no way invalidates your permission, as stealing => permission, not stealing <=> permission, and thus it is not necessarily the case that ~stealing => ~permission.
You could use some sort of cloud service: for example, Dropbox. One of the main ideas behind of Dropbox was to have a way for multiple people to easily edit stuff collaboratively. It has a very easy user interface for such things (just keep the deck in a synced folder), and you can do it even without all the technical fiddling you'd need for git.
By observing the lack of an unusual amount of paperclips in the world which Skynet inhabits.
I have some rambling thoughts on the subject. I just hope they aren't too stupid or obvious ;-)
Let's take as a framework the aforementioned example of the last digit of the zillionth prime. We'll say that the agent will be rewarded if it gets it right, on, shall we say, a log scoring rule. This means that the agent is incentivised to give the best (most accurate) probabilities it can, given the information it has. The more unreasonably confident it is, the more it loses, and the same with underconfidence.
By the way, for now I will assume the agent fully knows the scoring rule it will be judges by. It is quite... (read more)
If a comment has 100% upvotes, then obviously the amount of upvotes it got is exactly equal to the karma score of the post in question.
Finding a good decision theory is hard. Previous attempts, such as Timeless Decision Theory, work, it seems, in providing a stable, effective decision theory, but are mathematically complicated. Simpler theories, like CDT or EDT, are much more intuitive, but have deep flaws. They fail at certain problems, and thus violate the maxim that rational agents should win. This makes them imperfect.
But it seems to me that there is a relatively simple fix one could make to them, in the style of TDT, to extend their power considerably. Here I will show an implementation of such an extension of CDT, that wins on the problems that classic CDT fails on. It quite possibly... (read 1218 more words →)
Stupid mathematical nitpick:
Actually, it is more correct to say that .95 ^ 39 = 0.14.
If we calculate it out to a few more decimal places, we see that .95 ^ 39 is ~0.135275954. This is closer to 0.14 than to 0.13, and the mathematical convention is to round accordingly.