Posts
Wikitag Contributions
As I commented on your other post, there are two possibilities; either the universe is murphy-like and pessimizing your outcome, in which case sure, you might be in a worst case universe, and there is a bound on how well you can do, or some agent sent the rock, in which case you are playing the game against that agent, and would know that fact. Or, as I mentioned, yes, an ultrapowerful system can create situations where you lose because it can fool you, lying to you perfectly, and that is equivalent to a worst-case universe.
1As I commented on another post, It seems Eliezer already addressed the specific claim you made here via probabilistic LDT solutions, as Mikhail explained there, and in a comment here. (And the quoted solution was written before you wrote this post.)
Is there a version that the modification explains there fails to address?
I think your post misses the point made here.
What about a rock with $9 painted on it? The LDT agent in the problem reasons that the best action is to choose $1, so the rock gets $9.
Thus, $9 rock is more rational than LDT in this problem.
The solution above addresses this; by playing probabilistically, the rock gets a payoff somewhat less than $5 in expectation, so it does worse than an LDT agent.
I'm a bit confused how this is a problem.
Either there is an agent that stands to benefit from my acceding to a threat, or there is not. If an agent "sufficiently" turns itself into a rock for a single interaction, but reaps the benefit as an agent, it's a full-fledged agent. Same if it sends a minion, where the relevant agent is the one who sent the rock, not the rock. And if we have uncertainty about the situation, that's part of the game.
If the question is whether other players can deceive you about the nature of the game or the probabilities, sure, that is a possibility, but it is not really a question about LDT, it's just a question about whether we should expand every decision into a recursive web of uncertainties about all other possible agents - and, I suspect, come to the conclusion that smarter agents can likely fool you, and you shouldn't allow others with misaligned incentives to manipulate your information environment, especially if they have more optimization power than you do. But as we all should know, once we make misaligned super-intelligent systems, we stop being meaningful players anyways.
In this world, maybe you want to suppose the agent's terminal value is to cause me to pay some fixed cost, and it permanently disables itself to that end - but that makes it either a minion sent by something else, or a natural feature of a Murphy-like universe where you started out screwed, in which case you should treat the natural environment as an adversary. But that's not our situation, again, at least until ASI shows up.
cc: @Mikhail Samin - does that seem right to you?
1This article seems good, but this seems not to be the right place to post it, given that those here already are generally aware of this.
8 hours of clock time for an expert seems likely to be enough to do anything humans can do; people rarely productively work in longer chunks than that, and as long as we assume models are capable of task breakdown and planning, (which seems like a non trivial issue, but an easier one than the scaling itself,) that should allow it to parallelize and serialize chucks to do larger human-type tasks.
But it's unclear alignment can be solved by humans at all, and even if it can, of course, there is no reason to think these capabilities would scale as well or better for alignment than for capabilities and self-improvement, so this is not at all reassuring to me.
I mostly agree with you... but:
1. You created a false dichotomy where budgeting excludes investing, then said you can't ever make money with budgeting, because by definition anything that is in your budget cannot make money - but spending on a house instead of rent, for example, clearly violates that assumption.
2. Budgeting can include time, in addition to money, and among other things, that matters because income isn't a fixed quantity over time. Things that people can do to use time to drastically change income include starting businesses, or taking night classes to get a more lucrative job.
3. The last point is conflating questions, because yes, inheriting or a trust fund is already being rich (but inheritances and trust funds are not the same thing!) However, most second and third generation nouveau riche folks do, in fact, spend down and waste the inherited fortune, once they are old enough to actually inherit instead of living on a managed trust fund.
You interpreted this as defending boots theory, which wasn't my intent. I said that there was a real phenomena, not that boots theory is correct.
And sure, rent can come out ahead for some cases, that doesn't imply it's always better, or that upper middle class people generally actually come out ahead - because even where you could end up ahead renting, in fact, the money saved is often spent instead of invested.
Also, I think the claimed non-sequitur isn't one - lots of passive income routes don't require much financial investment, and many that do, like starting a company, can be financed with loans. The point is that people choose to invest their limited time and money in ways that do not build wealth. (Which isn't a criticism - there are plenty of other, better, goals in life.)
"More rational in a given case" isn't more rational! You might as well say it's more rational to buy a given lottery ticket because it's the winning ticket.