= 27d567bc2d48d9f40e9ccc20ea370b15)
benelliott comments on UFAI cannot be the Great Filter - Less Wrong Discussion
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (90)
That may be the case, but the fact that species did not hide is no evidence for it, as species could not realistically have hid even if it would have been optimal
Also, I would like to point out that where the hiding strategy is feasible, on the individual level, it is very common among animals.
In fact, arguably we do have a few cases of species 'hiding', such as the coelocanth, which has gone over a hundred million years without much significant evolution, vastly longer than most species survive.
Basically, I do not see any reason to believe either of the following assertions.
Perhaps I overstated my case. Hiding and expanding are different optimisation targets that pull in pretty different directions. It is challenging to expand while hiding effectively - since farming suns tends to leave a visible thermodynamic signature which is visible from far away and is expensive to eliminate. I expect civilizations will typically strongly prioritize expanding over hiding.
Okay, in that case I now agree with the first part of your claim, I will accept that there is certainly a trade-off, perfect expanding does not involve much hiding and perfect hiding does not involve much expanding.
So, to move on to the other side, why do you expect expanding to be more prevalent than hiding. It seems to me that Oscar Cunningham's argument for why hiding might be preferable is quite convincing, or at any rate reduces it to a non-trivial problem of game theory and risk aversion.
Camouflage is pretty unlikely to be an effective defense against an oncoming colonisation wave. I figure the defense budget will be spent more on growth and weapons than camouflage.
For reasons of technological progress, I suspect a hiding civilisation could destroy an younger expanding civilisation before it was hit by the colonisation wave. If this is the case, it becomes a matter of how likely you are to be the oldest civilisation, how likely the oldest civilisation is to expand or hide, and how much you value survival relative to growth. If the first is low and the last is high, then hiding seems like quite a good strategy.
That sounds like the wave's leading edge to me.
The issues as I see them are different. Much depends on whether progress "maxes out". If it doesn't the most mature civilization probaby just wins - in which case, hiding is irrelevant. If the adversaries are well matched they may attempt to find some other resolution besides a big fight which could weaken both of them. Again, hiding won't help.
IMO, assuming the oldest civilization is in hiding is not a good way to start analysing this issue.
I am confused by this sentence, and cannot parse what it means.
If it knows that its then oldest, then yes it wins. The whole point of Oscar-Cunningham's comment is that it might not know this.
To model it as a simple game, 100 people are all put in separate rooms. One of them is designated as the 'big player', nobody knows who they are including them. Each has two choices expand or hide. If the big player expands then they receive a large pay-off and everyone else gets nothing. If the big player hides, then everyone who hides gets a small pay-off, and everyone who expands gets nothing.
Obviously much depends of the relative size of the large and small pay-offs, but it is not trivially obvious to me that expanding is the optimal strategy here.
Against an equally matched foe, attempts to negotiate are inherently highly risky, if negotiations break down, then one of them may well destroy the other, given the possibility of a first mover advantage, one civilisation may decide to attack if negotiations merely look likely to break down, applying the game theory backwards, we get an extremely volatile situation where as soon as anything ceases to go absolutely perfectly both sides attack. Hiding from an equally matched civilisation may well be much safer than trying to talk to them.
Furthermore, if you become aware of an equally matched civilisation hiding from you, it may be better to continue to pretend you are not aware, rather than opening negotiations straight away. This may go to rather high levels of I know You know I know and as long a mutual knowledge isn't attained both can survive.
The more realistic version of that game always has expanding, since we know the total payout on expanding is greater than the total on hiding, and the big player is allowed to share the resources equally if she wants to.
I fail to see that this carries.
We modify the game to make sure that the pay-off for successful expanding exceeds all the hiding pay-offs put together, and we also allow the big player, after the fact, to share their expanding pay-out if they want to.
Clearly, not sharing produces a higher pay-off than sharing, so the big-player will not do this. Negotiating in advance doesn't work, as it requires revealing yourself, so once you've done that you've made your move and its no longer "in advance".
If a civilisations utility is linear in its size, then it is always wise for it to expand. If it is risk-averse, which seems plausible, (most of us would not accept a plan which had a 50% chance of colonising Mars and a 50% chance of wiping out humanity), then it may still be wise to hide. If all civilisations are risk averse, hiding is a Nash equilibrium.
The total payout must be higher because in the hiding scenario a lot of negentropy is wasted into nowhere, in the natural lifecycle of stars and the like. The universe is a pie of a fixed size, but one that gradually rots away you take to long deciding who gets to eat it.
And it might be the case that nobody in fact chose to share, but due to game theory it still matters that they had the option.
Also, things like TDT allow for coordination even while hiding, and in fact seems to be one of the assumptions behind this thing in the first place.