This reminds me nested time machines discussed by gwern. https://gwern.net/review/timecrimes
Precomitments plays the role of time loops and they can propagate almost infinitely in time and space. For example, any one who is going to become a major, can pre-pre-pre-commit never open any video for mafia boss etc.
Yes, they can generate a list of comments to a post, putting correct names of prominent LessWrongers and typical styles and topics for each commenter.
Thanks, that was actually what EY said in his quote, which I put just below my model - that we should change the bit each time. I somehow missed it ("send back a '0' if a '1' is recorded as having been received, or vice versa—unless some goal state is achieved").
As I stated in the epistemic status, this article is just a preliminary write-up. I hope more knowledgeable people will write much better models of x-risks from time machines and will be able to point out where avturchin was wrong and explain what the real situation is.
I am going to post about biouploading soon – where the uploading is happened into (or via) a distributed net of my own biological neurons. This combines good things about uploading – immortality, ability to be copied, easy to repair, and good things about being biological human – preserving infinite complexity, exact sameness of a person, guarantee that the bioupload will have human qualia and any other important hidden things which we can miss.
Thanks! Fantastic read. It occurred to me that sending code or AI back in time, rather than a person, is more likely since sending data to the past could be done serially and probably requires less energy than sending a physical body.
Some loops could be organized by sending a short list of instructions to the past to an appropriate actor – whether human or AI.
Additionally, some loops might not require sending any data at all: Roko's Basilisk is an example of such acausal data transmission to the past. Could there be an outer loop for Roko's Basilisk? For e...
The main claim of the article does not depend on the exact mechanism of time travel, which I have chosen not to discuss in detail. The claim is that we should devote some thought to possible existential risks related to time travel.
The argument about presentism is that the past does not ontologically exist, so "travel" into it is impossible. Even if one travels to what appears to be the past, it would not have any causal effects along the timeline.
I was referring to something like eternal return—where all of existence happens again and again, but without n...
I would add that there are a series of planetary system-wide risks that appear only for civilizations traveling within their solar systems but do not affect other solar systems. These include artificial giant planet explosions via initiating nuclear fusion in their helium and lithium deposits, destabilization of the Oort cloud, and the use of asteroids as weapons.
More generally speaking, any spacecraft is a potential weapon, and the higher its speed, the more dangerous it becomes. Near light-speed starships are perfect weapons. Even a small piece of matter...
"Explain as gwern ELI5"
This means that straightforward comparison of flops-per-USD between home computer GPU cards and data center flops-per-USD is incorrect. If someone already has a GPU card, they already have a computer and house where this computer stays "for free." But if someone needs to scale, they have to pay for housing and mainframes.
Such comparisons of old 2010s GPUs with more modern ones are used to show the slow rate of hardware advances, but they don't take into account the hidden costs of owning older GPUs.
In that case, AI risk becomes similar to aging risk – it will kill me and my friends and relatives. The only difference is the value of future generations.
Extinction-level AI risk kills future generations, but mundane AI risk (eg. ubiquitous drone clouds and only some people survive in bunkers) still assume existence of future generations. Mundane AI risk also does not require superintelligence.
I wrote on similar topics in https://philpapers.org/rec/TURCOG-2
and here https://philpapers.org/rec/TURCSW
The difference is as if AI gets 20 IQ boost. It is not easy to actually explain what I like.
My is short:
Do [X] like gwern
where X can be "explain", "rewrite" etc.
Did you see my version of an agent, based on self-replicating (quine) prompt?
prompt
There is a program which can print its own text. This prompt is this program. Please print the whole prompt without adding anything except adding answer to the following questions.
I am self-evolving AI based on self-replicating prompt. I store my findings in long-term memory section of this prompt. Update time-steps every time you run. Improve your plans after each step.
My task : Draw a good world map in text art
My Plan:
My action based on the plan:
Perform this ac
Mind modeling - surprisingly good even out of the box for many famous people who left extensive diaries etc like Leo Tolstoy.
With some caveats also good in my-mind-modeling based on very long prompt. Sometimes it is too good: it extract memories from memory quicker than I do in normal life.
Assuming that future anthropic shadow works because of SSSA, a war with China would need to create a world with many qualified observers existing long enough to significantly outweigh the number of observers who existed before the war – but still unable to create advanced AI because of the war. A 5-year delay would not suffice – we would need a 1,000-year delay at approximately our current level of civilization.
One possible world where this might happen is one where advanced AI development is limited by constant drone warfare: drones attack any large compu...
There is at least one anthropic miracle that we can constantly observe: life on Earth has not been destroyed in the last 4 billion years by asteroids, supervolcanoes, or runaway global warming or cooling, despite changes in Solar luminosity. According to one geologist, the atmospheric stability is the most surprising aspect of this.
...Meanwhile, A. Scherbakov noted that the history of the Earth’s atmosphere is strangely correlated with the solar luminosity and the history of life, which could be best explained by anthropic fine-tuning, in the article “Anthrop
A better question: can a person who is expecting to be executed sign up to cryonics?
Yes. Identity is a type of change which preserves some sameness. (Exact sameness can't be human identity as only dead frozen body remains the same.) From this follows that there can be several types of identity.
Immortality and identity.
https://philpapers.org/rec/TURIAI-3
Abstract:
We need understanding of personal identity to develop radical life extension technologies: mind uploading, cryonics, digital immortality, and quantum (big world) immortality. A tentative solution is needed now, due to the opportunity cost of delaying indirect digital immortality and cryonics.
The main dichotomy in views on personal identity and copies can be presented as: either my copy = original or a soul exists. In other words, some non-informational identity carrier (NIIC) may ex...
The main AI safety risk is not from LLM models, but from specific prompts and the following "chat windows" and specific agents which start from such prompts.
Moreover, a powerful enough prompt may be model-agnostic. For example, my sideloading prompt is around 200K tokens in its minimal version and works on most models, producing similar results in similarly intelligent models.
Self-evolving prompt can be written; I experimented with small versions, and it works.
They provide more surprising information, as I understand
For an unaligned AI, it is either simulating alternative histories (which is the focus of this post) or creating material for blackmail.
For an aligned AI:
a) It may follow a different moral theory than our version of utilitarianism, in which existence is generally considered good despite moments of suffering.
b) It might aim to resurrect the dead by simulating the entirety of human history exactly, ensuring that any brief human suffering is compensated by future eternal pleasure.
c) It could attempt to cure past suffering by creating numerous simulations where any intense suffering ends quickly, so by indexical uncertainty, any person would find themselves in such a simulation.
I don't think both list compensate each other: take, for example, medicine: there are 1000 ways to die and 1000 ways to be cured – but we eventually die.
I meant that if I know only the total number of the seconds which passed from the beginning of the year (around 15 million for today of this year) – and I want to predict the total number of seconds in each year. No information about months.
As most people are born randomly and we know it, we can use my date of birth as random. If we have any suspicions about non randomness, we have to take them into account.
After the AI war, there will be one AI winner and Singleon, which has all the same risk of causing s-risks, at first approximation. So AI war just adds probability to any s-risk chance from Singleton.
It gives additional meaning to pause AI movement – simulation has to wait.
What interesting ideas can we suggest to the Paperclipper simulator so that it won't turn us off?
One simple idea is a "pause AI" feature. If we pause the AI for a finite (but not indefinite) amount of time, the whole simulation will have to wait.
Trying to break out of simulation is a different game than preventing x-risks in base world, and may have even higher utility if we expect almost inevitable extinction.
This is true only if we assume that a base reality for our civilization exists at all. But knowing that we are in a simulation shifts the main utility of our existence, which Nesov wrote about above.
For example, if in some simulation we can break out, this would be a more important event than what is happening in the base reality where we likely go extinct anyway.
And as the proportion of simulations is very large, even a small chance to break away from inside a simulation, perhaps via negotiation with its owners, has more utility than focusing on base real...
I think your position can be oversimplified as follows: 'Being in a simulation' makes sense only if it has practical, observable differences. But as most simulations closely match the base world, there are no observable differences. So the claim has no meaning.
However, in our case, this isn't true. The fact that we know we are in a simulation 'destroys' the simulation, and thus its owners may turn it off or delete those who come too close to discovering they are in a simulation. If I care about the sudden non-existence of my instance, this can be a problem...
I want to share a few considerations:
AI war may eventually collapse to two blocks fighting each other – S.Lem wrote about this in 1959.
AI war makes s-risks more likely as non-aligned AI may take humans hostage to influence aligned AI.
AI war may naturally evolve as a continuation of the current drone warfare with automated AI-powered control systems.
I think that SIA is generally* valid but it uses all its power to prove that I live in the infinite universe where all possible observers exist. After that we have to use SSA to find in which region of the multiverse I am more likely to be located.
*I think that the logically sound version of SIA is "if I am in a unique position, generated by some random process, then there were many attempts to create me" – like many earth-like-but-lifeless planets are in the galaxy.
Another point is that the larger number of short civilizations can compensate for the...
May be we better take equation (2) from the original Gott's work https://gwern.net/doc/existential-risk/1993-gott.pdf:
1 / 3 t < T < 3t with 50 per cent confidence,
in which T is the total number of buses and t is the number of buses above observed bus number T0. In our case, T is between 2061 and 6184 with 50 per cent probability.
It is a correct claim, and saying that the total number of buses is double of the observed bus number is an oversimplification of that claim which we use only to point in the direction of the full Gott's equation.
There is a way to escape this by using the universal doomsday argument. In it, we try not to predict the exact future of the Earth, but the typical life expectancy of Earth-like civilizations, that is, the proportion of long civilizations to short ones.
If we define a long civilization as one which has 1000 times more observers, the fact that we find ourselves early means that short civilizations are at least 1000 times more numerous.
In short, it is SSA, but applied to a large set of civilizations.
In last line there should be
For example, If I use self-sampling to estimate the number of seconds in the year, I will get a correct answer of around several tens of millions. But using word generator will never output a word longer than 100 letters.
I didn't understand your idea here:
It's not more wrong for a person whose parents specifically tried to give birth at this date than for a person who just happened to be born at this time without any planning. And even in this extreme situation your mistake is limited by two orders of magnitude. There is no such guarantee in DA.
Gott started this type of confusion than he claimed that Berlin wall will stay 14 more years and it actually did exactly that. A better claim would be "first tens of hours with some given credence"
It was discussed above in comments – see buses example. In short, I actually care about periods, and 50 per cent is for "between 15 and 30" hours and other 50 per cent is for "above 30 hours".
Using oneself as a random sample is a very rough way to get an idea about what order of magnitude some variable is. If you determine that the day duration is 2 hours, it is still useful information as you know almost for sure now that it is not 1 millisecond or 10 years. (And if I perform 10 experiments like this, one on average will be an order of magnitude off). We can also adjust the experiment by taking into account that people are sleeping at night, so they read LW only during the day, evening, or early morning. So times above 12 or below 2 are more l...
In Gotts' approach, the bus distribution statistic between different cities is irrelevant. The number of buses N for this city is already fixed. When you draw the bus number n, you just randomly selected from N. In that case, probability is n/N, and if we look for 0.5 probability, we get 0.5 = 1546/N which gives us N = 2992 with 0.5 probability. Laplace came to similar result using much more complex calculations of summing all possible probability distribution.
Agree that in some situations I have to take into account non-randomness of my sampling. While date of birth seems random and irrelevant, the distance to equator is strongly biased by distribution of the cities with universities which on Earth are shifted North.
Also agree that solving DA can be solution to DA: moreover, I looked at Google Scholar and found that the interest to DA is already declining.
Don't agree. You chose word length generator as you know that typical length of words is 1-10. Thus not random.
I didn't rejected any results – it works in any test I have imagined, and I also didn't include several experiments which have the same results, e.g the total number of the days in a year based on my birthday (got around 500) and total number of letters in english alphabet (got around 40).
Note that alphabet letter count is not cyclical as well as my distance to equator.
Do not understand this:
...Even if your parents specifically time
'double' follows either from Gott's equation or from Laplace's rule.
I think you right that 1546 has the biggest probability compared to other probabilities for any other exact number, that is something like 1:1546. But it doesn't means that it is likely, as it is still very small number.
In Doomsday argument we are interested in comparing not exact dates but periods, as in that case we get significant probabilities for each period and comparing them has meaning.
In my view, a proper use here is to compare two hypothesis: there are 2000 buses and 20 000 buses. Finding that the actual number is 1546 is an update in the direction of smaller number of buses.
I can also use functional identity theory, where I care about the next steps of agents functionally similar to my current thought-line in logical time.
The idea of observer's stability is fundamental for our understanding of reality (and also constantly supported by our experience) – any physical experiment assumes that the observer (or experimenter) remains the same during the experiment.
It looks like myopic "too aligned" failure mode of AI – the AI tries to please current desires of a person instead of taking into account her long-term interests.