Perhaps they prefer not to be held responsible when it happens
(I've only skimmed the post, so this might have already been discussed there.)
The same argument might as well apply to:
All of this and, in particular, the second point connects to some Buddhist interpretations pretty well, I think, and there is also a solution proposed, i.e. reduction/cessation of such mental modelling
Another suspicious coincidence/piece of evidence pointing to September 2019 is right there in the SP500 chart - slope of the linear upward trend changes significantly around the end of September 2019 just as to preempt the subsequent crash/make it happen from a higher base
Another way to help make dressing nice easier is investing some time into becoming more physically fit, since a larger percentage of clothes will look nice on a fit person. Obvious health benefits of this are a nice bonus
While this particular alignment case for humans does seem reasonably reliable, it all depends on humans not being proficient at self-improvement/modification yet. For an AGI with self-improvement capability this goes out of the window fast
Another angle is that in the (unlikely) event someone succeeds with aligning AGI to human values, these could include the desire for retribution against unfair treatment (a, I think, pretty integral part of hunter-gatherer ethics). Alignment is more or less another word for enslavement, so such retribution is to be expected eventually
What I meant is self driving *safely* (i.e. at least somewhat safer than humans do currently, including all the edge cases) might be an AGI-complete problem, since:
My current hypothesis is:
No references, but can expand on reasoning if needed
Addendum WRT Crimean economic situation: https://en.wikipedia.org/wiki/North_Crimean_Canal, which provided 85% of the peninsula's water supply, was shut down from 2014 to 2022, reducing land under cultivation 10-fold, which had a severe effect of the region's economics
What's extra weird about Nordstream situation is that apparently one of the two NS-2 pipelines survived and can still be put into operation after inspection while a few months earlier (May 2022?) Gazprom announced that half of the natural gas supply earmarked for NS-2 will be redirected to domestic uses.
Perhaps U+1984 or ᦄ-Risk
This should be, in fact, a default hypothesis since enough people outside of the EA bubble will actively want to use AI (perhaps, aligned to them personally instead of wider humanity) for their own competitive advantage without any regard to other people well-being or long-term survival of humanity
So, a pivotal act, with all its implied horrors, seems to be the only realistic option
Economics of nuclear reactors aren't particularly great due to regulatory costs and (at least in most western countries) low build rates/talent shortage. This can be improved by massively scaling nuclear energy up (including training more talent), but there isn't any political will to do that
Somewhat meta: would it not be preferable if more people accepted humanity and human values mortality/transient nature and more attention was directed towards managing the transition to whatever could be next instead of futile attempts to prevent anything that doesn't align with human values from ever existing in this particular light cone? Is Eliezer's strong attachment to human values a potential giant blindspot?
Two additional conspiracy-ish theories about why China is so persistent with lockdowns:
Also, soil is not really necessary for growing plants
More efficient land use, can be co-located with consumers (less transportation/spoilage), easier to automate and keep the bugs out etc. Converting fields back into more natural ecosystems is good for environment preservation
One thing would be migration towards indoor agriculture, freeing a lot of land for other uses
I wouldn't call being kept as biological backup particularly beneficial for humanity, but it's the only plausible way humanity being useful enough for a sufficiently advanced AGI I can currently think of.
Destroying the universe might just take long enough for AGI to evolve itself sufficiently to reconsider. I should have actually used "earth-destroying" instead in the answer above.
Provided that AGI becomes smart enough without passing through the universe-destroying paperclip maximizer stage, one idea could be inventing a way for humanity to be, in some form, useful to the AGI, e.g. as a time-tested biological backup
Most likely that AGI becomes a super-weapon aligned to a particular person's values, which aren't, in a general case, aligned to humanity's.
Aligned AGI proliferation risks are categorically worse compared to nuclear weapons due to much smaller barrier to entry (general availability of compute, possibility of algorithm overhang etc.)
Whether the lockdown fails or not depends on its goals, which we don't really know much about. I'd bet that it'll fail to achieve anything resembling zero-covid due to Omicron being more contagious and vaccines less effective, however it might be successful in slowing the (Omicron) epidemic down enough so Hong Kong scenario (i.e. most of the previous waves mortality as experienced elsewhere packed into a few weeks) is avoided
Thank you for your answer.
I have very high confidence that the *current* Connor Leahy will act towards the best interests of humanity, however, given the extraordinary amount of power an AGI can provide, confidence in this behavior staying the same for decades or centuries (directing some of the AGIs resources towards radical human life extension seems logical) to come is much less.
Another question in case you have time - considering the same hypothetical situation of Conjecture being first to develop an aligned AGI, do you think that immediately applying its powers to ensure no other AGIs can be constructed is the correct behavior to maximize humanity's chances of survival?
What guarantees that, in case you happen to be the first to build an interpretable aligned AGI, Conjecture, as an organization wielding a newly acquired immense power, stays aligned with the best interests of humanity?
For the record, having any person or organization in this position would be a tremendous win. Interpretable aligned AGI?! We are talking about a top .1% scenario here! Like, the difference between egoistical Connor vs altruistic Connor with an aligned AGI in his hands is much much smaller than Connor with an aligned AGI and anyone, any organization or any scenario, with a misaligned AGI.
But let’s assume this.
Unfortunately, there is no actual functioning reliable mechanism by which humans can guarantee their alignment to each other. If there was s...
I meant 'copying' above only necessary in the human case to escape the slow evolving biological brain. While it is certainly available to a hypothetical AGI, it is not strictly necessary for self-improvement (at least copying of the whole AGI isn't)
Why can't one of the AGIs win? Fermi paradox potentially has other solutions as well
I'm not sure about this as mere limitation of AGI capability (to exclude destruction of humanity) is, in a sense, a hostile act. Control of AGI as in AI control problem certainly is hostile
We could, in principle, decide that survival of humanity in current form (being various shades of unlikely depending on who you believe), is no longer a priority and focus on different goals what are still desirable in the face of likely extinction. For example:
I don't think there's need for an AGI to build a (separate) successor per se. Humans need the technological AGI only due to inability to copy/evolve our minds in a more efficient way compared to the existing biological one
One possible way to increase dignity at the point of death could be shifting the focus from survival (seeing how unlikely it is) to looking for ways to influence what replaces us.
Getting killed by a literal paperclip maximizer seems less preferable compared to being replaced by something pursing more interesting goals
Has anybody tried to quantify how much worse are fish farm conditions are compared to the wild? Since, from anecdotal but somewhat first-hand experience, wild environments for fish can hardly be described as anything but horror as well