At least there was an interesting part reminiscent of Eliezer's Universal Fire:

Eliezer:

Matches catch fire because of phosphorus - "safety matches" have phosphorus on the ignition strip; strike-anywhere matches have phosphorus in the match heads. Phosphorus is highly reactive; pure phosphorus glows in the dark and may spontaneously combust. (Henning Brand, who purified phosphorus in 1669, announced that he had discovered Elemental Fire.) Phosphorus is thus also well-suited to its role in adenosine triphosphate, ATP, your body's chief method of storing chemical energy. ATP is sometimes called the "molecular currency". It invigorates your muscles and charges up your neurons. Almost every metabolic reaction in biology relies on ATP, and therefore on the chemical properties of phosphorus.

If a match stops working, so do you. You can't change just one thing.

The surface-level rules, "Matches catch fire when struck," and "Humans need air to breathe," are not obviously connected. It took centuries to discover the connection, and even then, it still seems like some distant fact learned in school, relevant only to a few specialists. It is all too easy to imagine a world where one surface rule holds, and the other doesn't; to suppress our credence in one belief, but not the other. But that is imagination, not reality. If your map breaks into four pieces for easy storage, it doesn't mean the territory is also broken into disconnected parts. Our minds store different surface-level rules in different compartments, but this does not reflect any division in the laws that govern Nature.

We can take the lesson further. Phosphorus derives its behavior from even deeper laws, electrodynamics and chromodynamics. "Phosphorus" is merely our word for electrons and quarks arranged a certain way. You cannot change the chemical properties of phosphorus without changing the laws governing electrons and quarks.

If you stepped into a world where matches failed to strike, you would cease to exist as organized matter.

Reality is laced together a lot more tightly than humans might like to believe.

From the article:

The team programmed some of the physics for aesthetic reasons. For instance, Duncan insisted on permitting moons to orbit closer to their planets than Newtonian physics would allow. When he desired the possibility of green skies, the team had to redesign the periodic table to create atmospheric particles that would diffract light at just the right wavelength.

“Because it’s a simulation,” Murray stated. “there’s so much you can do. You can break the speed of light—no problem. Speed is just a number. Gravity and its effects are just numbers. It’s our universe, so we get to be Gods in a sense.”

Even Gods though, have their limitations. The game’s interconnectivity means that every action has a consequence. Minor adjustments to the source code can cause mountains to unexpectedly turn into lakes, species to mutate, or objects to lose the property of collision and plummet to the center of a planet. “Something as simple as altering the color of a creature,” Murray noted, “can cause the water level to rise.”

I think most proceduraly generated games aren't that deeply interconnected with regard to their laws of physics.

Video games with procedural generation of the game universe have existed since forever, what's new here?

• "Bayes vs Science": Can you consistently beat the experts in (allegedly) evidence-based fields by applying "rationality"? AI risk and cryonics are specific instances of this issue.

• Can rationality be learned, or is it an essentially innate trait? If it can be learned, can it be taught? If it can be taught, do the "Sequences" and/or CFAR teach it effectively?

But the point is, who is in the wrong between the adopters and the non-adopters?

If the new evidence which is in favor of cryonics benefits causes no increase in adoption, then either there is also new countervailing evidence or changes in cost or non-adopters are the more irrational side. Since I can't think of any body of new research or evidence which should neutralize the many pro-cryonics lines of research over the past several decades, and the costs have remained relatively constant in real terms, that tends to leave the third option.

(Alternatively, I could be wrong about whether non-adopters have updated towards cryonics; I wasn't around for the '60s or '70s, so maybe all the neuroscience and cryopreservation work really has made a dent and people in general are much more favorable towards cryonics than they used to be.)

That's not true. I can think of at least 3 ways in which a society which has demonstrated successful revival could also still need to freeze people:

1. You could die of something that will be curable in a few years and you know to high confidence what you will wake up as because society or revival methods won't change much.
2. The emulation route could wind up being best long before magic nanobots cure all bodily ills, so you must die (so your brain is fixed well enough for slicing & scanning) but you know what you will wake up as almost immediately.
3. There could be treatments or cures but of poor enough efficacy that you rationally prefer the risks of immediate death-then-preservation than try them (you have a fatal disease which can be cured only by a prefrontal lobotomy; alternately, you can go into cryopreservation; which do you prefer?).

A major difference here is that if I sign up for those medical procedures, then I pretty much know what to expect: there is a slight chance that I get cured, and that's it. This is not the case with cryonics. I find it quite likely that cryonics would work, but there's hardly any certainty regarding happens then: I might wake up in just about any form (in a biological body, as an upload) in just about any kind of future society. I would have hardly any control over the outcome whatsoever.

Sure, maybe there would be many more who would sign up, but nevertheless I think it takes a very special kind of person to be ready to take such a leap into the unknown.

Probably not. If you look at the comments on posts about the Prize, you can see how clearly people have already set up their fallback arguments once the soldier of 'possible bad vitrification when scaled up to human brain size' has been knocked down. For example, on HN: https://news.ycombinator.com/item?id=11070528

• 'you may have preserved all the ultrastructure but despite the mechanism of crosslinking, I'm going to argue that all the real important information has been lost'
• 'we already knew that glutaraldehyde does a good job of fixating, this isn't news, it's just a con job looking for some free money'
• 'it irreversibly kills cells by fixing them in place so this is irrelevant'
• 'regardless of how good the scans look, this is just a con job'
• 'what's the big deal, we already know frogs can do this, but what does it have to do with humans; anyway, it's a quack science which we know will never work'

Even if a human brain is stored, successfully scanned, and emulated, the continued existence - nay, majority - of body-identity theorists ensures that there will always be many people who have a bulletproof argument against: 'yeah, maybe there's a perfect copy, but it'll never really be you, it's only a copy waking up'.

More broadly, we can see that there is probably never going to be any 'Sputnik moment' for cryonics, because the adoption curve of paid-up members or cryopreservations is almost eerily linear over the past 50 years and entirely independent of the evidence. Refutation of 'exploding lysosomes' didn't produce any uptick. Long-term viability of ALCOR has not produced any uptick. Discoveries always pointing towards memory being a durable feature of neuronal connections rather than, as so often postulated, an evanescent dynamic property of electrical patterns, have never produced an uptick. Continued pushbacks of 'death' have not produced upticks. No improvement in scanning technology has produced an uptick. Moore's law proceeding for decades has produced no uptick. Revival of rabbit kidney, demonstration of long-term memory continuity in revived C. elegans, improvements in plastination and vitrification - all have not or are not producing any uptick. Adoption is not about evidence.

Even more broadly, if you could convince anyone, how many do you expect to take action? To make such long-term plans on abstract bases for the sake of the future? We live in a world where most people cannot save for retirement and cannot stop becoming obese and diabetic despite knowing full well the highly negative consequences, and where people who have survived near-fatal heart attacks are generally unable to take their medicines and exercise consistently as their doctors keep begging them. And for what? Life sucks, but at least then you get to die. Even after a revival, I would predict that maybe 5% of the USA population (~16m people) would be meaningfully interested in cryonics, and of that only a fraction would go through with it, so 'millions' is an upper bound.

I'm not sure this distinction, while significant, would ensure "millions" of people wouldn't sign up.

Presumably, preserving a human brain "successfully", according to some reasonable definition of the term, would be a big deal and cause a lot of interest in cryonics. It would certainly seem like significant progress towards the sort of life-extension that LW's been clambering about.

Exactly how many new contracts they would get seems hard to predict, but I don't see a number larger than 1,000,000 to be unreasonable.

I'm aware of ELIZA, and of Yvain's post. ELIZA's very shallow, and the interactive setting gives it an easier job than coming up with 1000 words on "why to have goals" or "5 ways to be more productive". I do wonder whether some of the clickbait photo galleries are mechanically generated.

The difference with Markov models is they tend to overfit at that level. At 20 characters deep, you are just copy and pasting large sections of existing code and language. Not generating entirely unseen samples. You can do a similar thing with RNNs, by training them only on one document. They will be able to reproduce that document exactly, but nothing else.

To properly compare with a markov model, you'd need to first tune it so it doesn't overfit. That is, when it's looking at an entirely unseen document, it's guess of what the next character should be is most likely to be correct. The best setting for that is probably only 3-5 characters, not 20. And when you generate from that, the output will be much less legible. (And even that's kind of cheating, since markov models can't give any prediction for sequences it's never seen before.)

Generating samples is just a way to see what patterns the RNN has learned. And while it's far from perfect, it's still pretty impressive. It's learned a lot about syntax, a lot about variable names, a lot about common programming idioms, and it's even learned some English from just code comments.