LESSWRONG
LW

All of dankrad's Comments + Replies

Making Nanobots isn't a one-shot process, even for an artificial superintelligance
dankrad10

Thanks for engaging with my post. From my perspective you seem simply very optimistic on what kind of data can be extracted from unspecific measurements. Here is another good example on how Eliezer makes some pretty out there claims about what might be possible to infer from very little data: https://www.lesswrong.com/posts/ALsuxpdqeTXwgEJeZ/could-a-superintelligence-deduce-general-relativity-from-a -- I wonder what your intuition says about this?

But maybe your intuitions are wrong (or maybe both). I think a desirable property of plans/strategies for align

... (read more)
Reply
3Tor Økland Barstad
Likewise  :) Also, sorry about the length of this reply. As the adage goes: "If I had more time, I would have written a shorter letter." That seems to be one of the relevant differences between us. Although I don't think it is the only difference that causes us to see things differently. Other differences (I guess some of these overlap): * It seems I have higher error-bars than you on the question we are discussing now. You seem more comfortable taking the availability heuristic (if you can think of approaches for how something can be done) as conclusive evidence. * Compared to me, it seems that you see experimentation as more inseparably linked with needing to build extensive infrastructure / having access to labs, and spending lots of serial time (with much back-and-fourth). * You seem more pessimistic about the impressiveness/reliability of engineering that can be achieved by a superintelligence that lacks knowledge/data about lots of stuff. * The probability of having a single plan work, and having one of several plans (carried out in parallel) work, seems to be more linked in your mind than mine. * You seem more dismissive than me of conclusions maybe being possible to reach from first-principles thinking (about how universes might work). * I seem to be more optimistic about approaches to thinking that are akin to (a more efficient version of) "think of lots of ways the universe might work, do Montecarlo-simulations for how those conjectures would affect the probability of lots of aspects of lots of different observations, and take notice if some theories about the universe seem unusually consistent with the data we see". * I wonder if you maybe think of computability in a different way from me. Like, you may think that it's computationally intractable to predict the properties of complex molecules based on knowledge of the standard model / quantum physics. And my perspective would be that this is extremely contingent on the molecule, what the AI nee
Making Nanobots isn't a one-shot process, even for an artificial superintelligance
dankrad51

I think 1-4 are good summaries of the arguments I'm making about nanobots. I would add another point that the reason it is hard to make nanobots is not about a lack of computational abilities (although that could also be a bottleneck) but simply a lack of knowledge about the physical world that can only be resolved by learning more about the physical world in a way that is relevant to making nanobots.

On point 5, from my current perspective, I think the idea of pivotal acts is totalitarian, not a good idea and most likely to screw things up if ever attempte... (read more)

Reply