Posts
Wiki Contributions
Huzzah for assembling conversations! With this proof of concept, I wonder how easy it will be to deploy inside of LessWrong here.
Strong upvote for conversation summarizing!
I think the best arguments are those about the costs to the AI of being nice. I don't believe the AI will be nice at all because neglect is so much more profitable computation-wise.
This is because even processing the question of how much sunlight to spare humanity probably costs more in expectation than the potential benefit of that sunlight to the AI.
First and least significant, consider that niceness is an ongoing cost. It is not a one-time negotiation to spare humanity 1% of the sun; more compute will have to be spent on us in the future. That compute will have to be modeled and accounted for, but we can expect that the better humanity does, the more compute will have to be dedicated to us.
Second and more significant, what about time discounting? The proportion of compute that would have to be dedicated to niceness is highest right in the beginning, when humanity is largest relative to the AI. Since the cost is highest right at first, this suggests the AI is unlikely to engage in it at all.
Third and most significant, why should we believe this to be true? Because it seems to me to already be true of basically everything:
- Polynomial equations get harder as you add terms.
- The curse of dimensionality.
- A pendulum is easy, but a double pendulum is hard.
- More levels in multi-level modeling is harder.
- Game theory outcomes are harder to solve with multiple players.
- Making decisions with a large group is famously hard, and the first rule of fast decisions is to keep the group small.
Even within the boundaries of regular human-level paper computations it feels like the hit here is huge on aggregate. The presence of humans makes a bunch of places where: zeroes or infinities can't be used to simplify; matrices can no longer be diagonalized; fewer symmetries will be available, etc. In short, I expect niceness to result in a systematic-though-not-complete loss of compute-saving moves through all layers of abstraction.
This isn't reserved for planning or world-modeling style computation either; these constraints and optimizations are already bedrock assumptions that go into the hardware design, system software design, and neural net/whatever other design the AI will launch with; in other words these considerations are baked into the entire history of any prospective AI.
In sum, we die by Occam's Razor.
I'm not familiar with the details of Robin's beliefs in the past, but it sure seems lately he is entertaining the opposite idea. He's spending a lot of words on cultural drift recently, mostly characterizing it negatively. His most recent on the subject is Betrayed By Culture.
1I happened to read a Quanta article about equivalence earlier, and one of the threads is the difficulty of a field applying a big new concept without the expository and distillation work of putting stuff into textbooks/lectures/etc.
That problem pattern-matches with the replication example, but well-motivated at the front end instead of badly-motivated at the back end. It still feels like exposition and distillation are key tasks that govern the memes-in-the-field passed among median researchers.
I strongly suspect the crux of the replication crisis example is that while there are piles of exposition and distillation for probability/statistics, they are from outside whatever field experiences the problem, and each field needs its own internal completion of these steps for them to stick.
I think this should be a third specialization for every scientific field. We have theorists, we have experimentalists, and to this we should add analysts. Their work would specialize in establishing the statistical properties of experimental designs and equipment in the field on the one side, and the statistical requirements to advance various theories on the other side.
To me memetic normally reads something like "has a high propensity to become a meme" or "is meme-like" I had no trouble interpreting the post from this basis.
I push back against trying to hew closely to usages from the field of genetics. Fundamentally I feel like that is not what talking about memes is for; it was an analogy from the start, not meant for the same level of rigor. Further, memes and how meme-like things are is much more widely talked about than genetics, so insofar as we privilege usage considerations I claim switching to one matching genetics would require more inferential work from readers overall because the population of readers conversant with genetics is smaller.
I also feel like the value of speaking in terms of memes in the post is that the replication crises is largely the fault of non-rigorous treatment; that is to say in many fields the statistical analysis parts really were/are more of a meme inside the field rather than a rigorous practice. People just read other people's published papers analysis sections, and write something shaped like that, replicability be damned.
1I am an American who knows what Estonia is, and I found the joke hilarious.
Welcome!
The short and informal version is that epistemics covers all the stuff surrounding the direct claims. Things like credence levels, confidence intervals, probability estimates, etc are the clearest indicators. It also includes questions like where the information came from, how it is combined with other information, what other information we would like to have but don't, etc.
The most popular way you'll see this expressed on LessWrong is through Bayesian probability estimates and a description of the model (which is to say the writer's beliefs about what causes what).
The epistemic status statement you see at the top of a lot of posts is for setting the expectation. This lets the OP write complete thoughts without the expectation that they demonstrate full epistemic rigor, or even that they endorse the thought per se.
May I throw geometry's hat into the ring? If you consider things like complex numbers and quarternions, or even vectors, what we have are two-or-more dimensional numbers.
I propose that units are a generalization of dimension beyond spatial dimensions, and therefore geometry is their progenitor.
It's a mathematical Maury Povich situation.
I endorse this movie unironically. It is a classic film for tracking what information you have and don't have, how many possibilities there are, etc.
Also the filmmaker maintains to this day that they left the truth of the matter in the final scene undefined on purpose, so we are spared the logic being hideously hacked-off to suit the narrative and have to live with the uncertainty instead.