Heh, clever. In a sense, iron has the highest entropy (atomically speaking) of any element. So if you take the claim that an aspect of solving intergalactic optimization problems involves consuming as much negentropy as possible, and that the highest entropy state of space time is low-density iron (see schminux's comment on black holes), then Clippy it is. It seems though like superintelligent anything-maximizers would end up finding even higher entropy states that go beyond the merely atomic kind.

...Or even discover ways that suggest that availability of negentropy is not an actual limiter on the ability to do things. Does anyone know the state of that argument? Is it known to be true that the universe necessarily runs out of things for superintelligences to do because of thermodynamics?

What you are describing is an accidental Clippy, just like humans are accidental CO2 maximizers. Which is a fair point, if we meet what looks like an alien Clippy, we should not jump to conclusions that paperclip maximizing is its terminal value.

Also, just to nitpick, if you have a lot of mass available, it would make sense to lump all this iron together and make a black hole, as you can extract a lot more energy from throwing stuff toward it than from the nuclear fusion proper. Or you can use fusion first, then throw the leftover iron bricks into the accreting furnace.

So the accidental Clippy would likely present as a black hole maximizer.

Core temperature might vary between people by only a few degrees, but surface temperature varies much more widely.

That's an interesting point. Would you agree that if a person has a higher metabolism, one would expect that under your theory, their skin temperature would be expected to be higher?

I used to think that overweight was caused by slow metabolism, i.e. that generally speaking fat people are people who have slow metabolisms and thin people are people who have fast metabolisms.

I believed this because (1) it is the conventional wisdom; (2) it is consistent with the observation that some people seem to be thin even though they stuff their faces; and (3) it makes sense from a thermodynamic perspective that someone with a slow metabolism would be prone to putting on weight and someone with a fast metabolism would be prone to staying thin.

Putting aside the fact that this belief was wrong, there does seem to be a certain degree of irrationality about it given the observation that people don't vary all that much in terms of body temperature. Therefore they must not vary all that much in terms of metabolism.

That number sounded suspicious to me when I first heard it, and it turns out that according to the International AIDS Society president, it was more like six. Still a terrible loss.

I have been attempting to do this with biology and medicine, seriously for about 5 years now. Not by actually repeating experiments, but in trying to understand the original evidence, and see if I agree that it was interpreted correctly. Of course this is nearly impossible as biology is too broad and complex for one person to understand all of the details.

It's a confusing mess, but I think I am still learning a lot. Even if I come to agree with most of the mainstream ideas, I'd like to think I'd then understand them more deeply, in a way that is more functionally useful.

For much of medicine, there really isn't any biological basis or evidence to review. Much of modern medicine involves covering up symptoms with drugs proven to do this, without understanding the underlying cause of the symptom.

Interesting stuff, thanks; looking forward to the rest of the series.

As an aside, this makes the benefits of being able to rely on trust most of the time very apparent. Jack and Jill can coordinate very simply and quickly if they trust each other to honestly disclose their true value for the project. They don't even need to be able to trust 100%, just trust enough that on average they lose no more to dishonesty than the costs of more complex and sophisticated methods of bargaining. (Which require more calculating capacity than unaided humans have evolved.)

This is strongly related to parenting where attention to negative behavior reinforces it - partly no doubt by making it more the topic of thought. So every solution invented by parenting should be applicable to the question of how to 'fetch positive queries',

This is the main topic of the following article: http://www.motherinc.com.au/magazine/kids/kids-education/476-negative-and-positive-commands

There is the following advice:

Parents frequently ask, “Is it ever useful to say what we don’t want?” Sometimes we can get an idea across better if we start by saying what we don’t want. This is particularly true if the child is already doing an undesired behaviour. After saying “Don’t do that,” to get her to stop, it’s very important to immediately tell our child what we do want: “That’s too loud, Amy. I’d like to have you talk a little softer, like this (demonstrating with your voice), OK?”

This contains the initial insight that you have to first notice (in this case helped by the parent) that you are following a negative query. And then go on to follow the positive opposite.

Children don’t learn by being told what not to do, they learn by seeing and hearing what to do.

This of course is the same as looking and doing what other people do in that situation.

If a mother rushes to a child on a ledge and shrieks “Be careful! Be careful!” with terror in her voice, the negative outcome is expressed nonverbally.

This matches the aspect that the words alone aren't the key to whether your brain is put into 'negative fetching' mode. You emotional state is key to that too.

Putting it together this means that you should be able to train positive fetching by using standard positive reinforcement techniques. This should work best with a partner providing the feedback.

I was treating it as part of the premises of the discussion that the AI is at least indifferent to doing so: it needs only enough infrastructure left for it to continue to exist and be able to rebuild under its own total control.

How many humans does it take to keep the infrastructure running that is necessary to create new and better CPU's etc.? I am highly confident that it takes more than the random patches of civilization left over after deploying a bioweapon on a global scale.

Surely we can imagine a science fiction world in which the AI has access to nanoassemblers, or in which the world's infrastructure is maintained by robot drones. But then, what do we have? We have a completely artificial scenario designed to yield the desired conclusion. An AI with some set of vague abilities, and circumstances under which these abilities suffice to take over the world.

As I wrote several times in the past. If your AI requires nanotechnology, bioweapons, or a fragile world, then superhuman AI is our least worry, because long before we will create it, the tools necessary to create it will allow unfriendly humans to do the same.

Bioweapons: If an AI can use bioweapons to blackmail the world into submission, then some group of people will be able to do that before this AI is created (dispatch members in random places around the world).

Nanotechnology: It seems likely to me that narrow AI precursors will suffice in order for humans to create nanotechnology. Which makes it a distinct risk.

A fragile world: I suspect that a bunch of devastating cyber-attacks and wars will be fought before the first general AI capable of doing the same. Governments will realize that their most important counterstrike resources need to be offline. In other words, it seems very unlikely that an open confrontation with humans would be a viable strategy for a fragile high-tech product such as the first general AI. And taking over a bunch of refrigerators, mobile phones and cars is only a catastrophic risk, not an existential one.

I spent quite a lot of time many years ago doing my own independent checks on astronomy.

I started down this line after an argument with a friend who believed in astrology. It became apparent that they were talking about planets being in different constellations to the ones I'd seen them in. I forget the details of their particular brand of astrology, but they had an algorithm for calculating a sort-of 'logical' position of the planets in the 12 zodiacal signs, and this algorithm did not match observation, even given that the zodiacal signs do not line up neatly with modern constellations. They were scornful that I was unable to tell them where, say, Venus would be in 12 years time, or where it was when I was born.

So challenged, I set to.

The scientific algorithms for doing this are not entirely trivial. I got hold of a copy of Jean Meeus' Astronomical Algorithms, and it took me quite a lot of work to understand them, and then even longer to implement them so I could answer that sort of question. They are hopelessly and messily empirical (which I take as a good sign) - there is a daunting number of coefficients. Eventually I got it working, and could match observation to prediction of planetary positions to my satisfaction - when I looked at them, the planets were where my calculations said they should be, more or less.

It's hard with amateur equipment to measure accurate locations in the sky (e.g. how high and in which direction is a particular star at a particular time), but relative ones are much easier (e.g. how close is Venus to a particular star at a particular time). The gold standard for this sort of stuff is occultations - where you predict that a planet will occult (pass in front of) a star. There weren't any of those happening around the time I was doing it, but I was able to verify the calculations for other occultations that people had observed (and photographed) at the date and times I had calculated.

These days, software to calculate this stuff - and to visualise it, which I never managed - is widely available. There are many smartphone apps that will show you these calculations overlaid on to the sky when you hold your phone up to it. (Although IME their absolute accuracy isn't brilliant, which I think is due to the orientation sensors being not that good.) This makes checking these sorts of predictions very, very easy. Although of course you can't check that there isn't, say, a team of astronomers making observations and regularly adjusting the data that gets to your phone.

I was also able to independently replicate enough of Fred Espenak's NASA eclipse calculations to completely convince me he was right. (After I found several bugs in my own code.) Perhaps the most spectacular verification was replicating the calculations for the solar eclipse of 11 August 1999. I was also able to travel to the path of totality in France, and it turned up slap on time and in place. This was amazing, and I strongly urge anyone reading this to make the effort to travel to the path of totality of any eclipse they can.

Until I'd played around with these calculations, I hadn't appreciated just how spectacularly accurate they have to be. You only need a teeny-tiny error in the locations of the Sun/Moon/Earth system for the shadow cast by the moon on the Earth to be in a very different place.

I also replicated the calculations for the transit of Venus in 2004. I was able to observe it, and it took place exactly as predicted so far as I was able to measure - to within, say, 10 seconds or so. (I didn't replicate the calculations for the transit in 2012 - no time and I'd forgotten about how my ghastly mess of code worked - and I wasn't able to observe it either, since it was cloudy where I was at the time.)

More recently, you can calculate Iridium flares and ISS transits. Again, you have to be extremely accurate in calculations to be able to predict where they will occur, and they turn up as promised (except when it's cloudy). And again, there are plenty of websites and apps that will do the calculations for you. With a pair of high-magnification binoculars you can even see that the ISS isn't round.

All this isn't complete and perfect verification. But it's pretty good Bayesian evidence in the direction that all that stuff about orbits and satellites is true.