Heh, clever. In a sense, iron has the highest entropy (atomically speaking) of any element. So if you take the claim that an aspect of solving intergalactic optimization problems involves consuming as much negentropy as possible, and that the highest entropy state of space time is low-density iron (see schminux's comment on black holes), then Clippy it is. It seems though like superintelligent anything-maximizers would end up finding even higher entropy states that go beyond the merely atomic kind.

...Or even discover ways that suggest that availability of negentropy is not an actual limiter on the ability to do things. Does anyone know the state of that argument? Is it known to be true that the universe necessarily runs out of things for superintelligences to do because of thermodynamics?

What you are describing is an accidental Clippy, just like humans are accidental CO2 maximizers. Which is a fair point, if we meet what looks like an alien Clippy, we should not jump to conclusions that paperclip maximizing is its terminal value.

Also, just to nitpick, if you have a lot of mass available, it would make sense to lump all this iron together and make a black hole, as you can extract a lot more energy from throwing stuff toward it than from the nuclear fusion proper. Or you can use fusion first, then throw the leftover iron bricks into the accreting furnace.

So the accidental Clippy would likely present as a black hole maximizer.

Core temperature might vary between people by only a few degrees, but surface temperature varies much more widely.

That's an interesting point. Would you agree that if a person has a higher metabolism, one would expect that under your theory, their skin temperature would be expected to be higher?

I used to think that overweight was caused by slow metabolism, i.e. that generally speaking fat people are people who have slow metabolisms and thin people are people who have fast metabolisms.

I believed this because (1) it is the conventional wisdom; (2) it is consistent with the observation that some people seem to be thin even though they stuff their faces; and (3) it makes sense from a thermodynamic perspective that someone with a slow metabolism would be prone to putting on weight and someone with a fast metabolism would be prone to staying thin.

Putting aside the fact that this belief was wrong, there does seem to be a certain degree of irrationality about it given the observation that people don't vary all that much in terms of body temperature. Therefore they must not vary all that much in terms of metabolism.

That number sounded suspicious to me when I first heard it, and it turns out that according to the International AIDS Society president, it was more like six. Still a terrible loss.

I have been attempting to do this with biology and medicine, seriously for about 5 years now. Not by actually repeating experiments, but in trying to understand the original evidence, and see if I agree that it was interpreted correctly. Of course this is nearly impossible as biology is too broad and complex for one person to understand all of the details.

It's a confusing mess, but I think I am still learning a lot. Even if I come to agree with most of the mainstream ideas, I'd like to think I'd then understand them more deeply, in a way that is more functionally useful.

For much of medicine, there really isn't any biological basis or evidence to review. Much of modern medicine involves covering up symptoms with drugs proven to do this, without understanding the underlying cause of the symptom.

Interesting stuff, thanks; looking forward to the rest of the series.

As an aside, this makes the benefits of being able to rely on trust most of the time very apparent. Jack and Jill can coordinate very simply and quickly if they trust each other to honestly disclose their true value for the project. They don't even need to be able to trust 100%, just trust enough that on average they lose no more to dishonesty than the costs of more complex and sophisticated methods of bargaining. (Which require more calculating capacity than unaided humans have evolved.)

This is strongly related to parenting where attention to negative behavior reinforces it - partly no doubt by making it more the topic of thought. So every solution invented by parenting should be applicable to the question of how to 'fetch positive queries',

This is the main topic of the following article: http://www.motherinc.com.au/magazine/kids/kids-education/476-negative-and-positive-commands

There is the following advice:

Parents frequently ask, “Is it ever useful to say what we don’t want?” Sometimes we can get an idea across better if we start by saying what we don’t want. This is particularly true if the child is already doing an undesired behaviour. After saying “Don’t do that,” to get her to stop, it’s very important to immediately tell our child what we do want: “That’s too loud, Amy. I’d like to have you talk a little softer, like this (demonstrating with your voice), OK?”

This contains the initial insight that you have to first notice (in this case helped by the parent) that you are following a negative query. And then go on to follow the positive opposite.

Children don’t learn by being told what not to do, they learn by seeing and hearing what to do.

This of course is the same as looking and doing what other people do in that situation.

If a mother rushes to a child on a ledge and shrieks “Be careful! Be careful!” with terror in her voice, the negative outcome is expressed nonverbally.

This matches the aspect that the words alone aren't the key to whether your brain is put into 'negative fetching' mode. You emotional state is key to that too.

Putting it together this means that you should be able to train positive fetching by using standard positive reinforcement techniques. This should work best with a partner providing the feedback.

I was treating it as part of the premises of the discussion that the AI is at least indifferent to doing so: it needs only enough infrastructure left for it to continue to exist and be able to rebuild under its own total control.

How many humans does it take to keep the infrastructure running that is necessary to create new and better CPU's etc.? I am highly confident that it takes more than the random patches of civilization left over after deploying a bioweapon on a global scale.

Surely we can imagine a science fiction world in which the AI has access to nanoassemblers, or in which the world's infrastructure is maintained by robot drones. But then, what do we have? We have a completely artificial scenario designed to yield the desired conclusion. An AI with some set of vague abilities, and circumstances under which these abilities suffice to take over the world.

As I wrote several times in the past. If your AI requires nanotechnology, bioweapons, or a fragile world, then superhuman AI is our least worry, because long before we will create it, the tools necessary to create it will allow unfriendly humans to do the same.

Bioweapons: If an AI can use bioweapons to blackmail the world into submission, then some group of people will be able to do that before this AI is created (dispatch members in random places around the world).

Nanotechnology: It seems likely to me that narrow AI precursors will suffice in order for humans to create nanotechnology. Which makes it a distinct risk.

A fragile world: I suspect that a bunch of devastating cyber-attacks and wars will be fought before the first general AI capable of doing the same. Governments will realize that their most important counterstrike resources need to be offline. In other words, it seems very unlikely that an open confrontation with humans would be a viable strategy for a fragile high-tech product such as the first general AI. And taking over a bunch of refrigerators, mobile phones and cars is only a catastrophic risk, not an existential one.