I got an amazing amount of use out of Order of Magnitude Physics. It can get you in the habit of estimating everything in terms of numbers. I've found that relentlessly calculating estimates greatly reduces the number of biased intuitive judgments I make. A good class will include a lot of interaction and out-loud thinking about the assumptions your estimates are based on. Also or as an alternative, a high-level engineering design course can provide many of the same experiences within the context of a particular domain. (Aerospace/architecture/transportation/economic systems can all provide good design problems for this type of thinking - oddly, I haven't yet seen a computer science design problem example that works as well.)

Also, I'll second recommendations for just about any psychology course. And anywhere you see a course cross-listed between psychology and economics you'll have a good chance of learning about human bias.

Close, but the tricky part is that the universe can expand at greater than the speed of light. Nothing (like photons) that can influence cause and effect can travel faster than c but the fabric of spacetime itself can expand faster than the speed of light. Looking at the (models of) the first 10^-30 seconds highlights this to an extreme degree. Even now some of the galaxies that are visible to us are becoming further away from us by more than a light year per year. That means that the light they are currently emitting (if any) will never reach us.

To launch an AI out of our future light cone you must send it past a point at which the expansion of the universe makes that point further away from us at c. At that time it will be one of the points at the edge of our future light cone and beyond it the AI can never touch us.

You are mistaken.

Gravitomagnetism -- what's up with that?

It's an phrasing of how gravity works with equations that have the same form as Maxwell's equations. And frankly, it's pretty neat: writing the laws for gravity this way gets you mechanics while approximately accounting for general relativity (how approximate and what it leaves off, I'm not sure of).

When I first found out about this, it blew my mind to know that gravity acts just like electromagnetism, but for different properties. We all know about the parallel between Coulomb's law and Newton's law of gravitation, but the gravitoelectromagnetism (GEM) equations show that it goes a lot deeper.

Besides being a good way to ease into an intuitive understanding of the Einstein field equations, to me, it's basically saying that gravity and EM are both obeying some more general law. Anyone know if work has been done in unifying gravity and EM this way? All I hear about is that it's easy to unify strong, weak, and EM forces, but gravity is the stumbling block, so this should be something they'd want to explore more.

Yet when you go investigate "gravitational induction" to find out how the gravitic parallel to magnetic fields works, you find that this gravitomagnetic field is called the torsion field, and its existence is (at least approximately) implied by general relativity, but then the Wikipedia page says that the torsion field is a pseudoscientific concept. Hm...

So, anyone have an understanding of the GEM analogy and can make sense of this? Does it suggest a way to unify gravity and EM? Or how to create a coil of mass flow that can "gravitize" a region (as a coil of current magnitizes a metal bar)?

Another case I'd like to be considered more is "if we can't/shouldn't control the AIs, what can we do to still have influence over them?"

Thermite. Destroying or preventing them is the ONLY option in that situation. (Well, I suppose you could launch them out of our future light cone.)

An AI that "valued" keeping the world looking roughly the way it does now, that was specifically instructed never to seize control of more than X number of each of several thousand different kinds of resources, and whose principal intended activity was to search for, hunt down, and destroy AIs that seemed to be growing too powerful too quickly might be an acceptable risk.

This would not be acceptable to me, since I hope to be one of those AIs.

The morals of FAI theory don't mesh well at all with the morals of transhumanism. This is surprising, since the people talking about FAI are well aware of transhumanist ideas. It's as if people compartmentalize them and think about only one or the other at a time.

A prima facie case against the likelihood of a major-impact intelligence-explosion singularity:

Firstly, the majoritarian argument. If the coming singularity is such a monumental, civilization-filtering event, why is there virtually no mention of it in the mainstream? If it is so imminent, so important, and furthermore so sensitive to initial conditions that a small group of computer programmers can bring it about, why are there not massive governmental efforts to create seed AI? If nothing else, you might think that someone could exaggerate the threat of the singularity and use it to scare people into giving them government funds. But we don’t even see that happening.

Second, a theoretical issue with self-improving AI: can a mind understand itself? If you watch a simple linear Rube Goldberg machine in action, then you can more or less understand the connection between the low- and the high-level behavior. You see all the components, and your mind contains a representation of those components and of how they interact. You see your hand, and understand how it is made of fingers. But anything more complex than an adder circuit quickly becomes impossible to understand in the same way. Sure, you might in principle be able to isolate a small component and figure out how it works, but your mind simply doesn’t have the capacity to understand the whole thing. Moreover, in order to improve the machine, you need to store a lot of information outside your own mind (in blueprints, simulations, etc.) and rely on others who understand how the other parts work.

You can probably see where this is going. The information content of a mind cannot exceed the amount of information necessary to specify a representation of that same mind. Therefore, while the AI can understand in principle that it is made up of transistors etc., its self-representation necessary has some blank areas. I posit that the AI cannot purposefully improve itself because this would require it to understand in a deep, level-spanning way how it itself works. Of course, it could just add complexity and hope that it works, but that’s just evolution, not intelligence explosion.

So: do you know any counterarguments or articles that address either of these points?

Others complain that the existence of an easy medical solution prevents people from learning personal responsibility. But here we see the status-quo bias at work, and so can apply a preference reversal test. If people really believe learning personal responsibility is more important than being not addicted to heroin, we would expect these people to support deliberately addicting schoolchildren to heroin so they can develop personal responsibility by coming off of it. Anyone who disagrees with this somewhat shocking proposal must believe, on some level, that having people who are not addicted to heroin is more important than having people develop whatever measure of personal responsibility comes from kicking their heroin habit the old-fashioned way.

Now that's a good use of the reversal test!

Thoughts I found interesting:

The failures indicate that, instead of being threads in a majestic general theory, the successes were just narrow, isolated solutions to problems that turned out to be easier than they originally appeared.

Interesting because I don't think it's true. I think the problem is more about the need of AI builders to show results. Providing a solution (or a partial solution or a path to a solution) in a narrow context is a way to do that when your tools aren't yet powerful enough for more general or mixed approaches. Given the variety of identifiable structures in the human brain that gives us intelligence I strongly expect that an AI will be built by combining many specialized parts that will probably be based on multiple research areas we'd recognize today.

One obvious influence comes from computer science, since presumably AI will eventually be built using software. But this fact appears irrelevant to me, and so the influence of computer science on AI seems like a disastrous historical accident.

Interesting because it forced me to consider what I think AI is outside the context of computer science - something I don't normally do.

In my view, AI can and must become a hard, empirical science, in which researchers propose, test, refine, and often discard theories of empirical reality.

Interesting because I'm very curious to see what this means in the context of your coming proposal.

Hi all - been lurking since LW started and followed Overcoming Bias before that, too.