Is it always correct to choose that action with the highest expected utility?

Suppose I have a choice between action A, which grants -100 utilons with 99.9% chance and +1000000 utilons with 0.1% chance, or action B which grants +1 utilon with 100% chance. A has an expected utility of +900.1 utilons, while B has an expected utility of +1 utilon. This decision will be available to me only once, and all future decision will involve utility changes on the order of a few utilons.

Intuitively, it seems like action A is too risky. I'll almost certainly end up with a huge decrease in utility, just because there's a remote chance of a windfall. Risk aversion doesn't apply here, since we're dealing in utility, right? So either I'm failing to truly appreciate the chance at getting 1M utilons -- I'm stuck thinking about it as I would money -- or this is a case where there's reason to not take the action that maximizes expected value. Help?

EDIT: Changed the details of action A to what was intended