Specifically, will I die of radiation poisoning if I use VNM utility to make decisions when I'm uncertain about what my preferences even are? I.e., maximize expected utility, where the expectation is taken over my uncertainty about preferences in addition to any other uncertainty.

Yes. You can't compare or aggregate utilities from different utility functions. So at present, you basically have to pick one and hope for the best.

This is exactly what my brain claimed you said :) Now I can make my comment.

Game theorists do this all the time - at least economists. They'll create a game, then say something like "now let's introduce noise into the payoffs" but the noise ends up being in the utility function. Then they go and find an equilibrium or something using expected utility.

Now every practical example I can think of off the top of my hand, you can reinterpret the uncertainty as uncertainty about actual outcomes with utilities associated with those outcomes and the math goes through. Usually the situation is something like letting U($)=$ for simplicity because risk aversion is orthogonal to what they're interested in, so you can easily think about the uncertainty as being over $ rather than U($). This simplicity allows them to play fast and loose with VNM utility and get away with it, but I wouldn't be surprised if someone made a model where they really do mean for the uncertainty to be over one's own preferences and went ahead and used VNM utility. In any case, no one ever emphasized this point in any of the econ or game theory courses I've taken, grad or otherwise.