If an employee sucks at philosophy, how does he even recognize philosophical problems as problems that he needs to consult you for? Most people have little idea that they should feel confused and uncertain about things like epistemology, decision theory, and ethics. I suppose it might be relatively easy to teach an AI to recognize the specific problems that we currently consider to be philosophical, but what about new problems that we don't yet recognize as problems today?

Is this your reaction if you imagine delegating your affairs to an employee today? Are you making some claim about the projected increase in the importance of these philosophical decisions? Or do you think that a brilliant employees' lack of metaphilosophical understanding would in fact cause great damage right now?

I would divide the decisions requiring philosophical understanding into two main categories. One is decisions involved in designing/improving AI systems, like in the scenario above...

I agree that AI may increase the stakes for philosophical decisions. One of my points is that a natural argument that it might increase the stakes---by forcing us to lock in an answer to philosophical questions---doesn't seem to go through if you pursue this approach to AI control. There might be other arguments that building AI systems force us to lock in important philosophical views, but I am not familiar with those arguments.

I agree there may be other ways in which AI systems increase the stakes for philosophical decisions.

I like the bargaining example. I hadn't thought about bargaining as competitive advantage before, and instead had just been thinking about the possible upside (so that the cost of philosophical error was bounded by the damage of using a weaker bargaining scheme). I still don't feel like this is a big cost, but it's something I want to think about somewhat more.

If you think there are other examples like this that might help move my view. On my current model, these are just facts that increase my estimates for the importance of philosophical work, I don't really see it as relevant to AI control per se. (See the sibling, which is the better place to discuss that.)

one wrong move would cause me to lose control

I don't see cases where a philosophical error causes you to lose control, unless you would have some reason to cede control based on philosophical arguments (e.g. in the bargaining case). Failing that, it seems like there is a philosophically simple, apparently adequate notion of "remaining in control" and I would expect to remain in control at least in that sense.