Moral agents [..] have to respect each others values.

Have to, or else what?

Have to, logically. Like even numbers have to be divisible,

And how do we separate moral agents from agents that are not moral?

How do we recognise anything? They have behaviour and characteristics which match the definition.

Valuing rationality for what?

For itself. I do not accept that rationality can only be instrumental, a means to an end.

What would an agent which "values rationality" do?

The kind of thing EY, the CFAR and other promoters of rationality urge people to do.

If the solution is to build agents that "value rationality," can you explain how to do that?

In the same kind of very broad terms that MIRI can explain how to build Artificial Obsessive Compulsives.

If it's something so simple as to be analogous to adding brakes to a car,

The analogy was not about simplicity. Illustrative analogies are always simpler than what they are illustrating: that is where their usefulness lies.