There was a picture of a dress going viral where some people saw it as black in intense lighting and others as blue in dim lightning. Our sense of color is very far from the "apparent color". A red object in a dark room is still red. Our brain calculates away lightning differences.

If there were local red torches on Chiron Beta Prime you could have objects that are very luminous near torches and objects that are not. Thus you could differentiate between diamons and turquoises. But diamons and turquoises are both grue. However turquoises are not white. Therefore grue is not white.

Note also that red torches could be a recent innovation. Thus what is natural "in this universe" is technology level dependant.