I think this is an important problem.

First, I would say that "things" are not arbitrary. I don't think that makes any sense to say "my sock is arbitrary."

Arbitrariness is a property of hypotheses, and in other contexts, decisions. In fact, maybe that's it - decisions may be arbitrary, and usually the kind of decision being referred to as arbitrary is one that privileges a certain hypothesis over others that seem to have just as much validity.

I've been trying to work in Jaynes concept of a transformation group for a problem into the concept. If there is a transformation group for the problem that leaves you with the same problem, but making different selections, then your selection method is arbitrary.

I don't know if I'm getting much better than "without justification". Oh well, time for bed.