If it's worth saying, but not worth its own post (even in Discussion), then it goes here.
Because then any model of the full quantum category would describe the same reality: instead of the infinite-dimensional Hilbert space you could just choose say a much looser ontology with sets and relations between them (as in Spekken's toy model of quantum physics).
This arbitrariness would undermine the raison d'etre of the present model.