Not sure I understand your question, but:

• I assume that each civilization only cares about itself. So one civilization succeeding does not "lead to large positive utilities for all future civilisations", only for itself. If civilization A assigns positive or negative value to civilization B succeeding, the expected utility calculations become more complicated.
• You cannot "let the game end". The fact that the game ends when one player receives R only represents the fact that each player knows that no previous player has received R (i.e., we arguably know that no civilization so far has successfully colonized space in our neighborhood).