- Sister Y's The Right to Marry
- A Really, Really, Really Long Post About Gay Marriage That Does Not, In The End, Support One Side Or The Other also recommended by CharlieSheen
I can think of two big indirect benefits here. The first is perceived social legitimacy, which I don't think I need to elaborate much on: government recognition of a form of marriage implies formalized social approval of that type of relationship; "husband" and "wife" carry powerful private associations beyond "long-term girlfriend/boyfriend"; and in most modern societies the state's entangled enough with the institution of marriage that I'd expect a purely private claim to marriage status to feel like a cheat to many people.
The second seems more interesting. One of the reasons that "wife/husband" legitimizes a relationship and carries implications of stability, status, etc. is that it's potentially expensive: not so much in the sense of initial cost (which is usually substantial, but also almost entirely private), as of dissolution cost. If you get married, at least one party to the marriage is signing up for an institution that they can reasonably expect to extract a lot of money if the relationship is ever dissolved: legal fees, alimony, et cetera. (Child support used to fall into this category too; not so much anymore, but the previous state of affairs probably still contributes to the social position of marriage.)
The existence of these costs provides a very powerful and almost universally recognized signal of commitment, and of all the personal qualities that go with it. And many societies use that to informally gate positions: I've heard, for example, that it's hard to achieve higher officer ranks in the US military if you're not married, and similar conditions apply to at least some of the corporate hierarchies I've been exposed to. But most of these costs vanish if the union isn't legally recognized, and I'd expect a good chunk of the signaling value they carry to vanish with them.
There are other issues-- marriage carries a bunch of legal implications which are frequently to the advantage of the married couple. Offhand, there's hospital visitation, default inheritance, and immigration rights.