because in practice I'm concerned about people being coerced into signing contracts harmful to themselves. Not sure where I'd draw the line exactly; this is probably a Hard Problem.

Agreed.

Speaking personally, I'm also concerned about people willingly signing contracts harmful to themselves without coercion (since I don't believe that people are always correct, or even definitive, about what harms them). I'm also concerned about people willingly signing contracts that benefit them but are harmful to third parties far out of proportion to that benefit. In some cases I'm even concerned about people willingly signing contracts that benefit them proportionally to the harm they cause third parties.

As you say, it's a Hard Problem.

But, sure, within the context of a framework that avoids the more egregious harms, I'm all in favor of allowing people to do things in ways that suit them, including agreeing to binding contracts if that's what they want to do.