If these agents actually change their definition of fairness based on other agents definitions then they are trivially exploitable.

I'm not sure that is trivial. What is trivial is that some kinds of willingness to change their definition of fairness makes them exploitable. However this doesn't hold for all kinds of willingness to change fairness definition. Some agents may change their definition of fairness in their favour for the purpose of exploiting agents vulnerable to this tactic but not willing to change their definition of fairness when it harms them. The only 'exploit' here is 'prevent them from exploiting me and force them to use their default definition of fair'.

Using Eliezer's punishment solution instead of Stuart's seems to be pure blackmail.

At a limit of sufficiently intelligent agents with perfect exchange of decision algorithm source code (utility-function source code not required) rational agents implementing Eliezer's punishment-for-unfairness system will arrive at punishment factors approaching zero and the final decision will approach Stuart's Pareto-dominant solution.

When there is mutual trust in the decision algorithms of the other agents or less trust in the communication process then a greater amount of punishment for unfairness is desirable.

My intuition is more along the lines of:

Suppose there's a population of agents you might meet, and the two of you can only bargain by simultaneously stating two acceptable-bargain regions and then the Pareto-optimal point on the intersection of both regions is picked. I would intuitively expect this to be the result of two adapted Masquerade algorithms facing each other.

Most agents think the fair point is N and will refuse to go below unless you do worse, but some might accept an exploitive point of N'. The slope down from N has to be steep enough that having a few N'-accepting agents will not provide a sufficient incentive to skew your perfectly-fair point away from N, so that the global solution is stable. If there's no cost to destroying value for all the N-agents, adding a single exploitable N'-agent will lead each bargaining agent to have an individual incentive to adopt this new N'-definition of fairness. But when two N'-agents meet (one reflected) their intersection destroys huge amounts of value. So the global equilibrium is not very Nash-stable.

Then I would expect this group argument to individualize over agents facing probability distributions of other agents.

My solution Pareto-dominates that approach, I believe. It's precisely the best you can do, given that each player cannot win more than what the other thinks their "fair share" is.

It seems the logical extension of your finitely many step-downs in "fairness" would be to define a function f(your_utility) which returns the greatest utility you will accept the other agent receiving for that utility you receive. The domain of this function should run from wherever your magical fairness point is down to the Nash equilibrium. As long as it is monotonically increasing, that should ensure unexploitability for the same reasons your finite version does. The offer both agents should make is at the greatest intersection point of these functions, with one of them inverted to put them on the same axes. (This intersection is guaranteed to exist in the only interesting case, where the agents do not accept as fair enough each other's magical fairness point)

Curiously, if both agents use this strategy, then both agents seem to be incentivized to have their function have as much "skew" (as EY defined it in clarification 2) as possible, as both functions are monotonically increasing so decreasing your opponents share can only decrease your own. Asymptotically and choosing these functions optimally, this means that both agents will end up getting what the other agent thinks is fair, minus a vanishingly small factor!

Let me know if my reasoning above is transparent. If not, I can clarify, but I'll avoid expending the extra effort revising further if what I already have is clear enough.

Try to put meeting location in the title, just to save people not involved a click and better draw in people actually in the area

Why is "good" vs "stupid" a conflict? Are they contradictory?

Yes. Doing things that result in predictable bad outcomes is bad. "Meaning well" does not especially impress me (any more).

A couple others have mentioned warnings on doing something only to become attractive (e.g. You will tire of it or become resentful). Something like general fitness with multiple benefits likely isn't a problem, but there's also an alternate perspective that has worked really well for me. Instead of optimizing for attractiveness, consider optimizing for awesomeness. Being awesome will tend to make people attracted to you, but it has the added bonus of improving your self-confidence (which again increases attractiveness) and life-satisfaction.

As far as how to do this, I wouldn't mind tips myself, but the general gist of what I do is just keep that drive to be more awesome at the back of my mind when making decisions (in LW parlance, adopt awesomeness as an instrumental value). Anyone else have ideas?

Cultural evolution selects for belief systems which hold that eventually everyone will see their logic and adopt them.