Comment author: Dagon 10 August 2016 06:44:02AM 2 points [-]

I may have been unclear - if you disallow some data, but allow a bunch of things that correlate with that disallowed data, your results are the same as if you'd had the data in the first place. You can (and, in a good algorithm, do) back into the disallowed data.

In other words, if the disallowed data has no predictive power when added to the allowed data, it's either truly irrelevant (unlikely in real-world scenarios) or already included in the allowed data, indirectly.

Comment author: Stuart_Armstrong 10 August 2016 07:09:40PM -1 points [-]

The main point of these ideas is to be able to demonstrate that a classifying algorithm - which is often nothing more than a messy black box - is not biased. This is often something companies want to demonstrate, and may become a legal requirement in some places. The above seems a reasonable definition of non-bias that could be used quite easily.

Comment author: bogus 05 August 2016 09:20:17PM 4 points [-]

It's not clear to me how this "fairness" criteria is supposed to work. If you simply don't include S among the predictors, then for any given x in X, the classification of x will be 'independent' of S in that a counterfactual x' with the exact same features but different S would be classified the exact same way. OTOH if you're aiming to have Y be uncorrelated with S even without controlling for X, this essentially requires adding S as a 'predictor' too; e.g. consider the Simpson paradox. But this is a weird operationalization of 'fairness'.

Comment author: Stuart_Armstrong 09 August 2016 01:54:21PM -2 points [-]

in that a counterfactual x' with the exact same features but different S would be classified the exact same way.

Except that from the x, you can often deduce S. Suppose S is race (which seems to be what people care about in this situation) while X doesn't include race but does include, eg, race of parents.

And I'm not aiming for S uncorrelated with Y (that's what the paper's authors seem to want). I'm aiming for S uncorrelated with Y, once we take into account a small number of allowable variables T (eg income).

Comment author: Lumifer 05 August 2016 02:29:55PM 7 points [-]

I'm not sure of the point of all this. You're taking a well-defined statistical concept of independence and renaming it 'fairness' which is a very flexible and politically-charged word.

If there is no actual relationship between S and Y, you have no problem and a properly fit classifier will ignore S since it does not provide any useful information. If the relationship between S and Y actually exists, are you going to define fairness as closing your eyes to this information?

Comment author: Stuart_Armstrong 09 August 2016 01:50:39PM -2 points [-]

I'm reusing the term from the paper, and trying to improve on it (as fairness in machine learning is relatively hot at the moment).

If the relationship between S and Y actually exists, are you going to define fairness as closing your eyes to this information?

That's what the paper essentially does, and that's what I think is wrong. Race and income are correlated; being ignorant of race means being at least partially ignorant of income. I'm aiming for something like "once you know income (and other allowable variables) then race should not affect the decision beyond that".

Comment author: Dagon 05 August 2016 06:02:26PM 2 points [-]

I think there's a fundamental goal conflict between "fairness" and precision. If the socially-unpopular feature is in fact predictive, then you either explicitly want a less-predictive algorithm, or you end up using other features that correlate with S strongly enough that you might as well just use S.

If you want to ensure a given distribution of S independent of classification, then include that in your prediction goals: have your cost function include a homogeneity penalty. Not that you're now pretty seriously tipping the scales against what you previously thought your classifier was predicting. Better and simpler to design and test the classifier in a straightforward way, but don't use it as the sole decision criteria.

Redlining (or more generally, deciding who gets credit) is a great example for this. If you want accurate risk assessment, you must take into account data (income, savings, industry/job stability, other kinds of debt, etc.) that correlates with ethnic averages. The problem is not that the risk classifiers are wrong, the problem is that correct risk assessments lead to unpleasant loan distributions. And the sane solution is to explicitly subsidize the risks you want to encourage for social reasons, not to lie about the risk by throwing away data.

Comment author: Stuart_Armstrong 09 August 2016 01:32:46PM -2 points [-]

Redlining seems to go beyond what's economically efficient, as far as I can tell (see wikipedia).

Redlining (or more generally, deciding who gets credit) is a great example for this. If you want accurate risk assessment, you must take into account data (income, savings, industry/job stability, other kinds of debt, etc.) that correlates with ethnic averages.

Er, that's precisely my point here. My idea is to have certain types of data explicitly permitted; in this case I set T to be income. The definition of "fairness" I was aiming for is that once that permitted data is taken into account, there should remain no further discrimination on the part of the algorithm.

This seems a much better idea that the paper's suggestion of just balancing total fairness (eg willingness to throw away all data that correlates) with accuracy in some undefined way.

Fairness in machine learning decisions

-2 Stuart_Armstrong 05 August 2016 09:56AM

There's been some recent work on ensuring fairness in automated decision making, especially around sensitive areas such as racial groups. The paper "Censoring Representations with an Adversary" looks at one way of doing this.

It looks at a binary classification task where X ⊂ Rn and Y = {0, 1} is the (output) label set. There is also S = {0, 1} which is a protected variable label set. The definition of fairness is that, if η : X → Y is your classifier, then η should be independent of S. Specifically:

  • P(η(X)=1|S=1) = P(η(X)=1|S=0)

There is a measure of discrimination, which is the extent to which the classifier violates that fairness assumption. The paper then suggests to tradeoff optimise the difference between discrimination and classification accuracy.

But this is problematic, because it risks throwing away highly relevant information. Consider redlining, the practice of denying services to residents of certain areas based on the racial or ethnic makeups of those areas. This is the kind of practice we want to avoid. However, generally the residents of these areas will be poorer than the average population. So if Y is approval for mortgages or certain financial services, a fair algorithm would essentially be required to reach a decision that ignores this income gap.

And it doesn't seem the tradeoff with accuracy is a good way of compensating for this. Instead, a better idea would be to specifically allow certain variables to be considered. For example, let T be another variable (say, income) that we want to allow. Then fairness would be defined as:

  • ∀t, P(η(X)=1|S=1, T=t) = P(η(X)=1|S=0, T=t)

What this means is that T can distinguish between S=0 and S=1, but, once we know the value of T, we can't deduce anything further about S from η. For instance, once the bank knows your income, it should be blind to other factors.

Of course, with enough T variables, S can be determined with precision. So each T variable should be fully justified, and in general, it must not be easy to establish the value of S via T.

Comment author: capybaralet 30 July 2016 01:52:00PM 0 points [-]

"So conservation of expected moral evidence is something that would be automatically true if morality were something real and objective, and is also a desiderata when constructing general moral systems in practice."

This seems to go against your pulsar example... I guess you mean something like: "if [values were] real, objective, and immutable"?

Comment author: Stuart_Armstrong 01 August 2016 09:10:26AM 0 points [-]

Sorry, I don't get your point. Could you develop it?

Comment author: buybuydandavis 22 July 2016 10:56:22PM 1 point [-]

I always had the informal impression that the optimal policies were deterministic

So an impression that optimal memoryless polices were deterministic?

That seems even less likely to me. If the environment has state, and you're not allowed to, you're playing at a disadvantage. Randomness is one way to counter state when you don't have state.

But it really does seem that there is a difference between facing an environment and another player - the other player adapts to your strategy in a way the environment doesn't. The environment only adapts to your actions.

I still don't see a difference. Your strategy is only known from your actions by both another player and the environment, so they're in the same boat.

Labeling something the environment or a player seems arbitrary and irrelevant. What capabilities are we talking about? Are these terms of art for which some standard specifying capability exists?

What formal distinctions have been made between players and environments?

Comment author: Stuart_Armstrong 23 July 2016 04:56:32PM 1 point [-]

Take a game with a mixed strategy Nash equilibrium. If you and the other player follow this, using source of randomness that remain random for the other player, then it is never to your advantage to deviate from this. You play this game, again and again, against another player or against the environment.

Consider an environment in which the opponent's strategies are in an evolutionary arms race, trying to best beat you; this is an environmental model. Under this, you'd tend to follow the Nash equilibrium on average, but, at (almost) any given turn, there's a deterministic choice that's a bit better than being stochastic, and it's determined by the current equilibrium of strategies of the opponent/environment.

However, if you're facing another player, and you make deterministic choices, you're vulnerable if ever they figure out your choice. This is because they can peer into your algorithm, not just track your previous actions. To avoid this, you have to be stochastic.

This seems like a potentially relevant distinction.

Comment author: Lumifer 22 July 2016 07:44:20PM 1 point [-]

The environment only adapts to your actions.

Is this how you define environment?

Comment author: Stuart_Armstrong 23 July 2016 04:43:16PM 1 point [-]

At least as an informal definition, it seems pretty good.

Comment author: buybuydandavis 21 July 2016 12:21:25PM 1 point [-]

I always had the informal impression that the optimal policies were deterministic

Really? I wouldn't have ever thought that at all. Why do you think you thought that?

when facing the environment rather that other players. But stochastic policies can also be needed if the environment is partially observable

Isn't kind of what a player is? Part of the environment with a strategy and only partially observable states?

Although for this player, don't you have an optimal strategy, except for the first move? The Markov "Player" seems to like change.

Isn't this strategy basically optimal? ABABABABABAB... Deterministic, just not the same every round. Am I missing something?

Comment author: Stuart_Armstrong 22 July 2016 06:51:28PM 1 point [-]

ABABABABABAB...

It's deterministic, but not memoryless.

But it really does seem that there is a difference between facing an environment and another player - the other player adapts to your strategy in a way the environment doesn't. The environment only adapts to your actions.

I think for unbounded agents facing the environment, a deterministic policy is always optimal, but this might not be the case for bounded agents.

Comment author: Gram_Stone 19 July 2016 02:19:44PM *  4 points [-]

Is the Absent-minded Driver an example of a single-player decision problem whose optimal policy is stochastic? Isn't the optimal policy to condition your decision on an unbiased coin?

I ask because it seems like it might make a good intuitive example, as opposed to the POMDP in the OP. But I'm not sure who your intended audience is.

Comment author: Stuart_Armstrong 19 July 2016 05:16:43PM 3 points [-]

Yes, you can see this POMDP as a variant of the absent minded-driver, and get that result.

View more: Prev | Next