The morality you suggest is what Derek Parfit calls collectively self-defeating. This means that if everyone were to follow it perfectly, there could be empirical situations where your actual goals, namely the well-being of those closest to you, are achieved less well than they would be if everyone followed a different moral view. So there could be situations in which people have more influence on the well-being of the family of strangers, and if they'd all favor their own relatives, everyone would end up worse off, despite everyone acting perfectly moral. Personally I want a world where everyone acts perfectly moral to be as close to Paradise as is empirically possible, but whether this is something you are concerned about is a different question (that depends on what question your seeking to answer by coming up with a coherent moral view).