The issue is that it could create bad incentives. E.g. motorcyclists not wearing helmets and even acting inappropriately around self-driving cars, knowing it will avoid them, even if it causes it to crash. Or people stop buying safer cars because they are always chosen as "targets" by self-driving cars to crash into, making them statistically less safe.
I don't think the concerns are large enough to worry about, but hypothetically it's an interesting dilemma.
When I was a dumb kid, my friends and I regularly jaywalked (jayran?) across 3 lanes at a time of high speed traffic, just to get to a nicer place for lunch. Don't underestimate the populations of stupid and selfish people in the world, or the propensity to change behavior in response to changing incentives.
On the other hand, I'm not sure how the incentives here will change. Any self-driving car is going to be speckled with cameras, and "I know it will slam on the brakes or swerve to avoid me" might not be much temptation when followed with "then it will send my picture to the police".
When a collision is unavoidable, should a self-driving car try to maximize the survival chances of its occupants, or of all people involved?
http://www.wired.com/2014/05/the-robot-car-of-tomorrow-might-just-be-programmed-to-hit-you/