That's a good observation, but it doesn't completely solve the problem. The problem here is not just the trolley problem. The problem is that people disagree on whether not pushing the fat man is a value, or a bug. The trolley problem is just one example of the difficulty of determining this in general.

There is a large literature on the trolley problem, and on how to solve the trolley problem, and the view taken in the paper, which was arrived at by many experts after studying the problem and conducting polls and other research, is that humans have a moral value called the "principle of double effect":

Harming another individual is permissible if it is the foreseen consequence of an act that will lead to a greater good; in contrast, it is impermissible to harm someone else as an intended means to a greater good.

Is this a value, or a bug? As long as we can't all agree on that, there's no reason to expect we can correctly figure out what are values and what are bugs.

There's really two problems:

1. Come up with a procedure to determine whether a behavior is a value or an error.

2. Convince most other people in the world that your procedure is correct.

Personally, I think a reasonable first step is to try to restrict ethics to utilitarian approaches. We'll never reach agreement as long as there are people still trying to use rule-based ethics (such as the "double effect" rule). The difficulty of getting most people to agree that there are no valid non-utilitarian ethical frameworks is just a small fraction of the difficulty of the entire program of agreeing on human values.