As an ethicist who routinely rejects trolley problems, I feel I must respond to this.
The trolley problem was first formulated by Philippa Foot as a parody of the ridiculous ethical thought experiments developed by philosophers of the time. Its purpose was to cause the reader to observe that the thought experiment is a contrived scenario that will never occur (apparently, it serves that purpose in most untrained folks), and thus serves as an indictment of how divorced reasoning about ethics in philosophy had become from the real world of ethical decision-making.
When I hear a trolley problem, I immediately try to start filling in details. Who are the five people, and who is the one? Why are they on the trolley tracks? Why am I the only person who can do something about it? Are there really no other alternatives, and if so, how is this known to me?
And if the best "least convenient possible world" ends up being one which doesn't even remotely resemble reality, then I don't mind if my moral compass outputs an undefined value in those spaces; my morality is built for the real world.
But trolley-style problems have real application, e.g. for politicians. Someone with actual political power will frequently have lose-lose problems that aren't hypothetical, and know that they will be blamed whatever they do or don't.
The trolley problem is one of the more famous thought experiments in moral philosophy, and studies by psychologists and anthropologists suggest that the response distributions to its major permutations remain roughly the same throughout all human cultures. Most people will permit pulling the lever to redirect the trolley so that it will kill one person rather than five, but will balk at pushing one fat person in front of the trolley to save the five if that is the only available option of stopping it.
However, in informal settings, where the dilemma is posed by a peer rather than a teacher or researcher, it has been my observation that there is another major category which accounts for a significant proportion of respondents' answers. Rather than choosing to flip the switch, push the fat man, or remain passive, many people will reject the question outright. They will attack the improbability of the premise, attempt to invent third options, or appeal to their emotional state in the provided scenario ("I would be too panicked to do anything",) or some combination of the above, in order to opt out of answering the question on its own terms.
However, in most cases, these excuses are not their true rejection. Those who tried to find third options or appeal to their emotional state will continue to reject the dilemma even when it is posed in its most inconvenient possible forms, where they have the time to collect themselves and make a reasoned choice, but no possibility of implementing alternative solutions.
Those who appealed to the unlikelihood of the scenario might appear to have the stronger objection; after all, the trolley dilemma is extremely improbable, and more inconvenient permutations of the problem might appear even less probable. However, trolleylike dilemmas are actually quite common in real life, when you take the scenario not as a case where only two options are available, but as a metaphor for any situation where all the available choices have negative repercussions, and attempting to optimize the outcome demands increased complicity in the dilemma. This method of framing the problem also tends not to cause people to reverse their rejections.
Ultimately, when provided with optimally inconvenient and general forms of the dilemma, most of those who rejected the question will continue to make excuses to avoid answering the question on its own terms. They will insist that there must be superior alternatives, that external circumstances will absolve them from having to make a choice, or simply that they have no responsibility to address an artificial moral dilemma.
When the respondents feel that they can possibly opt out of answering the question, the implications of the trolley problem become even more unnerving than the results from past studies suggest. It appears that we live in a world where not only will most people refuse complicity in a disaster in order to save more lives, but where many people reject outright the idea that they should have any considered set of moral standards for making hard choices at all. They have placed themselves in a reality too accommodating of their preferences to force them to have a system for dealing with situations with no ideal outcomes.