There's no reason a priori to suppose that any positive or negative effects not currently priced will be of the same order of magnitude.

There are some a posteriori reasons though - there are numerous studies that reject a causal link between the number of firearms and homicides, for example. This indicates that firearm manufacturers do not cause additional deaths, and therefore it would be wrong to only internalize the negative costs.

If you think there are benefits to having a population where most people own guns that are not going to be captured by the incentives of individuals who purchase guns for their own purposes, it's better to try to estimate what that effect size is and then provide appropriate incentives to people who want to purchase guns.

That's not true. It is not better, because providing appropriate incentives is very likely impossible in this case, e.g.:
- due to irrational political reasons (people have irrational fear of guns and will oppose any efforts to incentivize their purchase, while supporting efforts to disincentivize it);
- due to the fact that a reward system for preventing crime can be easily gamed (cobra effect), not to mention the fact that it will probably be very costly to follow up on all cases when crime was prevented;
- due to the fact that positive outcomes of gun ownership are inherently hard to quantify, hence in reality they will not be quantified and will not be taken into account (McNamara fallacy).

Quite apart from the application of this argument to AI, the example of a gun shop/manufacturer is quite bad. One reason is that passing on the negative externalities of selling a gun without passing on the positive externalities* (which is never done in practice and would be very difficult to do) creates an assymetry that biases the cost of firearms to be higher than it would have been in rational circumstances.

(*) Positive externalities of manufacturing and selling a gun include a deterrent effect on crime ("I would rather not try to rob that store, the clerk might be armed"), direct prevention of crime by armed citizens (https://en.wikipedia.org/wiki/Defensive_gun_use) or very strong positive effects of population being armed in extreme (rare) scenarios such as foreign invasion or the government turning tyrannical. I would suspect you wouldn't want to reward firearms manufacturers for all these positive outcomes (or at least it would be difficult, since these effects are very hard to quantify).

I agree! I don't think consciousness can be further analyzed or broken down into its constituent parts. It's just a fundamental property of the universe. It doesn't mean, however, that human consciousness has no explanation. (An explanation for human consciousness would be nice, because otherwise we have two kinds of things in the world: the physical and the mental, and none of these would be explicable in terms of the other, except maybe via solipsism.) Human consciousness, along with everything physical, is well explained by Christian theism, according to which God created the material world, which is inert and wholly subject to him, and then created mankind in His image. Man belongs both to the physical and the mental world and (s)he can be described as a consciousness made in the likeness of the Creator. Humans have/are a consciousness because God desired a personal relationship with them; for this reason they are not inert substances, but have free will.

@Gunnar_Zarncke 

Clearly, a process of experiencing is going on in humans. I don't dispute that. But that is strictly a different argument.

No, the process of experiencing is the main thing that distinguishes the mental (consciousness) from the physical. In fact, one way to define the mental is this (R. Swinburne): mental events are those that cannot happen without being experienced/observed. Mental events are not fully determined by the physical events, e.g. in the physical world there are no colors, only wavelengths of light. It is only in our consciousness that wavelengths of light acquire the quality of being a certain color, and even that may differ between one individual and another (what you see as green I might see as red).

The statement that they are potential lives is incorrect. An embryo is already alive and, since it has continuity through time with an adult human being (obviously actual living human), it has human identity as well. Therefore, it is a living human being.

"Only one life can come out of this process" is also incorrect. This is like having 4 teenagers and choosing 3 of them to be shot, and then concluding that "only one adult can come out of this process, therefore the 3 teenagers are merely potential lives and can be destroyed".

Why would inherent moral worth depend on the number of neurons or complexity of the brain? 

no, why?

the existence of decision-making beings is the best thing ever

I didn't say it's the best thing ever. Why are you misrepresenting what I said?

Effects caused by natural laws aren't "caused by God". They are caused by natural laws. It's not the same thing. God did create natural laws, but they serve a number of good purposes as I began to outline above.

what caused the evils of the Thirty Year War?

Struggle for power between the Habsburgs and France?

Oh, an one more thing. My updated premise 2 is:

2'. Whenever John says that X, then X. ( ∀ X:proposition, says(John, X) ⇒ X )

Note that X here is not a statement (grammatically valid sentence?), but a proposition. John can express it however he likes: by means of written word, by means of a demonstration or example, by means of a telepathy, etc. There is no need, specifically, to convert a proposition to a string or vice versa; as long as (1) is true and we most likely understand what proposition John is trying to convey, we will most likely believe in the correct normative proposition (that, if expressed in a statement, requires an "ought").

"It's all for the best in the end" is not a good argument, no. Such things are justified because the kind of world that serves the purposes God had in mind when creating it (for example, world in which moral agents exist and in which their choices are meaningful, i.e. make a practical difference) requires regular and predictable natural laws, and these (again, in the presence of meaningfully moral agents) have the side-effect of causing suffering from time to time. People have the option of committing good or committing evil, and these options are open to them only because certain actions lead to consequences that are considered good or evil: for example, if I hit my brother with a stone, I know that he might die. Thus if I want to kill my brother, I have the option of hitting him with a stone. This is so because of the presence of natural laws that connect my action to the desired effect. These natural laws also imply that a stone might fall on my brother's head by accident, not thrown by anyone in particular, and he might die. 

A world with meaningfully moral agents is an immensely good world, much better than a world without free agents. It is good that a person has the power to decide on the path the world (or part of it) would take, for it confers on them a creative function, a good in itself. The so-called natural evils(*) are an unfortunate side effect of us being such agents.

(*) The expression natural evil doesn't really sit right with me, because the concept of evil presupposes an agent causing it. Nothing that is inert can be good or evil. It can cause happiness or suffering, but it's not good or evil, strictly speaking.

Of course the above account is not consistent with utilitarian ethics, but utilitarian ethics is rejected by the Bible anyway, so that's not a problem.

For more along these lines I recommend Richard Swinburne, The Existence of God.