I'm pleased to see this posted as I think the idea of machine ethics represents an enormous opportunity for proponents of FAI, etc. Machine ethics allow us to start talking about the extent to which human values are encoded in algorithms and machines in the present day, and allows us to explain why tackling that problem here and now is worth doing. This is true even if the more apocalyptic claims in the Futurist/Superintelligence/Singularity meme-cluster never come to pass.

I would agree with you assessment that many of the people thinking about this are people with a humanities background, and that they don't seem particularly well-versed as a whole on some of the technical/mathematical issues underlying some of these problems.

I'm particularly interested in this issue because machines seem to operate better than humans at multi-dimensional optimization problems, which seems at least superficially similar to the complexity of value thesis.