I received a letter telling me on no uncertain terms that if [US Customs] found another shipment of modafinil addressed to me, they would prosecute me as a drug smuggler.

You mean something like this? That's not really as meaningful as it seems. There is always some legal risk associated with doing anything since there are so many US laws that no one has even managed to count them, but a pretty serious search through legal database turns up no records of people being prosecuted for modafinil importation, ever. So that letter is 100% posturing by US Customs.

You should probably conclude from this that you're more likely to be prosecuted for illegally downloading music or jay walking.

And it's obviously everyones' personal choice to decide what level of legal risk they are comfortable with. But a rational person who wanted modafinil should be willing to order it from an online pharmacy at least as often as they're willing to pirate music or jaywalk without fear of prosecution. Otherwise their preferences for assuming legal risk are inconsistent.