It seems like this is a resurgence of the ought-from-is dilemma. For this reason I think accepting that "pain is bad" as a primitive is acceptable, as there doesn't seem any way to derive it from anything. Off hand I might venture that more specifically we call pain "bad" because creatures strive so strongly to avoid it, this too being a primitive. (And like you mentioned, we appreciate it when it helps us avoid other things we want to avoid even more, like death).

What makes this different from lutefisk is how strongly we try to avoid it: whatever you say, when choosing between being branded and lutefisk, we'll choose lutefisk. And then most moral theories posit that the suffering of eating lutefisk is minor enough to not be worth serious moral consideration. But if we held someone down to feed them lutefisk (or tied them up to funnel sauerkraut into their mouth) and they were crying and screaming terribly and honestly willing to run across hot coals to avoid said ethnic provender, we might well consider lutefisk "bad", at that time and for that person.