If we don't actually have a common understanding of what "regret" refers to, it's probably best to stop using the term altogether.

If I'm always less likely to implement a given decision procedure D because implementing D in the past had a bad outcome, and always more likely to implement D because doing so had a good outcome (which is what I understand Quill_McGee to be endorsing, above), I run the risk of being less likely to implement a correct procedure as the result of a chance event.

There are more optimal approaches.

I endorse re-evaluating strategies in light of surprising outcomes.(It's not necessarily a bad thing to do in the absence of surprising outcomes, but there's usually something better to do with our time.) A bad outcome isn't necessarily surprising -- if I call "heads" and the coin lands tails, that's bad, but unsurprising. If it happens twice, that's bad and a little surprising. If it happens ten times, that's bad and very surprising.