In my opinion, one should always regret choices with bad outcomes and never regret choices with good outcomes. For Lo It Is Written ""If you fail to achieve a correct answer, it is futile to protest that you acted with propriety."" As well It Is Written "If it's stupid but it works, it isn't stupid." More explicitly, if you don't regret bad outcomes just because you 'did the right thing,' you will never notice a flaw in your conception of 'the right thing.' This results in a lot of unavoidable regret, and so might not be a good algorithm in practice, but at least in principle it seems to be better.