The explore-exploit tradeoff is a fundamental thing in learning in complex environments (in AI this is studied in reinforcement learning). The way this often comes up for people is when ordering food (new restaurant / old favorite, favorite order / new order).