An awareness of the context-sensitivity of our desires only leads me to value novelty (diversity on a personal-experience level). Not only do I expect some drift, I intend to enjoy it.

I haven't thought about how I'd like protection against black-holes of desire to be implemented. It's easy to imagine some hypothetical future-me resenting a growing novelty-itch I committed to.

Since I expect my preference for novelty to inform CEV, if you're making an argument against CEV that might persuade me, it must be that the optimization problem is too hard - that the result will either deny unexpected pleasures that could have otherwise been reached, or that the pull of expected pleasures will be stronger than expected and lead to a fixed state (violating our present belief in our preference for novelty).