There is a cost to doing nothing as well. Calling them koans potentially has the following effects:
The question is whether it is more costly to make the change or not. How costly is the change? Are the costs long-term or short-term? (The costs of not making the change are mostly long-term.)
Also relevant: Apart from avoiding the above costs, are there benefits to changing it to something else? (For example, a better term could make the articles more interesting and intuitive to beginners than "koan" does.)
Knowing the kind of people who read LW, I guess that on reading “koan” more people will think about hacker koans than Zen kōans (also given no macron on the O).
If it's worth saying, but not worth its own post, even in Discussion, it goes here.