That's surprising. Muscle cramps are a symptom of hypokalemia (low potassium). Hyperkalemia is usually asymptomatic until you have heart beat irregularities. Also, based on the renal excretion of potassium, I would expect it doesn't accumulate much, and it would probably reach equilibrium in much less than a week, in a person with normal kidney function.

To determine whether the problem with bananas is potassium, try other sources, eg, eat a potato every day. or KCl.