To connect this to the problem of (cost) function minimization, which is important e.g. in machine learning, changing a single variable at a time is even worse than gradient descent, which is a common method there. Not only do you only look in the immediate neighborhood, you also only look along a single axis at a time. It's no wonder that you don't find a local minimum this way!