State and local governments in the US have primary responsibility over education, road construction, utilities, policing, the courts, licensing restrictions, zoning laws, and public transportation. The federal government can and does sometimes overrule them in these areas, but they are mostly left to themselves. Federal government action tends to be restricted towards various welfare programs, foreign policy, and the health care system. I would not describe state and local governments as having little power.

State and local governments may have significant spending power, but limited freedom of using it. The central government can very effectively pressure states to comply with its policies by providing or withdrawing federal subsidies (e.g., NCLB). It can also directly veto any politically controversial decision by a state on how to spend its money (e.g., California Proposition 187).
Historically, the evolution of government systems was mainly driven by violence, with invasions and revolutions being the principal agents of selection process. The rules of the game were predetermined by our environment - land was a limited resource, for which our ancestors had to compete, if only to ensure the survival of their descendants.
The 20th century introduced a game changer. As agricultural productivity in developed countries rose by orders of magnitude and natural population growth practically came to a halt, possessing a large territory stopped being a necessity. Countries with little arable land, ultra-high population density and no natural resources can now not only feed their population, but also achieve top living standards. These changes may open a fundamentally different route for societal evolution – one that would not be based on violence or compulsion.
A small thought experiment - imagine what would happen if central governments cede most powers to smaller territorial units:
Unfortunately, there are serious obstacles to the successful implementation of this idea:
Do you think these problems are solvable?