Aren't Turing and von Neumann (surely they invented "computers" as much as anyone) counterexamples to your thesis?

No, not if you actually read into the history.

Turing published some conceptual math papers that would eventually get the field of computability and thus computer science started, but by no means did he invent the computer.

Computer evolution was already well under way when Turing published his paper on computability introducing Turing Machines in 1936.

The early British programmable digital computer, the Colossus, was developed by colleagues/contemporaries of Turing, but Turing was not involved, and at the time his abstract Turing Machine concept was not viewed as important:

It was thus not a fully general Turing-complete computer, even though Alan Turing worked at Bletchley Park. It was not then realized that Turing completeness was significant; most of the other pioneering modern computing machines were also not Turing complete (e.g. the Atanasoff–Berry Computer, the Bell Labs relay machines (by George Stibitz et al.), or the first designs of Konrad Zuse). The notion of a computer as a general purpose machine—that is, as more than a calculator devoted to solving difficult but specific problems—did not become prominent for several years.

Colossus was designed by the engineer Tommy Flowers.

The first Turing complete computer was the Z3, developed in germany by the engineer Konrad Zuse. Zuse is unlikely to have even heard of Turing, and the Z3 wasn't proven Turing Complete until many decades later.

Concerning Von Neumman's architecture:

This architecture is to this day the basis of modern computer design, unlike the earliest computers that were 'programmed' by altering the electronic circuitry. Although the single-memory, stored program architecture is commonly called von Neumann architecture as a result of von Neumann's paper, the architecture's description was based on the work of J. Presper Eckert and John William Mauchly, inventors of the ENIAC at the University of Pennsylvania.[51]

Eckert was an electrical engineer, Mauchly a physicist.

Turing and von Neumman both made lasting contributions in the world of ideas, but they did not invent computers, not even close.