Michael Strevens teaches philosophy at New York University. In his book, The Knowledge Machine: How Irrationality Created Modern Science, he argues that what makes modern science so productive is the peculiar behavior of scientists. From the publisher’s site:

Like such classic works as Karl Popper’s The Logic of Scientific Discovery and Thomas Kuhn’s The Structure of Scientific Revolutions, The Knowledge Machine grapples with the meaning and origins of science, using a plethora of . . .  examples to demonstrate that scientists willfully ignore religion, theoretical beauty, and . . . philosophy to embrace a constricted code of argument whose very narrowness channels unprecedented energy into empirical observation and experimentation. Strevens calls this scientific code the iron rule of explanation, and reveals the way in which the rule, precisely because it is unreasonably close-minded, overcomes individual prejudices to lead humanity inexorably toward the secrets of nature.

Here Strevens presents a very helpful explanation of quantum mechanics, while explaining that physicists (most of them anyway) are following Newton’s example when they use the theory to make exceptionally accurate predictions, even though the theory’s fundamental meaning is mysterious (in the well-known phrase, they “shut up and calculate”):

To be scientific simply was to be Newtonian. The investigation of nature [had] changed forever. No longer were deep philosophical insights of the sort that founded Descartes’s system considered to be the keys to the kingdom of knowledge. Put foundational matters aside, Newton’s example seemed to urge, and devote your days instead to the construction of causal principles that, in their forecasts, follow precisely the contours of the observable world. . . .

[This is] Newton’s own interpretation of his method, laid out in a postscript to the Principia’s second edition of 1713. There Newton summarizes the fundamental properties of gravitational attraction—that it increases “in proportion to the quantity of solid matter” and decreases in proportion to distance squared—and then continues:

I have not as yet been able to deduce from phenomena the reason for these properties of gravity, and I do not feign hypotheses. For whatever is not deduced from the phenomena must be called a hypothesis; and hypotheses, whether metaphysical or physical, or based on occult qualities, or mechanical, have no place in experimental philosophy. . . . It is enough that gravity really exists and acts according to the laws that we have set forth and is sufficient to explain all the motions of the heavenly bodies and of our sea.

The thinkers around and after Newton got the message, one by one.

[Jumping ahead three centuries:]

According to Roger Penrose, one of the late twentieth century’s foremost mathematical physicists, quantum mechanics “makes absolutely no sense.” “I think I can safely say that nobody understands quantum mechanics,” remarked Richard Feynman. How can a theory be widely regarded both as incomprehensible and also as the best explanation we have of the physical world we live in?

. . . Quantum theory derives accurate predictions from a notion, superposition, that is quite beyond our human understanding. Matter, says quantum mechanics, occupies the state called superposition when it is not being observed [or measured]. An electron in superposition occupies no particular point in space. It is typically, rather, in a kind of “mix” of being in many places at once. The mix is not perfectly balanced: some places are far more heavily represented than others. So a particular electron’s superposition might be almost all made up from positions near a certain atomic nucleus and just a little bit from positions elsewhere. That is the closest that quantum mechanics comes to saying that the electron is orbiting the nucleus.

As to the nature of this “mix”—it is a mystery. We give it a name: superposition. But we can’t give it a philosophical explanation. What we can do is to represent any superposition with a mathematical formula, called a “wave function.” An electron’s wave function represents its physical state with the same exactitude that, in Newton’s physics, its state would be represented by numbers specifying its precise position and velocity. You may have heard of quantum mechanics’ “uncertainty principle,” but forget about uncertainty here: the wave function is a complete description that captures every matter of fact about an electron’s physical state without remainder.

So far, we have a mathematical representation of the state of any particular piece of matter, but we haven’t said how that state changes in time. This is the job of Schrödinger’s equation, which is the quantum equivalent of Newton’s famous second law of motion F = ma, in that it spells out how forces of any sort—gravitational, electrical, and so on—will affect a quantum particle. According to Schrödinger’s equation, the wave function will behave in what physicists immediately recognize as a “wavelike” way. That is why, according to quantum mechanics, even particles such as electrons conduct themselves as though they are waves.

In the early days of quantum mechanics, Erwin Schrödinger, the Austrian physicist who formulated the equation in 1926, and Louis de Broglie, a French physicist—both eventual Nobel Prize winners—wondered whether the waves described by quantum mechanics might be literal waves traveling through a sea of “quantum ether” that pervades our universe. They attempted to understand quantum mechanics, then, using the old model of the fluid.

This turned out to be impossible for a startling reason: it is often necessary to assign a wave function not to a single particle, like an electron, but to a whole system of particles. Such a wave function is defined in a space that has three dimensions for every particle in the system: for a 2-particle system, then, it has 6 dimensions; for a 10-particle system, 30 dimensions. Were the wave to be a real entity made of vibrations in the ether, it would therefore have to be flowing around a space of 6, or 30, or even more dimensions. But our universe rather stingily supplies only three dimensions for things to happen in. In quantum mechanics, as Schrödinger and de Broglie soon realized, the notion of substance as fluid fails completely.

There is a further component to quantum mechanics. It is called Born’s rule, and it says what happens when a particle’s position or other state is measured. Suppose that an electron is in a superposition, a mix of being “everywhere and nowhere.” You use the appropriate instruments to take a look at it; what do you see? Eerily, you see it occupying a definite position. Born’s rule says that the position is a matter of chance: the probability that a particle appears in a certain place is proportional to the degree to which that place is represented in the mix.

It is as though the superposition is an extremely complex cocktail, a combination of various amounts of infinitely many ingredients, each representing the electron’s being in a particular place. Taste the cocktail, and instead of an infinitely complex flavor you will—according to Born’s rule—taste only a single ingredient. The chance of tasting that ingredient is proportional to the amount of the ingredient contained in the mixture that makes up the superposition. If an electron’s state is mostly a blend of positions near a certain atomic nucleus, for example, then when you observe it, it will most likely pop up near the nucleus.

One more thing: an observed particle’s apparently definite position is not merely a fleeting glimpse of something more complex. Once you see the particle in a certain position, it goes on to act as though it really is in that position (until something happens to change its state). In mixological terms, once you have sampled your cocktail, every subsequent sip will taste the same, as though the entire cocktail has transformed into a simple simple solution of this single ingredient. It is this strange disposition for matter, when observed, to snap into a determinate place that accounts for its “particle-like” behavior.

To sum up, quantum mechanical matter—the matter from which we’re all made—spends almost all its time in a superposition. As long as it’s not observed, the superposition, and so the matter, behaves like an old-fashioned wave, an exemplar of liquidity (albeit in indefinitely many dimensions). If it is observed, the matter jumps randomly out of its superposition and into a definite position like an old-fashioned particle, the epitome of solidity.

Nobody can explain what kind of substance this quantum mechanical matter is, such that it behaves in so uncanny a way. It seems that it can be neither solid nor fluid—yet these exhaust the possibilities that our human minds can grasp. Quantum mechanics does not, then, provide the kind of deep understanding of the way the world works that was sought by philosophers from Aristotle to Descartes. What it does supply is a precise mathematical apparatus for deriving effects from their causes. Take the initial state of a physical system, represented by a wave function; apply Schrödinger’s equation and if appropriate Born’s rule, and the theory tells you how the system will behave (with, if Born’s rule is invoked, a probabilistic twist). In this way, quantum theory explains why electrons sometimes behave as waves, why photons (the stuff of light) sometimes behave as particles, and why atoms have the structure that they do and interact in the way they do.

Thus, quantum mechanics may not offer deep understanding, but it can still account for observable phenomena by way of . . . the kind of explanation favored by Newton . . . Had Newton [engaged with scientists like Bohr and Einstein at conferences on quantum mechanics] he would perhaps have proclaimed:

I have not as yet been able to deduce from phenomena the nature of quantum superposition, and I do not feign hypotheses. It is enough that superposition really exists and acts according to the laws that we have set forth and is sufficient to explain all the motions of the microscopic bodies of which matter is made.

Newton . . .  was the chief architect of modern science’s first great innovation. Rather than deep philosophical understanding, Newton pursued shallow explanatory power, that is, the ability to derive correct descriptions of phenomena from a theory’s causal principles, regardless of their ultimate nature and indeed regardless of their very intelligibility. In so doing, he was able to build a gravitational theory of immense capability, setting an example that his successors were eager to follow.

Predictive power thereby came to override metaphysical insight. Or as the historian of science John Heilbron, writing of the study of electricity after Newton, put it:

When confronted with a choice between a qualitative model deemed intelligible and an exact description lacking clear physical foundations, the leading physicists of the Enlightenment preferred exactness.

So it continued to be, as the development and acceptance of quantum mechanics, as unerring as it is incomprehensible, goes to show. The criterion for explanatory success inherent in Newton’s practice became fixed for all time, founding the procedural consensus that lies at the heart of modern science.