Inductive negatives – when can you know apriori something cannot be the case?

This post is somewhat diffuse. It does not concern my field so much apart from isolated incidents, but I noticed some of these views in friends, family and beginning students.

What are inductive negatives? These are negative statements based on empirical evidence. As such they are quite problematic. Former secretary of defense Donald Rumsfeld gave a most wonderful and memorable Haiku about inductive negatives: “The absence of evidence does not equal evidence of absence.” (More egregiously though, he went on to suggest that no evidence was needed to assert that Saddam Hussein possessed weapons of mass destructions.)

The difficulty of negative empirical evidence is immediately clear when considering capital crimes like murder: it is hard to decisively prove that you were not at the crime scene at the time of the murder unless you have an alibi. The fact that you were definitely somewhere else during the crime implies necessarily that you cannot have been at the crime scene. The alibi turns the inductive into a deductive statement. Other deductive proofs of negatives can be found in mathematics: if you can show convincingly that something you want to disprove inevitably leads to a false statement, you have disproved it successfully. Usually the absence of evidence is not sufficient, and although nobody had ever found a counter example for Fermat’s conjecture the conjecture was not proved and did not become a theorem until 1994.

This logical structure of proof by contradiction though does not work for most real-world facts. Such facts are usually more like the “black swan problem”. For many years, people believed there were no black swans – until one day a black swan was sighted in Australia. It was a clear case of the confusion of absence of evidence for evidence of absence. There is no logical deduction possible that could be used to show that the assumption of any kind of swan color leads to a clear contradiction or logical impossibility; no sighting of any swan color, apart from black, implies anything about the presence of absence of black swans. A “proof” for the absence of black swans would thus have to be purely inductive, and such induction –other than in mathematics again- is necessarily incomplete.

This might sound quite academic. You can remember the short version: negative statements about empirical facts should perk up your ears. Most of my colleagues probably would not fall into the trap of brazen inductive negatives, but some lesser variants can be observed. First, there is skepticism about data, and, second, about quantitative models.

Skepticism about data is the least tenable of skepticisms: somebody alleging your data “are wrong”, is either directly challenging your integrity, or has a hard time accepting your data because they are inconsistent with a model view of the world that is very important to your challenger.

Fundamental skepticism about the endeavor of quantitative models is sometimes encountered with beginning students or with people outside the positive sciences. I have a friend, a sharp social scientist, who told me with some disgust about a quantitative model of election dynamics in a post-Soviet country. She seemed to be offended at the very idea of formulating such a quantitative model. When I pressed her why, she reacted strongly and gave a colorful expletive that professed her incredulity how such a model could ever do sufficient justice to reality. (I stopped pressing at that point.) I have witnessed similar reactions in seminars with advanced undergraduate and beginning graduate students.

First off, aesthetic or other diffuse disgust at the idea of quantitative models is misplaced for a rigorous scientific discipline. Second, a model of course cannot do justice to the full reality – that’s kind of the point (just as a map is not as large as the area it depicts). The model should capture one important aspect of reality and successfully predict that aspect out of sample. Whether it is the diagnostic severity for Alzheimer’s, or the party with a majority vote – the model ideally captures a few important ingredients for predictive success, and leaves out a lot of less important complexity. It should be as simple as possible.

To the diffuse feeling that “such a model would never work” we can just answer with a counter question: “How would you know? What Scholastic or Aristotelian certainty do you possess to be so convinced?” If the model indeed does not work, it will be clear in its lack of predictive success. Any good modeler should have empirical tests and information-theoretic criteria to avoid over-fitting in their tool kit. You should not have to worry that a quantitative model would be peddled that clearly does not work, unbeknown to its users: ascertainment of the empirical prediction success is already part of the model-building exercise. Also, you can check for yourself and don’t have to take it on faith from a higher authority.

Don’t get me wrong: I am also skeptical about the invasion of algorithms into every facet of life, with far-reaching consequences. The bar for empirical proof of successful prediction should be high indeed if human decision making is delegated to algorithms; important decisions in labeling of the “gold standard” in the training data for tuning such models should be discussed vigorously as well. All that is undoubtedly true. But meaningful criticism of the “algorithmicization” of everyday life and the hype cycle of buzzwords like “deep learning” needs better general stochastic and quantitative literacy. A gut feeling of “it would never work” is unfortunately not good enough. Some of the models might do some good too. Self-driving cars for instance might radically reduce the rate of traffic accidents. That’s a good thing, surely.