Shufflebrain

Carl came into my lab one day, arms loaded with sheaves of statisticated data. He spread them out on a long high lab bench, then tried to talk. Instead he began to giggle. The giggle turned into a laugh. The laugh became a fit. I feared he might collapse into a tall stack of dirty dishes on the soapstone sink behind him. Finally, he managed to tell me some of what was in the findings.

One-Eye (the mean avoidances for animals minus a natural eye) had indeed learned more slowly than Two-Eye, which sounded just fine to me. And Triclops (the mean values for animals with the third eye mounted on the head) learned faster than Two-Eye. Well great! I thought. Moreover, the One-Eye versus Two-Eye difference when added back to Two-Eye did equal Triclops, which seemed marvelous to me!

"What's so funny?" I asked, puzzled. It's swell to be delighted but that shouldn't kick you into a hebephrenic jag. Carl supported himself against the gray steel fume hood next to the sink. "Heee..." Now I'm also chuckling, although I don't know why. "Cyclops...[gasp!]...Cyclops learned faster than all the rest...Heeeeee..." Carl managed one last clause before his puffed eyes closed tight and he became incoherent: the Cyclops data were statistically significant. "Very highly significant..." And then I broke down, too.

***

Why was all this so hilarious to us? One-Eye, Two-Eye and Triclops had behaved according to our a priori expectations. But Cyclops, with one transplanted eye, should have learned at about the same rate as One-Eye ("plus or minus a shkoshi bit to account for the location", as I told our phamacologist). But certainly not faster than Triclops! Our results looked flukey. Mother Nature had played a practical joke on us, it seemed. And she was rubbing it in with statistics--a very high level of statistical significance, at that. It was the statistics that made Carl, and eventually me, come apart. The Cyclops data look preposterous. And statistics turned them into absrudity.

I'm not saying it's funny when ridiculous results become statistically significant. That happens all the time, really. Indeed, it's tragic when statistics prevent a scientist from recognizing absurdity. This too frequently happens.

Statistics furnish a rational test for whether or not differences can be accounted for merely from random individual variations within a population. The tests don't guarantee a difference. Also, statistical correlations let an observer decide on formal grounds rather than intuitively, what the random chances are of two sets of events occurring together, given normal variations among the samples. And statistics let us compare results against, say, the honest roulette wheel: they give the odds against winning when we bet on an alleged difference or correlation. There's really no way to conduct quantitative research with "stats."

But statistics aren't the same thing as truth. They're not the same thing as being right or wrong. And the term significance only refers to how many times in a hundred, thousand, million, etc., you can obtain the same result at an unrigged crap table. People have been known to roll eight or nine sevens in a row, which isn't very likely, statistically. An application of a given statistical test to a body of data may unwittingly violate unknown mathematical conditions. Cross the wrong abstract boundary and you may quickly generate absurdity without knowing it and without being able to control the source of the error. (I've often wondered how many heavy users of statistics have actually examined the theorems and proofs underlying their tests.) Then too, there's the matter of criteria. Much IQ gospel and parapsychology data, for instance, depend on level of significance and coefficients of correlation that would be useless in, say, quantum chemistry or statistical mechanics. On the other extreme, some of the most important discoveries in the history of science are statistically insignificant: Few of Pasteur's experiments were sufficiently replicated or adequately sampled for commonly used statistical tests. None of Koch's were. Nor Galileo's or Newton's.