Scientific practice and scientific progress: Integration and testing of rival hypotheses

In school I still learned that it is good argumentative practice (and a better strategy) to deal with potential counter arguments and criticisms by taking them apart in the course of your argument. Science is about making decisions about rival hypotheses, interpretations of evidence based on a set of observations, tests, experiments (depending on in which field you work and what is feasible). Thus, it is good scientific practice to compare rival hypotheses / theories (i.e. systems of interpretation of facts, data, or other evidence), by testing them with a set of data.

Rival hypotheses might (or rather often are indeed) special cases of an underlying reality. There is some truth in all observations, some are better than others, some are more suitable than others given a specific context. Think of the parts of an elephant that is examined by several blind men. Everyone comes up with different observations and theories about “reality”. 

That is what we actually often can observe in scientific practice: ideologically 'blinded' representatives of schools of thoughts mindlessly hurling arguments about “reality” at each other – based on selective interpretations of data – without looking for an integrative theory. 

Thus systems of scientific thought have the ability to press observations into procrustes beds that seem to lead to different “proven” true interpretations of reality – which are thus artifacts of more or less subtle differences in scientific worldviews, i.e. perspectives on the underlying reality .

Integration of opposing, rival views can be achieved if scientists (just as “ordinary” people) are able to switch their perspective and manage to develop dialectically the synthesis from thesis and antithesis. A set of literary examples that nicely shows how the integrative method works are Arthur Conan Doyle’s stories about Sherlock Holmes. Holmes generates a number of partial hypotheses based on the integration of facts known so far - which are proven 'wrong' by the some new detail until he stumbles across the truth by some coincidence. 

(Arthur Conan Doyle was incidentally influenced by Charles S. Peirce’ pragmatist philosophy, which stresses abduction (something akin to intuition) as source of knowledge. I cannot claim to be an expert on Peirce, but my understanding of Peirce’ abduction is that it is this process of generating new knowledge by integrating controversial elements into a larger picture.)

Resurrected and revised post from the Organization and Markets blog a few years ago.