Critics of sociology stating that the discipline has no ‘excess empirical content’, ignore efforts of methodological rigor without which ‘anything goes’

It is sometimes said that sociology is particularly well fit to kick in open doors. It is a response to sociological research findings is that people think they already knew the outcome of this study. Sure, sometimes this may be the case, but most often one may have a suspicion about the outcomes – rather than actual knowledge.

In philosophical sense, this criticism boils down to arguin that most sociological theories have no excess content: they have nothing new to add to what we already knew. One argued that is often heard is that in sociology so many methods are used that one can always find a method – or alter one – to support one’s own ideas: sociology would lack methodological rigor.

The term ‘excess empirical content’ was coined by Imre Lakatos. In my dissertation I write about my philosophy of science:

Based on Lakatosian philisophy of science, the interaction between opportunities and interests means that we must reject explanations of women’s employment solely based on the concept of opportunities. Lakatos argued that: “For the sophisticated falsificationist a scientific the- ory T is falsified if and only if another theory T’ has been proposed with the following characteristics: (1) T’ has excess empirical content over T: that is, it predicts novel facts, that is, facts improbable in the light of, or even forbidden, by T; (2) T’ explains the previous success of T, that is, all the unrefuted content of T is included (within the limits of observational error) in the content of T’; and (3) some of the excess content of T’ is corroborated.” (Lakatos, 1978, p. also see: Levels & Nieuwenhuis, 2011).

Paul Feyerabend argued for methodological pluralism, stating that ‘anything goes’. This has been interpreted as indicating that any method should be endorsed, or tolerated. I do not think this is a correct interpretation. I interpret Feyerabend as suggesting that one should look at problems in different ways – and see whether news ideas reject old conceptions of truth. The methodological rigor is then found in the attempt to relate the findings based on different methodologies to each other – within a common theoretical framework.

In my dissertation I have closed three open doors, so to say, or rejected (at least) three theories:

• Pettit and Hook (2009) argued that (parental) leave is a mechanism of exclusion of women from the labor market. In my analyses I found that short-term leave includes women on the labor market, and only very long periods of leave exclude women from the labor market
• As argued in the quote above, I reject explanations of the outcomes of family policies solely based on opportunities, as it is the interaction of opportunities and women’s interest in employment that drive the outcomes of family policies.
• The incomplete revolution thesis by Esping-Andersen, who argued that increasing rates of women’s employment resulted in higher degrees of inequality between households. I could not confirm this claim, and rather argued that “The conditions for women’s earnings to increase inequality between households are hard to meet“.

——
This is a series on the 10 propositions that are part of my PhD dissertation. These propositions are a Dutch tradition to highlight key findings of a dissertation and some additional insights by the author. My dissertation is titled “Family Policy Outcomes: Combining Institutional and Demographic Explanations of Women’s Employment and Earnings Inequality in OECD countries, 1975-2005″ and I will defend my dissertation on January 10 2014. So, this series is also a count down. Find out more about my dissertation.

In the world as we understand it, based on Einstein, nothing can go faster than light. This prediction based on the general theory of relativity has proven itself countless times in empirical research. And now, lo and behold, a group at CERN has observed neutrino’s racing through earth from France/Switzerland to Italy at the World-record breaking speed of slightly above light-speed! This finding has received a lot of coverage in the media. Why is this so interesting?

Of course this research finding has given rise to wild speculations, including the possibility of time travel (would Richard Branson be willing to start a company on this one?). Various wild theories have already been formulated that possibly explain how both Einstein and the CERN-researchers can be right. It could be the case that further replications will show that in fact neutrino’s cannot exceed light speed. And finally it could eventually turn out that Einstein was wrong.

The observation of these speeding neutrino’s contradicts our best understanding of physical reality. We simply cannot explain these observations. What I find the most interesting about this all, are the huge efforts of scholars really trying to understand and especially to replicate these highly counterintuitive observations. The group at CERN has done its own replications, accounting for some of the criticism on the original approach, and calls for other groups to do the same. This response to conflicts between theory and empirical reality is to me a sign of a healthy scientific practice.

The social sciences do not deal with neutrino’s, but can learn a lot from this scientific practice. Recently the Dutch science community was stirred up by a very serious case of fraud in social-psychology research, with a prime investigator faking experimental data. Enough has been said about this, and there is no need to discuss it all over again here. A comment by one of his non-fraudulent co-authors, however, was very interesting. This co-author was given a table with the results of analyses on a new set of data (that eventually turned out to be made up completely, but she did not know that at the time), and used that information to test their hypotheses. The results supported the hypotheses, as a result of which, she argued, she didn’t question the validity of the data as much as she should have.

Now we know how unfortunate that turned out. But there is a point in this: people, including scientists, tend to be less critical when what we observe aligns with what we know. In the social sciences this all too often results in a tendency to find support for theories. And while any theory needs some level of support, the consequence of this practice is that all too often many theories are formulated on a specific phenomenon that are intrinsically different, while no tests are performed which of the theories actually performs best. Factors such as this confirmation bias, but also other factors such as the theory-ladenness of observations, the unmeasured variable problem, our tendency to downgrade the perceived importance of exceptions, and our mind that is trained to immediately formulate ad-hoc explanations for observations, dictate that we cannot simply base a scientific discipline on confirmation and illustration.

So, in line with the response to the speeding neutrino’s, social science could benefit from more replications, more attempts to reject theories, fewer attempts to ‘prove’ theories by finding supportive evidence, and especially more attention to observations that do not fit a theory. Instead of yet another confirmation, solving puzzles, contradictions, and paradoxes should be the principal task of the social sciences.

The OPERA Collaboraton: T. Adam et al. (2011). Measurement of the neutrino velocity with the OPERA detector in the CNGS beam Arxiv arXiv: 1109.4897v2