Uncertainty in scientific method: survey on subjective health

Research on the social determinants of good / poor health has a long tradition, in which the finding that those more privileged in society are generally in better health is one of the most consistent finding in social epidemiology (House,2002) ((J. S. House. Understanding social factors and inequalities in health: 20th century progress and 21st century prospects. Journal of health and social behavior, 43(2):125–42, Jul 2002.)) . In this research tradition many measures of health status have been developed, of which the SF-36 health survey (Ware,1993) ((J. Ware, K. Snow, M. Kosinski, and B. Gandek. Sf-36 health survey manual and interpretation guide. Technical report, MA: The Health Institute, New England Medical Center, 1993.)) is one of best validated subjective health measures in the field. One of the items of this measurement instrument is regarded as a very strong measure on its own (Mirowsky,2003) ((J. Mirowsky and C. E. Ross. Education, Social status and Health. Aldine de Gruyter, New York, 2003.)) . It states “In general, would you say your health is very good, good, satisfactory, poor, or very poor?“. This five-point likert scale is able to predict mortality risk better than many other indicators, amongst which physician health assessments and risk behavior such as smoking.

Not surprisingly, this single item measure is often used as an indicator of subjective health in large scale surveys, for instance used in comparative research searching for social determinants of health. This appliance is based on the methodological theory that every respondent is exposed to an identical stimulus (the survey-item) and that thereby differences in responses are caused only by different levels of some latent concept (here: subjective health) associated with the respondent and not by other influences. Thereby, it is also assumed that the same question means exactly the same for each respondent, meaning that all respondents answer from the same frame of reference (Linders, 1995; Philips,1973) ((H. Linders and E. Hijmans. Kwalitatief survey. In H. Hüttner, K. Renckstorf, and F. Wester, editors, Onderzoekstypen in de communicatiewetenschap, pages 539–558. Bohn Stafleu Van Loghum, 1995.)) ((D. Philips. Abandoning Method. Jossey-Bass, San Fransisco, 1973.)) . In this type of research, where the focus lies on being able to distinguish between (socio-economic) groups, these assumptions are made in an early phase of the research: before data are gathered, analyzed and conclusions are drawn. In terms of uncertainty these assumptions are part of the uncertainty about the method.

In another research project this assumption was not made in such an early stage. Using a qualitative design, Simon and her colleagues (Simon,2005) ((J. G. Simon, J. B. D. Boer, I. M. A. Joung, H. Bosma, and J. P. Mackenbach. How is your health in general? a qualitative study on self-assessed health. European journal of public health, 15(2):200–8, Jun 2005)) asked respondents exactly the same survey-item and recorded the indicated score. After that, she used an semi-structured interview technique to chart the respondents’ explanation of their own response. The transcripts of these explanations were analyzed in such a way that gradually an overall categorization scheme emerged. The single most interesting finding in this research is that people in different situations have a different kind of explanation for their answer on the survey item. Often in the explanation other dimensions than health experience came forward, such as the ability to cope with difficulties and the health history. Additionally, people with low scores on the subjective health item indicated to have had different considerations in answering the question than people with high scores.

What we see here, is that two types of research using exactly the same survey question, but make different assumptions at different moments during the process can complement each other when we allow assumptions to be discussed. In this case, the early made assumption that all respondents in survey research answer from an identical frame of reference to this specific survey question appears to be violated. This calls for an investigation of what the consequences of this are for the use of this item in survey research, so that perhaps these can be taken into account. But more importantly, this interplay of studies show how a violation of an important assumption can be discovered, which would not have been possible from the perspective of the comparative research only.

Simon, J. (2005). How is your health in general? A qualitative study on self-assessed health. The European Journal of Public Health, 15(2), 200-208. DOI: 10.1093/eurpub/cki102

Leave a Reply