Skip Nav


Navigation menu

❶Suppose you read an article about a Swedish study of a new exercise program for male workers with back pain.

Improving External Validity

Resource Links:
Threats to External Validity
Secondary navigation

Transferability is applied by the readers of research. Although generalizability usually applies only to certain types of quantitative methods, transferability can apply in varying degrees to most types of research. Unlike generalizability, transferability does not involve broad claims, but invites readers of research to make connections between elements of a study and their own experience.

For instance, teachers at the high school level might selectively apply to their own classrooms results from a study demonstrating that heuristic writing exercises help students at the college level. Generalizability and transferability are important elements of any research methodology, but they are not mutually exclusive: It is important for researchers to understand the implications of these twin aspects of research before designing a study.

Researchers who intend to make a generalizable claim must carefully examine the variables involved in the study. Among these are the sample of the population used and the mechanisms behind formulating a causal model. Calling this factor Z , we again average the z -specific effect of X on Y in the experimental sample, but now we weigh it by the "causal effect" of X on Z. A typical example of this nature occurs when Z is a mediator between the treatment and outcome, For instance, the treatment may be a cholesterol- reducing drug, Z may be cholesterol level, and Y life expectancy.

Here, Z is both affected by the treatment and a major factor in determining the outcome, Y. Suppose that subjects selected for the experimental study tend to have higher cholesterol levels than is typical in the general population.

The estimate obtained will be bias-free even when Z and Y are confounded — that is, when there is an unmeasured common factor that affects both Z and Y.

The precise conditions ensuring the validity of this and other weighting schemes are formulated in Bareinboim and Pearl, [14] and Bareinboim et al. In many studies and research designs, there may be a "trade-off" between internal validity and external validity: When measures are taken or procedures implemented aiming at increasing the chance for higher degrees of internal validity, these measures may also limit the generalizability of the findings.

This situation has led many researchers call for "ecologically valid" experiments. By that they mean that experimental procedures should resemble "real-world" conditions. They criticize the lack of ecological validity in many laboratory-based studies with a focus on artificially controlled and constricted environments. Some researchers think external validity and ecological validity are closely related in the sense that causal inferences based on ecologically valid research designs often allow for higher degrees of generalizability than those obtained in an artificially produced lab environment.

However, this again relates to the distinction between generalizing to some population closely related to concerns about ecological validity and generalizing across subpopulations that differ on some background factor. Some findings produced in ecologically valid research settings may hardly be generalizable, and some findings produced in highly controlled settings may claim near-universal external validity. Thus, external and ecological validity are independent — a study may possess external validity but not ecological validity, and vice versa.

Within the qualitative research paradigm, external validity is replaced by the concept of transferability. Transferability is the ability of research results to transfer to situations with similar parameters, populations and characteristics. It is common for researchers to claim that experiments are by their nature low in external validity. Some claim that many drawbacks can occur when following the experimental method. By the virtue of gaining enough control over the situation so as to randomly assign people to conditions and rule out the effects of extraneous variables, the situation can become somewhat artificial and distant from real life.

However, both of these considerations pertain to Cook and Campbell's concept of generalizing to some target population rather than the arguably more central task of assessing the generalizability of findings from an experiment across subpopulations that differ from the specific situation studied and people who differ from the respondents studied in some meaningful way.

Critics of experiments suggest that external validity could be improved by use of field settings or, at a minimum, realistic laboratory settings and by use of true probability samples of respondents.

However, if one's goal is to understand generalizability across subpopulations that differ in situational or personal background factors, these remedies do not have the efficacy in increasing external validity that is commonly ascribed to them. If background factor X treatment interactions exist of which the researcher is unaware as seems likely , these research practices can mask a substantial lack of external validity. Dipboye and Flanagan , writing about industrial and organizational psychology, note that the evidence is that findings from one field setting and from one lab setting are equally unlikely to generalize to a second field setting.

It depends in both cases whether the particular treatment effect studied would change with changes in background factors that are held constant in that study. If one's study is "unrealistic" on the level of some background factor that does not interact with the treatments, it has no effect on external validity. It is only if an experiment holds some background factor constant at an unrealistic level and if varying that background factor would have revealed a strong Treatment x Background factor interaction, that external validity is threatened.

Research in psychology experiments attempted in universities are often criticized for being conducted in artificial situations and that it cannot be generalized to real life. As noted above, this is in the hope of generalizing to some specific population.

Realism per se does not help the make statements about whether the results would change if the setting were somehow more realistic, or if study participants were placed in a different realistic setting. If only one setting is tested, it is not possible to make statements about generalizability across settings.

However, many authors conflate external validity and realism. There is more than one way that an experiment can be realistic:. This is referred to the extent to which an experiment is similar to real-life situations as the experiment's mundane realism. It is more important to ensure that a study is high in psychological realism —how similar the psychological processes triggered in an experiment are to psychological processes that occur in everyday life.

Psychological realism is heightened if people find themselves engrossed in a real event. To accomplish this, researchers sometimes tell the participants a cover story —a false description of the study's purpose. If however, the experimenters were to tell the participants the purpose of the experiment then such a procedure would be low in psychological realism.

That's the major thing you need to keep in mind. Recall that validity refers to the approximate truth of propositions, inferences, or conclusions. So, external validity refers to the approximate truth of conclusions the involve generalizations. Put in more pedestrian terms, external validity is the degree to which the conclusions in your study would hold for other persons in other places and at other times.

In science there are two major approaches to how we provide evidence for a generalization. I'll call the first approach the Sampling Model. In the sampling model, you start by identifying the population you would like to generalize to. Then, you draw a fair sample from that population and conduct your research with the sample.

Finally, because the sample is representative of the population, you can automatically generalize your results back to the population. There are several problems with this approach. First, perhaps you don't know at the time of your study who you might ultimately like to generalize to. Second, you may not be easily able to draw a fair or representative sample.

Third, it's impossible to sample across all times that you might like to generalize to like next year.

Main navigation

Main Topics

Privacy Policy

Start studying Research Methods: Sampling, Generalizability, and Inclusion/Exclusion Criteria. Learn vocabulary, terms, and more with flashcards, games, and other study tools.

Privacy FAQs

Refers to the generalizability of the results to other populations. Ex: occurs when the researcher cannot attain the ideal sample population.

About Our Ads

Nov 13,  · then, that generalizability in qualitative research refers to the extent to which theory developed within one study may be exported (K.M. Melia, personal communication) to provide explanatory theory for the experiences of other individuals who are in comparable situations. This position is supported by the comments of Popay et al. Generalizability as a folk notion of science Traditionally, generalizability refers to the ability to apply the results of research conducted on a sample of a population to a broader population (Babbie, ).

Cookie Info

the relevance of the research to professional practice. In the second and next section of this essay,we describe the misapplication (or perhaps,the overap-plication) of the concept of statistical,sampling-based generalizability in IS research. After that,we offer a critique of statistical,sampling-based generalizability. The results of this research often provide insights into how work and health interact in those groups. But how do we know if a study's results can be applied to another group or population? To answer this question, we first need to understand the concept of generalizability.