126 ■ Chapter 5: Sampling Logic giving the Republican contender, Alf Landon, a or may not have correctly represented the voting stunning 57 to 43 percent landslide over the in- intentions of telephone subscribers and automobile cumbent, President Franklin Roosevelt. The editors owners. Unfortunately for the editors, it decidedly modestly cautioned, did not represent the voting intentions of the popu- lation as a whole. We make no claim to infallibility. We did not coin the phrase “uncanny accuracy” which has President Thomas E. Dewey been so freely applied to our Polls. We know only too well the limitations of every straw The 1936 election also saw the emergence of a vote, however enormous the sample gathered, young pollster whose name would become syn- however scientific the method. It would be a onymous with public opinion. In contrast to the miracle if every State of the forty-eight behaved Literary Digest, George Gallup correctly predicted on Election Day exactly as forecast by the Poll. that Roosevelt would beat Landon. Gallup’s suc- cess in 1936 hinged on his use of something called (Literary Digest 1936a: 6) quota sampling, which we’ll look at more closely later in the chapter. For now, it’s enough to know Two weeks later, the Digest editors knew the that quota sampling is based on a knowledge of limitations of straw polls even better: The v oters the characteristics of the population being sampled: gave Roosevelt a second term in office by the what proportion are men, what proportion are largest landslide in history, with 61 percent of women, what proportions are of various incomes, the vote. Landon won only 8 electoral votes to ages, and so on. Quota sampling selects people to Roosevelt’s 523. match a set of these characteristics: the right num- ber of poor, white, rural men; the right number of The editors were puzzled by their unfortunate rich, African American, urban women; and so on. turn of luck. A part of the problem surely lay in the The quotas are based on those variables most rel- 22 percent return rate garnered by the poll. The evant to the study. In the case of Gallup’s poll, the editors asked, sample selection was based on levels of income; the selection procedure ensured the right proportion of Why did only one in five voters in Chicago to respondents at each income level. whom the Digest sent ballots take the trouble to reply? And why was there a preponderance of Gallup and his American Institute of Public Republicans in the one-fifth that did reply? . . . Opinion used quota sampling to good effect in We were getting better cooperation in what 1936, 1940, and 1944—correctly picking the presi- we have always regarded as a public service dential winner each of those years. Then, in 1948, from Republicans than we were getting from Gallup and most political pollsters suffered the em- Democrats. Do Republicans live nearer to mail- barrassment of picking Governor Thomas Dewey boxes? Do Democrats generally disapprove of of New York over the incumbent, President Harry straw polls? Truman. The pollsters’ embarrassing miscue con- tinued right up to election night. A famous photo- (Literary Digest 1936b: 7) graph shows a jubilant Truman—whose followers’ battle cry was “Give ’em hell, Harry!”—holding Actually, there was a better explanation—what aloft a newspaper with the banner headline is technically called the sampling frame used by the “Dewey Defeats Truman.” Digest. In this case, the sampling frame consisted of telephone subscribers and automobile own- Several factors accounted for the pollsters’ ers. In the context of 1936, this design selected a failure in 1948. First, most pollsters stopped poll- disproportionately wealthy sample of the voting ing in early October despite a steady trend toward population, especially coming on the tail end of T ruman during the campaign. In addition, many the worst economic depression in the nation’s his- voters were undecided throughout the campaign, tory. The sample effectively excluded poor people, and the poor voted predominantly for Roosevelt’s New Deal recovery program. The Digest’s poll may
A Brief History of Sampling ■ 127 W. Eugene Smith/Time & Life Pictures/Getty Images Based on early political polls that showed Dewey leading Truman, the Chicago Tribune sought to scoop the competition with this unfortunate headline. and these went disproportionately for Truman Two Types of Sampling Methods when they stepped into the voting booth. By 1948, some academic researchers had already More important, Gallup’s failure rested been experimenting with a form of sampling based on the unrepresentativeness of his samples. Quota on probability theory. This technique involves the sampling—which had been effective in earlier selection of a “random sample” from a list contain- years—was Gallup’s undoing in 1948. This tech- ing the names of everyone in the population being nique requires that the researcher know something sampled. By and large, the probability-sampling about the total population (of voters in this in- methods used in 1948 were far more accurate than stance). For national political polls, such informa- quota-sampling techniques. tion came primarily from census data. By 1948, however, World War II had produced a massive Today, probability sampling remains the pri- movement from the country to cities, radically mary method of selecting large, representative changing the character of the U.S. population from samples for social research, including national what the 1940 census showed, and Gallup relied on political polls. At the same time, probability sam- 1940 census data. City dwellers, moreover, tended pling can be impossible or inappropriate in many to vote Democratic; hence, the overrepresentation research situations. Accordingly, before turning of rural voters in his poll had the effect of underes- to the logic and techniques of probability sam- timating the number of Democratic votes. pling, we’ll first take a look at techniques for
128 ■ Chapter 5: Sampling Logic nonprobability sampling and how they’re used in are not feasible. Even when this method is justified social research. on grounds of feasibility, researchers must exercise great caution in generalizing from their data. Also, Nonprobability Sampling they should alert readers to the risks associated with this method. Social research is often conducted in situations that do not permit the kinds of probability samples used University researchers frequently conduct sur- in large-scale social surveys. Suppose you wanted veys among the students enrolled in large lecture to study homelessness: There is no list of all home- classes. The ease and frugality of such a method less individuals, nor are you likely to create such a e xplains its popularity, but it seldom produces data list. Moreover, as you’ll see, there are times when of any general value. It may be useful for pretest- probability sampling wouldn’t be appropriate even ing a questionnaire, but such a sampling method if it were possible. Many such situations call for should not be used for a study purportedly describ- nonprobability sampling. ing students as a whole. In this section, we’ll examine four types of Consider this report on the sampling design in nonprobability sampling: reliance on available an examination of knowledge and opinions about subjects, purposive (judgmental) sampling, snow- nutrition and cancer among medical students and ball sampling, and quota sampling. We’ll conclude family physicians: with a brief discussion of techniques for obtaining information about social groups through the use of The fourth-year medical students of the informants. U niversity of Minnesota Medical School in Minneapolis comprised the student popula- Reliance on Available Subjects tion in this study. The physician population consisted of all physicians attending a “Family Relying on available subjects, such as stopping Practice Review and Update” course sponsored people at a street corner or some other location, is by the University of Minnesota Department of sometimes called “convenience” or “haphazard” Continuing Medical Education. sampling. This is a common method for journalists in their “person-on-the-street” interviews, but it (Cooper-Stephenson and Theologides 1981: 472) is an extremely risky sampling method for social research. Clearly, this method does not permit any After all is said and done, what will the results control over the representativeness of a sample. It’s of this study represent? The data do not provide a justified only if the researcher wants to study the meaningful comparison of medical students and characteristics of people passing the sampling point family physicians in the United States or even in at specified times or if less-risky sampling methods Minnesota. Who were the physicians who attended the course? We can guess that they were probably nonprobability sampling Any technique in more concerned about their continuing educa- which samples are selected in some way not sug- tion than other physicians were, but we can’t say gested by probability theory. Examples include for sure. Although such studies can provide useful reliance on available subjects as well as purposive insights, we must take care not to overgeneralize (judgmental), quota, and snowball sampling. from them. purposive (judgmental) sampling A type of nonprobability sampling in which the units to be Purposive or Judgmental observed are selected on the basis of the researcher’s Sampling judgment about which ones will be the most useful or representative. Sometimes it’s appropriate to select a sample on the basis of knowledge of a population, its elements, and the purpose of the study. This type of sampling is called purposive or judgmental sampling. In the initial design of a questionnaire, for example,
Nonprobability Sampling ■ 129 you might wish to select the widest variety of course of your analysis of the earlier interviews, respondents to test the broad applicability of ques- you may find several references to interactions tions. Although the study findings would not with faculty members in one of the social science represent any meaningful population, the test run departments. As a consequence, you may expand might effectively uncover any peculiar defects in your sample to include faculty in that department your questionnaire. This situation would be consid- and other students that they interact with. This is ered a pretest, however, rather than a final study. called “theoretical sampling,” since the evolving theoretical understanding of the subject directs the In some instances, you may wish to study a sampling in certain directions. small subset of a larger population in which many members of the subset are easily identified, but the Snowball Sampling enumeration of them all would be nearly impos- sible. For example, you might want to study the Another nonprobability sampling technique, which leadership of a student protest movement; many some consider to be a form of accidental sampling, of the leaders are easily visible, but it would not is called snowball sampling. This procedure is be feasible to define and sample all the leaders. In appropriate when the members of a special popu- studying all or a sample of the most visible leaders, lation are difficult to locate, such as homeless you may collect data sufficient for your purposes. individuals, migrant workers, or undocumented immigrants. In snowball sampling, the researcher Or let’s say you want to compare left-wing and collects data on the few members of the target right-wing students. Because you may not be able population he or she can locate, then asks those to enumerate and sample from all such students, individuals to provide the information needed to you might decide to sample the memberships of locate other members of that population whom left- and right-leaning groups, such as the Green they happen to know. “Snowball” refers to the Party and the Tea Party. Although such a sample process of accumulation as each located subject design would not provide a good description of suggests other subjects. Because this procedure also either left-wing or right-wing students as a whole, results in samples with questionable representative- it might suffice for general comparative purposes. ness, it’s used primarily for exploratory purposes. Field researchers are often particularly in- Suppose you wish to learn a community terested in studying deviant cases—cases that organization’s pattern of recruitment over time. You don’t fit into fairly regular patterns of attitudes and might begin by interviewing fairly recent recruits, behaviors—in order to improve their understand- asking them who introduced them to the group. ing of the more-regular pattern. For example, you You might then interview the people named, ask- might gain important insights into the nature of ing them who introduced them to the group. You school spirit, as exhibited at a pep rally, by inter- might then interview those people named, asking, viewing people who did not appear to be caught in part, who introduced them. Or, in studying a up in the emotions of the crowd or by interviewing loosely structured political group, you might ask students who did not attend the rally at all. Select- one of the participants who he or she believes to ing deviant cases for study is another example of be the most influential members of the group. You purposive study. might interview those people and, in the course of the interviews, ask who they believe to be the most In qualitative research projects, the sampling influential. In each of these examples, your sample of subjects may evolve as the structure of the situ- ation being studied becomes clearer and certain snowball sampling A nonprobability sampling types of subjects seem more central to understand- method, often employed in field research, whereby ing than others do. Let’s say you’re conducting an each person interviewed may be asked to suggest interview study among the members of a radical additional people for interviewing. political group on campus. You may initially focus on friendship networks as a vehicle for the spread of group membership and participation. In the
130 ■ Chapter 5: Sampling Logic would “snowball” as each of your interviewees various age categories, educational levels, ethnic suggested other people to interview. groups, and so forth. In establishing a national quota sample, you might need to know what pro- Examples of this technique in social science portion of the national population is urban, eastern, research abound. Karen Farquharson (2005) male, under 25, white, working class, and the like, provides a detailed discussion of how she used and all the possible combinations of these attributes. snowball sampling to discover a network of tobacco policy makers in Australia: both those at the core Once you’ve created such a matrix and assi of the network and those on the periphery. gned a relative proportion to each cell in the Kath Browne (2005) used snowballing through matrix, you proceed to collect data from people social networks to develop a sample of non having all the characteristics of a given cell. You heterosexual women in a small town in the United then assign to all the people in a given cell a weight Kingdom. She reports that her own membership appropriate to their portion of the total population. in such networks greatly facilitated this type of When all the sample elements are so weighted, the sampling, and that potential subjects in the study overall data should provide a reasonable represen- were more likely to trust her than to trust hetero- tation of the total population. sexual researchers. Although quota sampling resembles probability In more-general, theoretical terms, Chaim Noy sampling, it has several inherent problems. First, argues that the process of selecting a snowball sam the quota frame (the proportions that different cells ple reveals important aspects of the populations represent) must be accurate, and it’s often difficult being sampled, uncovering “the dynamics of natu- to get up-to-date information for this purpose. The ral and organic social networks” (2008: 329). Do Gallup failure to predict Truman as the presiden- the people you interview know others like them- tial victor in 1948 was due partly to this problem. selves? Are they willing to identify those people to Second, the selection of sample elements within researchers? Thus, snowball sampling can be more a given cell may be biased even though its pro- than a simple technique for finding people to study. portion of the population is accurately estimated. It can be a revealing part of the inquiry. Instructed to interview five people who meet a given, complex set of characteristics, an interviewer Quota Sampling may still avoid people living at the top of seven-story walk-ups, having particularly run-down homes, or Quota sampling is the method that helped owning vicious dogs. George Gallup avoid disaster in 1936—and set up the disaster of 1948. Like probability sampling, In recent years, attempts have been made to quota sampling addresses the issue of representa- combine probability- and quota-sampling meth- tiveness, although the two methods approach the ods, but the effectiveness of this effort remains to issue quite differently. be seen. At present, you would be advised to treat quota sampling warily if your purpose is statistical Quota sampling begins with a matrix, or table, description. describing the characteristics of the target popula- tion. Depending on your research purposes, you At the same time, the logic of quota sampling may need to know what proportion of the popula- can sometimes be applied usefully to a field re- tion is male and what proportion female, as well search project. In the study of a formal group, for as knowing what proportions of each sex fall into example, you might wish to interview both lead- ers and nonleaders. In studying a student political quota sampling A type of nonprobability sampling organization, you might want to interview radical, in which units are selected into a sample on the basis moderate, and conservative members of that group. of prespecified characteristics, so that the total sample You may be able to achieve sufficient representa- will have the same distribution of characteristics tiveness in such cases by using quota sampling to assumed to exist in the population being studied. ensure that you interview both men and women, both younger and older people, and so forth.
Nonprobability Sampling ■ 131 Selecting Informants almost always be somewhat “marginal” or atypi- cal within their group. Sometimes this is obvious. When field research involves the researcher’s at- Other times, however, you’ll learn about their mar- tempt to understand some social setting—a juvenile ginality only in the course of your research. gang or local neighborhood, for example—much of that understanding will come from a collaboration In Jeffrey Johnson’s study, a county agent with some members of the group being studied. identified one fisherman who seemed squarely in Whereas social researchers speak of respondents as the mainstream of the community. Moreover, he people who provide information about themselves, was cooperative and helpful to Johnson’s research. allowing the researcher to construct a composite The more Johnson worked with the fisherman, picture of the group those respondents represent, however, the more he found the man to be a mar- an informant is a member of the group who can ginal member of the fishing community. talk directly about the group per se. First, he was a Yankee in a southern town. Especially important to anthropologists, infor- Second, he had a pension from the Navy [so he mants are important to other social researchers as was not seen as a “serious fisherman” by others well. If you wanted to learn about informal social in the community]. . . . Third, he was a major networks in a local public-housing project, for Republican activist in a mostly Democratic example, you would do well to locate individuals village. Finally, he kept his boat in an isolated who could understand what you were looking for anchorage, far from the community harbor. and help you find it. (1990: 56) When Jeffrey Johnson (1990) set out to study a salmon-fishing community in North Carolina, Informants’ marginality may not only bias the view he used several criteria to evaluate potential infor- you get, but their marginal status may also limit mants. Did their positions allow them to interact their access (and hence yours) to the different sec- regularly with other members of the camp, for ex- tors of the community you wish to study. ample, or were they isolated? (In this case, he found that the carpenter had a wider range of interactions These comments should give you some sense than the boat captain did.) Was their information of the concerns involved in nonprobability sam- about the camp pretty much limited to their specific pling, typically used in qualitative research projects. jobs, or did it cover many aspects of the operation? I conclude with the following injunction: These and other criteria helped determine how use- ful the potential informants might be. Your overall goal is to collect the richest possible data. By rich data, we mean a wide and diverse Usually, you’ll want to select informants some- range of information collected over a relatively what typical of the groups you’re studying. Other prolonged period of time in a persistent and wise, their observations and opinions may be mis- systematic manner. Ideally, such data enable leading. Interviewing only physicians will not give you to grasp the meanings associated with the you a well-rounded view of how a community medi- actions of those you are studying and to under- cal clinic is working, for example. Along the same stand the contexts in which those actions are lines, an anthropologist who interviews only men in embedded. a society where women are sheltered from outsiders will get a biased view. Similarly, although informants (Lofland et al. 2006: 15) fluent in English are convenient for English-speaking researchers from the United States, they do not typify In other words, nonprobability sampling does the members of many societies nor even many sub- have its uses, particularly in qualitative research groups within English-speaking countries. projects. But researchers must take care to Simply because they’re the ones willing to informant Someone who is well versed in the work with outside investigators, informants will social phenomenon that you wish to study and who is willing to tell you what he or she knows about it. Not to be confused with a respondent.
132 ■ Chapter 5: Sampling Logic acknowledge the limitations of nonprobability FIGURE 5-2 Ceng sampling, especially regarding accurate and precise Babbie representations of populations. This point will be- A Population of 100 Folks. Typically, sampling aims to reflect the come clearer as we discuss the logic and techniques characteristics and dynamics of large populations. For the purpose of Social of probability sampling. some simple illustrations, let’s assume our total population only has 100 members. 1-133-04 The Theory and Logic of Probability Sampling variations that exist in the population. This isn’t as simple as it might seem, however. Let’s take a min- However appropriate to some research purposes, ute to look at some of the ways researchers might nonprobability sampling methods cannot guarantee go astray. Then, we’ll see how probability sampling that the sample we observed is representative of the provides an efficient method for selecting a sample whole population. When researchers want precise, that should adequately reflect variations that exist statistical descriptions of large populations—for in the population. example, the percentage of the population who is unemployed, plan to vote for Candidate X, or feel a Conscious and Subconscious rape victim should have the right to an abortion— Sampling Bias they turn to probability sampling. All large-scale surveys use probability-sampling methods. At first glance, it may look as though sampling is pretty straightforward. To select a sample of Although the application of probability sam- 100 university students, you might simply inter pling involves some sophisticated use of statistics, view the first 100 students you find walking the basic logic of probability sampling is not difficult around campus. This kind of sampling method is to understand. If all members of a population were often used by untrained researchers, but it runs a identical in all respects—all demographic charac- high risk of introducing biases into the samples. teristics, attitudes, experiences, behaviors, and so on—there would be no need for careful sampling In connection with sampling, bias simply procedures. In this extreme case of perfect homoge- means that those selected are not typical or repre- neity, in fact, any single case would suffice as a sam- sentative of the larger populations they have been ple to study characteristics of the whole population. chosen from. This kind of bias does not have to be intentional. In fact, it is virtually inevitable when In fact, of course, the human beings who you pick people by the seat of your pants. compose any real population are quite heteroge- neous, varying in many ways. Figure 5-2 offers a Figure 5-3 illustrates what can happen when simplified illustration of a heterogeneous popula- researchers simply select people who are conve- tion: The 100 members of this small population nient for study. Although women are only 50 per- differ by sex and race. We’ll use this hypothetical micropopulation to illustrate various aspects of probability sampling. The fundamental idea behind probability sampling is this: To provide useful descriptions of the total population, a sample of individuals from a population must contain essentially the same probability sampling The general term for samples selected in accord with probability theory, typically involving some random-selection mechanism. Specific types of probability sampling include EPSEM, PPS, simple random sampling, and systematic sampling.
The Theory and Logic of Probability Sampling ■ 133 FIGURE 5-3 A Sample of Convenience: Easy, but Not Representative. Simply selecting and observing those people who are most readily at hand is the simplest method, perhaps, but it’s unlikely to provide a sample that accurately reflects the total population. the researcher (in the lower right corner) happen university library, you could not be sure of a rep- to be 70 percent women, and although the popu- resentative sample, because different types of stu- lation is 12 percent African American, none was dents visit the library with different frequencies. selected into the sample. Your sample would overrepresent students who visit the library more often than others do. Beyond the risks inherent in simply studying people who are convenient, other problems can The possibilities for inadvertent sampling bias arise. To begin with, the researcher’s personal lean- are endless and not always obvious. Fortunately, ings may affect the sample to the point where it many techniques can help us avoid bias. does not truly represent the student population. Suppose you’re a little intimidated by students who Representativeness and look particularly “cool,” feeling they might ridicule Probability of Selection your research effort. You might consciously or subconsciously avoid interviewing such people. Or, Although the term representativeness has you might feel that the attitudes of “super-straight- no precise, scientific meaning, it carries a looking” students would be irrelevant to your research purposes and so avoid interviewing them. representativeness That quality of a sampCle eofn g a g e L e a r n i n g Even if you sought to interview a “balanced” having the same distribution of characteristiBcsaabs tbhiee: The Practice of group of students, you wouldn’t know the exact tpioopnu, ldaetsiocrnipftrioomnswanhdichexipt lwanasatsieolnesctdeedr.ivBeydimfrSopmloiccaa-inal Research, 13/e proportions of different types of students making Fig. 7-3 up such a balance, and you wouldn’t always be analysis of the sample may be assumed to re1p-r1e3s3en-0t4979-6 able to identify the different types just by watching them walk by. similar ones in the population. Representativeness is Even if you made a conscientious effort to enhanced by probability sampling and provides for generalizability and the use of inferential statistics.
134 ■ Chapter 5: Sampling Logic commonsense meaning that makes it useful here. a nonprobability sample to be representative of the For our purpose, a sample is representative of the population from which it is drawn. population from which it is selected if the aggregate characteristics of the sample closely approximate Second, and more important, probability the those same aggregate characteristics in the popula- ory permits us to estimate the accuracy or rep- tion. If, for example, the population contains 50 resentativeness of the sample. Conceivably, an percent women, then a sample must contain “close uninformed researcher might, through wholly to” 50 percent women to be representative. Later, haphazard means, select a sample that nearly per- we’ll discuss “how close” in detail. fectly represents the larger population. The odds are against doing so, however, and we would be Note that samples need not be representative unable to estimate the likelihood that he or she in all respects; representativeness is limited to those has achieved representativeness. The probability characteristics that are relevant to the substantive sampler, on the other hand, can provide an acc interests of the study. However, you may not know urate estimate of success or failure. We’ll see in advance which characteristics are relevant. exactly how this estimate can be achieved. A basic principle of probability sampling is that I’ve said that probability sampling ensures that a sample will be representative of the population samples are representative of the population we from which it is selected if all members of the pop wish to study. As we’ll see in a moment, probability ulation have an equal chance of being selected in sampling rests on the use of a random-selection the sample. (We’ll see shortly that the size of the procedure. To develop this idea, though, we need sample selected also affects the degree of represen- to give more-precise meaning to two important tativeness.) Samples that have this quality are often terms: element and population.* labeled EPSEM samples (EPSEM stands for “equal probability of selection method”). Later, we’ll dis- An element is that unit about which infor- cuss variations of this principle, which forms the mation is collected and that provides the basis of basis of probability sampling. analysis. Typically, in survey research, elements are people or certain types of people. However, other Moving beyond this basic principle, we must kinds of units can constitute the elements for social realize that samples—even carefully selected EPSEM research: Families, social clubs, or corporations samples—seldom if ever perfectly represent the pop- might be the elements of a study. In a given study, ulations from which they are drawn. Nevertheless, elements are often the same as units of analysis, probability sampling offers two special advantages. though the former are used in sample selection and the latter in data analysis. First, probability samples, although never perfectly representative, are typically more rep- Up to now we’ve used the term population to resentative than other types of samples, because mean the group or collection that we’re interested the biases previously discussed are avoided. In in generalizing about. More formally, a population is practice, a probability sample is more likely than the theoretically specified aggregation of study ele- ments. Whereas the vague term Americans might be EPSEM (equal probability of selection the target for a study, the delineation of the popu- method) A sample design in which each member of lation would include the definition of the element a population has the same chance of being selected Americans (for example, citizenship, residence) and into the sample. the time referent for the study (Americans as of when?). Translating the abstract “adult New York- element That unit of which a population is com- ers” into a workable population would require a posed and which is selected in a sample. Distin- guished from units of analysis, which are used in data *I would like to acknowledge a debt to Leslie Kish and analysis. his excellent textbook Survey Sampling. Although I’ve modified some of the conventions used by Kish, his population The theoretically specified aggregation presentation is easily the most important source of this of the elements in a study. discussion.
The Theory and Logic of Probability Sampling ■ 135 specification of the age defining adult and the heads or tails), the “selection” of a head or a tail boundaries of New York. Specifying the term college is independent of previous selections of heads or student would include a consideration of full- and tails. No matter how many heads turn up in a row, part-time students, degree candidates and non the chance that the next flip will produce “heads” degree candidates, undergraduate and graduate is exactly 50–50. Rolling a perfect set of dice is students, and so forth. a nother example. A study population is that aggregation Such images of random selection, although of elements from which the sample is actually useful, seldom apply directly to sampling methods selected. As a practical matter, researchers are in social research. More typically, social researchers seldom in a position to guarantee that every use tables of random numbers or computer pro- element meeting the theoretical definitions laid grams that provide a random selection of sampling down actually has a chance of being selected in units. A sampling unit is that element or set of the sample. Even where lists of elements exist for elements considered for selection in some stage of sampling purposes, the lists are usually somewhat sampling. In Chapter 8, on survey research, we’ll incomplete. Some students are always inadver- see how computers are used to select random tently omitted from student rosters. Some tele- telephone numbers for interviewing, a technique phone subscribers request that their names and called random-digit dialing. numbers be unlisted. The reasons for using random-selection meth- Often, researchers decide to limit their study ods are twofold. First, this procedure serves as a populations more severely than indicated in the check on conscious or unconscious bias on the part preceding examples. National polling firms may of the researcher. The researcher who selects cases limit their national samples to the 48 adjacent on an intuitive basis might very well select cases states, omitting Alaska and Hawaii for practical rea- that would support his or her research expecta- sons. A researcher wishing to sample psychology tions or hypotheses. Random selection erases this professors may limit the study population to those danger. More importantly, random selection offers in psychology departments, omitting those in other access to the body of probability theory, which pro- departments. Whenever the population under vides the basis for estimating the characteristics of examination is altered in such fashions, you must the population as well as estimating the accuracy of make the revisions clear to your readers. samples. Let’s now examine probability theory in greater detail. Random Selection Probability Theory, Sampling With these definitions in hand, we can define the Distributions, and Estimates ultimate purpose of sampling: to select a set of of Sampling Error elements from a population in such a way that descriptions of those elements accurately portray Probability theory is a branch of mathematics that the total population from which the elements are provides the tools researchers need to devise sam selected. Probability sampling enhances the likeli- pling techniques that produce representative hood of accomplishing this aim and also provides methods for estimating the degree of probable study population That aggregation of elements success. from which a sample is actually selected. Random selection is the key to this process. In random selection A sampling method in which random selection, each element has an equal each element has an equal chance of selection inde- chance of selection independent of any other event pendent of any other event in the selection process. in the selection process. Flipping a coin is the most frequently cited example: Provided that the coin is sampling unit That element or set of elements perfect (that is, not biased in terms of coming up considered for selection in some stage of sampling.
136 ■ Chapter 5: Sampling Logic FIGURE 5-4 A Population of 10 People with $0–$9. Let’s simplify matters even more now by imagining a population of only 10 people with differing amounts of money in their pockets—ranging from $0 to $9. samples and to analyze the results of their sampling distributions. A single sample selected from a statistically. More formally, probability theory pro- population will give an estimate of the population vides the basis for estimating the parameters of a parameter. Other samples would give the same or population. A parameter is the summary descrip- slightly different estimates. Probability theory tells tion of a given variable in a population. The mean us about the distribution of estimates that would be income of all families in a city is a parameter; so is produced by a large number of such samples. To see the age distribution of the city’s population. When how this works, we’ll look at two examples of sam- researchers generalize from a sample, they’re pling distributions, beginning with a simple example using sample observations to estimate population in which our population consists of just ten cases, p arameters. Probability theory enables them to then moving on to a case of percentages that allows both make these estimates and arrive at a judg- a clear illustration of probable margin of error. ment of how likely the estimates will accurately represent the actual parameters in the population. The Sampling Distribution of Ten Cases For example, probability theory allows pollsters to infer from a sample of 2,000 voters how a popula- Suppose there are ten people in a group, and tion of 100 million voters is likely to vote—and to specify exactly what the probable margin of error each has a certain amount of money in his or her of the estimates is. pocket. To simplify, let’s assume that one pCeresonng a g e L e a r n i n g Probability theory accomplishes these seemingly has no money, another has one dollar, anBotahberbie: The Practice of magical feats by way of the concept of sampling nhainsetwdoolldaorsll.aFrisg,uarned5s-o4fporrethseunptsttohtehpeoppeurlsaSotnioocwniaiotlhfResearch, 13/e 1-133-04979-6 Fig. 7-4 parameter The summary description of a given ten people.* variable in a population. *I want to thank Hanan Selvin for suggesting this method of introducing probability sampling.
The Theory and Logic of Probability Sampling ■ 137 FIGURE 5-5 FIGURE 5-6 The Sampling Distribution of Samples of 1. In this simple example, The Sampling Distribution of Samples of 2. By merely increasing the mean amount of money these people have is $4.50 ($45/10). If our sample size to 2, we get possible samples that provide somewhat we picked 10 different samples of 1 person each, our “estimates” of better estimates of the mean. We couldn’t get either $0 or $9, and the mean would range all across the board. the estimates are beginning to cluster around the true value of the mean: $4.50. Our task is to determine the average amount of money one person has: specifically, the mean samples: [$0 $1], [$0 $2], . . . [$7 $8], [$8 $9]. number of dollars. If you simply add up the money shown in Figure 5-4, you’ll find that the total is Moreover, some of those samples produce the same $45, so the mean is $4.50. Our purpose in the rest of this exercise is to estimate that mean without means. For example, [$0 $6], [$1 $5], and [$2 $4] actually observing all ten individuals. We’ll do that by selecting random samples from the population all produce means of $3. In Figure 5-6, the three and using the means of those samples to estimate the mean of the whole population. dots shown above the $3 mean represent those To start, suppose we were to select—at random— three samples. a sample of only one person from the ten. Our ten possible samples thus consist of the ten cases Moreover, the 45 samples are not evenly dis- shown in Figure 5-4. tributed, as they were when the sample size was The ten dots shown on the graph in Figure 5-5 represent these ten samples. Because we’re tak- only one. Rather, they’re somewhat clustered ing samples of only one, they also represent the “means” we would get as estimates of the popu- around the true value of $4.50. Only two possible lation. The distribution of the dots on the graph is called the sampling distribution. Obviously, it samples deviate by as much as $4 from the true wouldn’t be a very good idea to select a sample of only one, because the chances are great that we’ll value ([$0 $1] and [$8 $9]), whereas five of the miss the true mean of $4.50 by quite a bit. samples would give the true estimate of $4.50; Now suppose we take a sample of two. As shown in Figure 5-6, increasing the sample size im- a nother eight samples miss the mark by only proves our estimations. There are now 45 possible 50 cents (plus or minus). Now ysuopupCtohseiennkwgtehasaegtlewecitlLledveoeantorlanorugi enrregssatimmpalteess. of Ce What do Bab the mean? FBiguarbeb5i-e7:pTrheesenPtrsatchteicseamopf ling distri- butions of samSpolecsiaolf R3,e4s,e5a, racnhd, 61.3/e So The prog1re-1ss3i3o-n04o9f7s9a-m6 pling Fdiigs.tr7ib-5utions is 1-13 clear. Every increase in sample size improves the distribution of estimates of the mean. The limiting case in this procedure, of course, is to select a sam- ple of ten. There would be only one possible sam- ple (everyone) and it would give us the true mean
138 ■ Chapter 5: Sampling Logic FIGURE 5-7 The Sampling Distributions of Samples of 3, 4, 5, and 6. As we increase the sample size, the possible samples cluster evermore tightly around the true value of the mean. The chance of extremely inaccurate estimates is reduced at the two ends of the distribution, and the percentage of the samples near the true value keeps increasing.
The Theory and Logic of Probability Sampling ■ 139 FIGURE 5-8 FIGURE 5-9 Ce Bab Range of Possible Sample Study Results. Shifting to a more realistic Results Produced by Three Hypothetical Studies. Assuming a large example, let’s assume that we want to sample student attitudes con- student body, let’s suppose that we selected three different samples, So cerning a proposed conduct code. Let’s assume that 50 percent of the each of substantial size. We would not necessarily expect those whole student body approves and 50 percent disapproves—though samples to perfectly reflect attitudes of the whole student body, but 1-13 the researcher doesn’t know that. they should come reasonably close. of $4.50. As we’ll see shortly, this principle applies numbers from a table of random numbers. Then to actual sampling of meaningful populations. The we interview the 100 students whose numbers larger the sample selected, the more accurate it is as have been selected and ask for their attitudes to- an estimation of the population from which it was ward the student code: whether they approve or drawn. disapprove. Suppose this operation gives us 48 stu- dents who approve of the code and 52 who disap- Sampling Distribution prove. This summary description of a variable in a and Estimates of Sampling Error sample is called a statistic. We present this statistic by placing a dot on the x axis at the point repre- Let’s turn now to a more realistic sampling situ- senting 48 percent. ation involving a much larger population and see how the notion of sampling distribution ap- Now let’s suppose we select another sample of plies. Assume that we wish to study the student 100 students in exactly the same fashion and mea- population of State University (SU) to determine sure their approval or disapproval of the student the percentage of students who approve or disap- code. Perhaps 51 students in the second sample prove of a student-conduct code proposed by the a pprove of the code. We place another dot in the administration. The study population will be the appropriate place on the x axis. Repeating this pro- aggregation of, say, 20,000 students contained in a cess once more, we may discover that 52 students student roster: the sampling frame. The elements in the third sample approve of the code. will be the individual students at SU. We’ll select a random sample of, say, 100 students for the pur- Figure 5-9Cperensegnatsgthee tLhreeae rdinffienregnt sample poses of estimating the entire student body. The seotafactthhisetoicsfsttuhrdeeeptnrheBtrsSceeaoenobdtrcbeiani.anigeTdlh:toRheTmeehbspaseeaesimarPccrreprcaunlhecltes,ati1gwcoe3fehs/reooaofnfasdptopumrdoevsneatdms i-n variable under consideration will be attitudes toward pling is that su1-c1h33sa-0m4p9l7e9s-d6rawn Ffrigo.m7-a8population the code, a binomial variable: approve and disapprove. (The logic of probability sampling applies to the give estimates of the parameter that exists in the examination of other types of variables, such as total population. Each of the random samp les, mean income, but the computations are somewhat then, gives us an estimate of the percentage of stu more complicated. Consequently, this introduction dents in the total student body who approve of focuses on binomials.) the student code. Unhappily, however, we have The horizontal axis of Figure 5-8 presents all statistic The summary description of a variable in a possible values of this parameter in the population— sample, used to estimate a population parameter. from 0 percent to 100 percent approval. The mid- point of the axis—50 percent—represents half the students approving of the code and the other half disapproving. To choose our sample, we give each student on the student roster a number and select 100 random
140 ■ Chapter 5: Sampling Logic FIGURE 5-10 The Sampling Distribution. If we were to select a large number of good samples, we would expect them to cluster around the true value (50 percent), but given enough such samples, a few would fall far from the mark. selected three samples and now have three sepa- Thus, although Figure 5-10 shows a wide range rate estimates. of estimates, more of them are in the vicinity of 50 percent than elsewhere in the graph. Probability To retrieve ourselves from this problem, let’s theory tells us, then, that the true value is in the draw more and more samples of 100 students each, vicinity of 50 percent. question each of the samples concerning their ap- proval or disapproval of the code, and plot the new Second, probability theory gives us a formula sample statistics on our summary graph. In draw- for estimating how closely the sample statistics ing many such samples, we discover that some of are clustered around the true value. To put it an the new samples provide duplicate estimates, as in other way, probability theory enables us to estimate the illustration of ten cases. Figure 5-10 shows the the sampling error—the degree of error to be sampling distribution of, say, hundreds of samples. expected for a given sample design. This formula This is often referred to as a normal curve. contains three factors: the parameter, the sam- ple size, and the standard error (a measure of Note that by increasing the number of samples sampling error): selected and interviewed, we’ve also increased the range of estimates provided by the sampling opera- s5 P3Q tion. In one sense we’ve increased our dilemma in n attempting to guess the parameter in the popula- tion. Probability theory, however, provides certain The symbols P and Q in the formula equal the important rules regarding the sampling distribution presented in Figure 5-10. popu lation parameters for the binomial: If 60 per- First, if many independent random samples cent of the student body approve of the code and are selected from a population, the sample statis- tics provided by those samples will be distributed 40 percent disapprove, P and Q are 60 perCceentnagnda g e L e a r n i n g around the population parameter in a known way. Q40=p1er–cePnat,nrdesPp=ec1tiv–eQly.,Tohre0s.6ymanbdol0n.4e.qNuBoaatlesbttbhheaiet : The Practice of sampling error The degree of error to be e xpected number of cases in each sample, and s is thSeosctaianl- Research, 13/e by virtue of studying a sample instead of e veryone. For probability sampling, the maximum error dard error. 1-133-04979-6 Fig. 7-10 depends on three factors: the sample size, the d iversity of the population, and the confidence level. Let’s assume that the population parameter in the student example is 50 percent approving of the code and 50 percent disapproving. Recall that we’ve been selecting samples of 100 cases each. When these numbers are put into the formula, we find that the standard error equals 0.05, or 5 percent.
The Theory and Logic of Probability Sampling ■ 141 In probability theory, the standard error is a if P = 0.8, PQ = 0.16; if P = 0.99, PQ = 0.0099. By valuable piece of information because it indicates extension, if P is either 0.0 or 1.0 (either 0 percent the extent to which the sample estimates will be or 100 percent approve of the student code), the distributed around the population parameter. (If standard error will be 0. If everyone in the popu- you’re familiar with the standard deviation in sta- lation has the same attitude (no variation), then tistics, you may recognize that the standard error, every sample will give exactly that estimate. in this case, is the standard deviation of the sam- pling distribution.) Specifically, probability theory The standard error is also a function of the indicates that certain proportions of the sample es- sample size—an inverse function. As the sample timates will fall within specified increments—each size increases, the standard error decreases. As the equal to one standard error—from the population sample size increases, the several samples will be parameter. Approximately 34 percent (0.3413) of clustered nearer to the true value. Another gen- the sample estimates will fall within one standard eral guideline is evident in the formula: Because error increment above the population parameter, of the square-root formula, the standard error is and another 34 percent will fall within one stan- reduced by half if the sample size is quadrupled. dard error below the parameter. In our example, In our present example, samples of 100 produce a the standard error increment is 5 percent, so we standard error of 5 percent; to reduce the standard know that 34 percent of our samples will give es- error to 2.5 percent, we must increase the sample timates of student approval between 50 percent size to 400. (the parameter) and 55 percent (one standard error above); another 34 percent of the samples will give All of this information is provided by established estimates between 50 percent and 45 percent (one probability theory in reference to the selection of standard error below the parameter). Taken to- large numbers of random samples. (If you’ve taken gether, then, we know that roughly two-thirds a statistics course, you may know this as the Cen- (68 percent) of the samples will give estimates tral Tendency Theorem.) If the population param within 5 percent of the parameter. eter is known and many random samples are selected, we can predict how many of the sample Moreover, probability theory dictates that estimates will fall within specified intervals from roughly 95 percent of the samples will fall within the parameter. plus or minus two standard errors of the true value, and 99.9 percent of the samples will fall within plus Recognize that this discussion illustrates only or minus three standard errors. In our present ex- the logic of probability sampling; it does not de- ample, then, we know that only one sample out of scribe the way research is actually conducted. Usu a thousand would give an estimate lower than 35 ally, we don’t know the parameter: The very reason percent approval or higher than 65 percent. we conduct a sample survey is to estimate that value. Moreover, we don’t actually select large The proportion of samples falling within one, numbers of samples: We select only one sample. two, or three standard errors of the parameter Nevertheless, the preceding discussion of probabil- is constant for any random sampling procedure ity theory provides the basis for inferences about such as the one just described, providing that a the typical social research situation. Knowing what large number of samples are selected. The size of it would be like to select thousands of samples al- the standard error in any given case, however, is lows us to make assumptions about the one sample a function of the population parameter and the we do select and study. sample size. If we return to the formula for a mo- ment, we note that the standard error will increase Confidence Levels and Confidence Intervals as a function of an increase in the quantity P times Q. Note further that this quantity reaches its maxi- Whereas probability theory specifies that 68 percent mum in the situation of an even split in the popu- of that fictitious large number of samples would lation. If P = 0.5, PQ = 0.25; if P = 0.6, PQ = 0.24; produce estimates falling within one standard error of the parameter, we can turn the logic around and
142 ■ Chapter 5: Sampling Logic infer that any single random sample estimate has a from 40 to 60 percent is the confidence interval. 68 percent chance of falling within that range. This (At the 68 percent confidence level, the confidence observation leads us to the two key components of interval would be 45–55 percent.) sampling error estimates: confidence level and confidence interval. We express the accuracy The logic of confidence levels and confidence of our sample statistics in terms of a level of con intervals also provides the basis for determining the fidence that the statistics fall within a specified appropriate sample size for a study. Once you’ve interval from the parameter. For example, we may decided on the degree of sampling error you can say we are 95 percent confident that our sample tolerate, you’ll be able to calculate the number of statistics (for example, 50 percent favor the new cases needed in your sample. Thus, for example, student code) are within plus or minus 5 per if you want to be 95 percent confident that your centage points of the population parameter. As the study findings are accurate within plus or minus confidence interval is expanded for a given statistic, 5 percentage points of the population parameters, our confidence increases. For example, we may say you should select a sample of at least 400. (Appen- that we are 99.9 percent confident that our statistic dix F is a convenient guide in this regard.) falls within three standard errors of the true value. (Now perhaps you can appreciate the humorous This, then, is the basic logic of probability sam- quip of unknown origin: Statistics means never pling. Random selection permits the researcher having to say you are certain.) to link findings from a sample to the body of probability theory so as to estimate the accuracy Although we may be confident (at some level) of those findings. All statements of accuracy in of being within a certain range of the parameter, sampling must specify both a confidence level and we’ve already noted that we seldom know what a confidence interval. The researcher must report the parameter is. To resolve this problem, we sub- that he or she is x percent confident that the popu- stitute our sample estimate for the parameter in the lation parameter is between two specific values. formula; that is, lacking the true value, we substi- In this example, I’ve demonstrated the logic of tute the best available guess. sampling error using a variable analyzed in per- centages. A different statistical procedure would be The result of these inferences and estimations required to calculate the standard error for a mean, is that we can estimate a population parameter for example, but the overall logic is the same. and also the expected degree of error on the basis of one sample drawn from a population. Begin- Notice that nowhere in this discussion of sam- ning with the question “What percentage of the ple size and accuracy of estimates did we consider student body approves of the student code?” you the size of the population being studied. This is could select a random sample of 100 students and because the population size is almost always irrel- interview them. You might then report that your evant. A sample of 2,000 respondents drawn prop- best estimate is that 50 percent of the student body erly to represent Vermont voters will be no more approves of the code and that you are 95 percent accurate than a sample of 2,000 drawn properly confident that between 40 and 60 percent (plus to represent all voters in the United States, even or minus two standard errors) approve. The range though the Vermont sample would be a substan- tially larger proportion of that small state’s voters confidence level The estimated probability that a than would the same number chosen to represent population parameter lies within a given confidence the nation’s voters. The reason for this counter- interval. Thus, we might be 95 percent confident intuitive fact is that the equations for calculating that between 35 and 45 percent of all voters favor sampling error all assume that the populations Candidate A. being sampled are infinitely large, so every sample would equal 0 percent of the whole. confidence interval The range of values within which a population parameter is estimated to lie. Of course, this is not literally true in practice. However, a sample of 2,000 represents only 0.61 percent of the Vermonters who voted for president
Populations and Sampling Frames ■ 143 in the 2008 election, and a sample of 2,000 U.S. The theory of sampling distribution makes assump- voters represents a mere 0.0015 percent of the tions that almost never apply in survey conditions. national electorate. Both of these proportions are The exact proportion of samples contained within sufficiently small as to approach the situation with specified increments of standard errors, for ex- infinitely large populations. ample, mathematically assumes an infinitely large population, an infinite number of samples, and Unless a sample represents, say, 5 percent or sampling with replacement—that is, every sam- more of the population it’s drawn from, that pro- pling unit selected is “thrown back into the pot” portion is irrelevant. In those rare cases of large and could be selected again. Second, our discus- proportions being selected, a “finite population cor- sion has greatly oversimplified the inferential jump rection” can be calculated to adjust the confidence from the distribution of several samples to the intervals. The following formula calculates the pro- probable characteristics of one sample. portion to be multiplied against the calculated error. I offer these cautions to provide perspective on finite population correction = N 2 n the uses of probability theory in sampling. Social N21 researchers often appear to overestimate the preci- sion of estimates produced by the use of probability In the formula, N is the population size and theory. As I’ll mention elsewhere in this chapter n is the size of the sample. Notice that in the ex and throughout the book, variations in sampling treme case where you studied the whole popula- techniques and nonsampling factors may further tion (hence N = n), the formula would yield zero as reduce the legitimacy of such estimates. For exam- the finite population correction. Multiplying zero ple, those selected in a sample who fail or refuse to times the sampling error calculated by the earlier participate detract further from the representative- formula would give a final sampling error of zero, ness of the sample. which would, of course, be precisely the case since you wouldn’t have sampled at all. Nevertheless, the calculations discussed in this section can be extremely valuable to you in under- Lest you weary of the statistical nature of this standing and evaluating your data. Although the discussion, it is useful to realize what an amazing calculations do not provide as precise estimates as thing we have been examining. There is remark- some researchers might assume, they can be quite able order within what might seem random and valid for practical purposes. They are unquestion- chaotic. One of the researchers to whom we owe ably more valid than less-rigorously derived esti- this observation is Sir Francis Galton (1822–1911), mates based on less-rigorous sampling methods. Most important, being familiar with the basic logic Order in Apparent Chaos—I know of scarcely underlying the calculations can help you react sen- anything so apt to impress the imagination as sibly both to your own data and to those reported the wonderful form of cosmic order expressed by others. by the “Law of Frequency of Error.” The law would have been personified by the Greeks Populations and deified, if they had known of it. It reigns and Sampling Frames with serenity and in complete self-effacement amidst the wildest confusion. The huger the The preceding section introduced the theoretical mob, and the greater the apparent anarchy, the model for social research sampling. Although as more perfect is its sway. It is the supreme law students, research consumers, and researchers we of Unreason (1889: 66). need to understand that theory, it’s no less impor- tant to appreciate the less-than-perfect conditions Two cautions are in order before we conclude that exist in the field. In this section we’ll look this discussion of the basic logic of probability sam- at one aspect of field conditions that requires a pling. First, the survey uses of probability theory as discussed here are technically not wholly justified.
144 ■ Chapter 5: Sampling Logic compromise with idealized theoretical conditions Properly drawn samples provide information and assumptions: the congruence of or disparity appropriate for describing the population of ele- between populations of sampling frames. ments composing the sampling frame—nothing more. I emphasize this point in view of the all-too- Simply put, a sampling frame is the list or common tendency for researchers to select samples quasi list of elements from which a probability from a given sampling frame and then make asser- sample is selected. If a sample of students is selected tions about a population similar to, but not iden- from a student roster, the roster is the sampling tical to, the population defined by the sampling frame. If the primary sampling unit for a complex frame. population sample is the census block, the list of census blocks composes the sampling frame—in For example, take a look at this report, which the form of a printed booklet or, better, some digital discusses the drugs most frequently prescribed by format permitting computer manipulation. Here U.S. physicians: are some reports of sampling frames appearing in research journals. In each example I’ve italicized Information on prescription drug sales is not the actual sampling frames. easy to obtain. But Rinaldo V. DeNuzzo, a professor of pharmacy at the Albany College We purchased a list of 50,000 Maryland resi- of Pharmacy, Union University, Albany, NY, dents who were registered to vote from Aristotle, has been tracking prescription drug sales for which maintains a national database including 25 years by polling nearby drugstores. He pub- 175 million registered voters. We refer to these lishes the results in an industry trade magazine, residents as “registered voters” even though MM&M. some of them have not actually gone to the polls in some time. The Aristotle database is compiled DeNuzzo’s latest survey, covering 1980, from state records, county boards of elections, is based on reports from 66 pharmacies in state boards of registrars, etc. 48 communities in New York and New Jersey. Unless there is something peculiar about that (Tourangeau et al. 2010: 416) part of the country, his findings can be taken as representative of what happens across the Respondents were undergraduates enrolled in country. introductory psychology classes at Ohio State University in spring 2001. (Moskowitz 1981: 33) (Chang and Krosnick, 2010: 155) What is striking in the excerpt is the casual com- ment about whether there is anything peculiar The data reported in this paper . . . were gath- about New York and New Jersey. There is. The ered from a probability sample of adults aged 18 lifestyle in these two states hardly typifies the other and over residing in households in the 48 contiguous 48. We cannot assume that residents in these large, United States. Personal interviews with 1,914 urbanized, eastern seaboard states necessarily have respondents were conducted by the Survey the same drug-use patterns that residents of Missis- Research Center of the University of Michigan sippi, Nebraska, or Vermont do. during the fall of 1975. Does the survey even represent prescription (Jackman and Senter 1980: 345) patterns in New York and New Jersey? To deter- mine that, we would have to know something sampling frame That list or quasi list of units com- about the way the 48 communities and the 66 phar- posing a population from which a sample is selected. macies were selected. We should be wary in this If the sample is to be representative of the popula- regard, in view of the reference to “polling nearby tion, it is essential that the sampling frame include drugstores.” As we’ll see, there are several methods all (or nearly all) members of the population. for selecting samples that ensure representative- ness, and unless they’re used, we shouldn’t gener- alize from the study findings.
Populations and Sampling Frames ■ 145 A sampling frame, then, covers the population taxpayers, business permit holders, licensed profes- we wish to study. In the simplest sample design, sionals, and so forth. Although it may be difficult the sampling frame is a list of the elements com- to gain access to some of these lists, they provide posing the study population. In practice, though, excellent sampling frames for specialized research existing sampling frames often define the study purposes. population rather than the other way around. That is, we often begin with a population in mind for Of course, the sampling elements in a study our study; then we search for possible sampling need not be individuals. Social researchers might frames. Having examined and evaluated the frames use lists of universities, businesses, cities, academic available for our use, we decide which frame pres- journals, newspapers, unions, political clubs, pro- ents a study population most appropriate to our fessional associations, and so forth. needs. Telephone directories were once used for Studies of organizations are often the simplest “quick-and-dirty” public opinion polls. They’re easy from a sampling standpoint because organizations and inexpensive to use—no doubt the reason for typically have membership lists. In such cases, the their popularity. And, if you want to make asser- list of members constitutes an excellent sampling tions about telephone subscribers, the directory is a frame. If a random sample is selected from a mem- fairly good sampling frame. (Realize, of course, that bership list, the data collected from that sample a given directory will not include new subscribers may be taken as representative of all members—if or those who have requested unlisted numbers. all members are included in the list. Sampling is further complicated by the directories’ inclusion of nonresidential listings.) Unfortunately, Populations that can be sampled from good telephone directories are all too often used as a organizational lists include elementary school, listing of a city’s population or of its voters. Of the high school, and university students and faculty; many defects in this reasoning, the chief one in- church members; factory workers; fraternity or volves a bias, as we have seen. Poor people are less sorority members; members of social, service, likely to have telephones; rich people may have or political clubs; and members of professional more than one line. A telephone directory sample, associations. therefore, is likely to have a middle- or upper-class bias. As we’ll see a little later, the telephone direc- The preceding comments apply primarily to tory may produce an age bias, since many young local organizations. Often, statewide or national people have only cell phones. organizations do not have a single membership list. There is, for example, no single list of Episcopalian The class bias inherent in telephone direc- church members. However, a slightly more com- tory samples is often hidden. Pre-election polls plex sample design could take advantage of local conducted in this fashion are sometimes quite church membership lists by first sampling churches accurate, perhaps because of the class bias evident and then subsampling the membership lists of in voting itself: Poor people are less likely to vote. those churches selected. (More about that later.) Frequently, then, these two biases nearly coincide, so that the results of a telephone poll may come Other lists of individuals may be especially rel- very close to the final election outcome. Unhappily, evant to the research needs of a particular study. you never know for sure until after the election. Government agencies maintain lists of registered And sometimes, as in the case of the 1936 Literary voters, for example, and some political pollsters use Digest poll, you may discover that the voters have registration-based sampling (RBS), using those lists. not acted according to the expected class biases. In some cases, there may be delays in keeping such The ultimate disadvantage of this method, then, is files up-to-date, and a person who is registered the researcher’s inability to estimate the degree of to vote may not actually do so in the election of error to be expected in the sample findings. interest. In Chapter 8 we’ll return to the matter of Other lists that may be available contain the sampling telephones, in connection with survey names of automobile owners, welfare recipients,
146 ■ Chapter 5: Sampling Logic research. We’ll examine random-digit dialing, 2. Often, sampling frames do not truly include which was developed to resolve some of the prob- all the elements their names might imply. lems just discussed, and we’ll see that the growth Omissions are almost inevitable. Thus, a first in popularity of cell phones has further complicated concern of the researcher must be to assess the matters. extent of the omissions and to correct them if possible. (Of course, the researcher may Street directories and tax maps are sometimes feel that he or she can safely ignore a small used for easy samples of households, but they may number of omissions that cannot easily be present incompleteness and bias. For example, in corrected.) strictly zoned urban regions, illegal housing units are unlikely to appear on official records. As a re- 3. To be generalized even to the population sult, such units could not be selected, and sample composing the sampling frame, all elements findings could not be representative of those units, must have equal representation in the frame. which are often poorer and more crowded than the Typically, each element should appear only average. once. Elements that appear more than once will have a greater probability of selection, and The preceding comments apply to the United the sample will, overall, overrepresent those States but not to all countries. In Japan, for ex- elements. ample, the government maintains quite accurate population registration lists. Moreover, citizens Other, more practical matters relating to are required by law to keep their information populations and sampling frames will be treated up-to-date, such as changes in residence or elsewhere in this book. For example, the form of births and deaths in the household. As a conse- the sampling frame—such as a list in a publica- quence, you can select simple random samples tion, a 3-by-5 card file, CD-ROM, or USB storage of the population more easily in Japan than in drive—can affect how easy it is to use. And ease the United States. Such a registration list in the of use may often take priority over scientific con- United States would conflict directly with this siderations: An “easier” list may be chosen over a country’s norms regarding individual privacy. “harder” one, even though the latter is more ap- propriate to the target population. We should not In recent years, American researchers have take a dogmatic position in this regard, but every begun experimenting with address files maintained researcher should carefully weigh the relative ad- by the U.S. Postal Service, such as the Special De- vantages and disadvantages of such alternatives. livery Sequence File. As problems have increasingly arisen with regard to the sampling of telephone Types of Sampling Designs numbers (discussed further in Chapter 8), address- based sampling (ABS) for use in mail surveys has Up to this point, we’ve focused on simple random been improving (Link et al. 2008). sampling. Indeed, the body of statistics typically used by social researchers assumes such a sample. Review of Populations As you’ll see shortly, however, you have several and Sampling Frames options in choosing your sampling method, and you’ll seldom if ever choose simple random sam- Because social research literature gives surprisingly pling. There are two reasons for this. First, with all little attention to the issues of populations and but the simplest sampling frame, simple random sampling frames, I’ve devoted special attention to sampling is not feasible. Second, and probably sur- them. Here is a summary of the main guidelines to prisingly, simple random sampling may not be the remember: most accurate method available. Let’s turn now to a discussion of simple random sampling and the 1. Findings based on a sample can be taken as other options available. representing only the aggregation of elements that compose the sampling frame.
Types of Sampling Designs ■ 147 Simple Random Sampling select every tenth element for your sample. To en- sure against any possible human bias in using this As noted, simple random sampling (SRS) is method, you should select the first element at ran- the basic sampling method assumed in the statisti- dom. Thus, in the preceding example, you would cal computations of social research. Because the begin by selecting a random number between one mathematics of random sampling are especially and ten. The element having that number is in- complex, we’ll detour around them in favor of cluded in the sample, plus every tenth element d escribing the ways of employing this method in following it. This method is technically referred to the field. as a systematic sample with a random start. Two terms are frequently used in connection with systematic Once a sampling frame has been properly sampling. The sampling interval is the standard established, to use simple random sampling the distance between elements selected in the sample: researcher assigns a single number to each element ten in the preceding sample. The sampling ratio in the list, not skipping any number in the process. is the proportion of elements in the population that A table of random numbers (Appendix C) is then are selected: 1⁄10 in the example. used to select elements for the sample. See the Tips and Tools feature, “Using a Table of Random Num- sampling interval = population size bers” for more about this process. sample size If your sampling frame is in a machine-readable sampling ratio = sample size form, such as CD-ROM or USB storage drive, a population size computer can automatically select a simple random sample. (In effect, the computer program numbers In practice, systematic sampling is virtually the elements in the sampling frame, generates its identical to simple random sampling. If the list of own series of random numbers, and prints out the elements is indeed randomized before sampling, list of elements selected.) one might argue that a systematic sample drawn from that list is in fact a simple random sample. Figure 5-11 offers a graphic illustration of sim- By now, debates over the relative merits of simple ple random sampling. Note that the members of our random sampling and systematic sampling have hypothetical micropopulation have been numbered from 1 to 100. Moving to Appendix C, we decide simple random sampling (SRS) A type of prob- to use the last two digits of the first column and to ability sampling in which the units composing a begin with the third number from the top. This population are assigned numbers. A set of random yields person number 30 as the first one selected numbers is then generated, and the units having into the sample. Number 67 is next, and so forth. those numbers are included in the sample. (Person 100 would have been selected if “00” had come up in the list.) systematic sampling A type of probability sam- pling in which every kth unit in a list is selected for Systematic Sampling inclusion in the sample—for example, every 25th student in the college directory of students. You Simple random sampling is seldom used in p ractice. compute k by dividing the size of the population As you’ll see, it’s not usually the most efficient by the desired sample size; k is called the sampling method, and it can be laborious if done manually. interval. Within certain constraints, systematic sam- Typically, simple random sampling requires a list of pling is a functional equivalent of simple random elements. When such a list is available, researchers sampling and is usually easier to do. Typically, the usually employ systematic sampling instead. first unit is selected at random. In systematic sampling, every kth element in sampling interval The standard distance between the total list is chosen (systematically) for inclusion elements selected from a population for a sample. in the sample. If the list contained 10,000 elements and you wanted a sample of 1,000, you would sampling ratio The proportion of elements in the population that are selected to be in a sample.
148 ■ Chapter 5: Sampling Logic Tips and Tools Using a Table of Random Numbers the first page—10480—we’d consider only the 104. (We could agree to take the digits farthest to the right, 480, or the middle In social research, it’s often appropriate to select a set of random num- three digits, 048, and any of these plans would work.) They key is bers from a table such as the one in Appendix C. Here’s how to do that. to make a plan and stick with it. For convenience, let’s use the left- most three digits. Suppose you want to select a simple random sample of 100 people (or other units) out of a population totaling 980. 5. We can also choose to progress through the tables any way we want: down the columns, up them, across to the right or to the left, 1. To begin, number the members of the population: in this case, or diagonally. Again, any of these plans will work just fine as long from 1 to 980. Now the task is to select 100 random numbers. as we stick to it. For convenience, let’s agree to move down the Once you’ve done that, your sample will consist of the people columns. When we get to the bottom of one column, we’ll go to having the numbers you’ve selected. (Note: It’s not essential to the top of the next. actually number them, as long as you’re sure of the total. If you have them in a list, for example, you can always count through the 6. Now, where do we start? You can close your eyes and stick a pencil list after you’ve selected the numbers.) into the table and start wherever the pencil point lands. (I know it doesn’t sound scientific, but it works.) Or, if you’re afraid you’ll 2. The next step is to determine the number of digits you’ll need hurt the book or miss it altogether, close your eyes and make up a in the random numbers you select. In our example, there are column number and a row number. (“I’ll pick the number in the 980 members of the population, so you’ll need three-digit fifth row of column 2.”) Start with that number. numbers to give everyone a chance of selection. (If there were 11,825 members of the population, you’d need to select five-digit 7. Let’s suppose we decide to start with the fifth number in column 2. numbers.) Thus, we want to select 100 random numbers in the If you look on the first page of Appendix C, you’ll see that the range from 001 to 980. starting number is 39975. We’ve selected 399 as our first random number, and we have 99 more to go. Moving down the second 3. Now turn to the first page of Appendix C. Notice there are several column, we select 069, 729, 919, 143, 368, 695, 409, 939, and rows and columns of five-digit numbers, and are two pages, with so forth. At the bottom of column 2 (on the second page of this the columns continuing from the first page to the second. The table table), we select number 017 and continue to the top of column 3: represents a series of random numbers in the range from 00001 to 015, 255, and so on. 99999. To use the table for your hypothetical sample, you have to answer these questions: 8. See how easy it is? But trouble lies ahead.When we reach column 5, we’re speeding along, selecting 816, 309, 763, 078, 061, 277, a. How will you create three-digit numbers out of five-digit 988 . . . Wait a minute! There are only 980 students in the senior numbers? class. How can we pick number 988? The solution is simple: Ignore it. Any time you come across a number that lies outside your range, b. What pattern will you follow in moving through the table to skip it and continue on your way: 188, 174, and so forth. The same select your numbers? solution applies if the same number comes up more than once. If you select 399 again, for example, just ignore it the second time. c. Where will you start? 9. That’s it.You keep up the procedure until you’ve selected 100 random E ach of these questions has several satisfactory answers. The key is to numbers. Returning to your list, your sample consists of person create a plan and follow it. Here’s an example. number 399, person number 69, person number 729, and so forth. 4. To create three-digit numbers from five-digit numbers, let’s agree to select five-digit numbers from the table but consider only the left-most three digits in each case. If we picked the first number on
Types of Sampling Designs ■ 149 FIGURE 5-11 A Simple Random Sample. Having numbered everyone in the population, we can use a table of random numbers to select a representative sample from the overall population. Anyone whose number is chosen from the table is in the sample. been resolved largely in favor of the latter, simpler of elements is arranged in a cyclical pattern that method. Empirically, the results are virtually identi- c oincides with the sampling interval, a grossly cal. And, as you’ll see in a later section, systematic biased sample might be drawn. Here are two sampling, in some instances, is slightly more accu- e xamples that illustrate this danger. rate than simple random sampling. In a classic study of soldiers during World War There is one danger involved in systematic II, the researchers selected a systematic sample sampling. The arrangement of elements in the list from unit rosters. Every tenth soldier on the roster can make systematic sampling unwise. Such an was selected for the study. The rosters, however, arrangement is usually called periodicity. If the list were arranged in a table of organizations: sergeants
150 ■ Chapter 5: Sampling Logic first, then corporals and privates, squad by squad. Stratification is not an alternative to these meth- Each squad had ten members. As a result, every ods; rather, it represents a possible modification of tenth person on the roster was a squad sergeant. their use. The systematic sample selected contained only ser- geants. It could, of course, have been the case that Simple random sampling and systematic sam no sergeants were selected for the same reason. pling both ensure a degree of representativeness and permit an estimate of the error present. Stratified As another example, suppose we select a sam sampling is a method for obtaining a greater degree ple of apartments in an apartment building. If the of representativeness by decreasing the probable sample is drawn from a list of apartments arranged sampling error. To understand this method, we in numerical order (for example, 101, 102, 103, must return briefly to the basic theory of sampling 104, 201, 202, and so on), there is a danger of the distribution. sampling interval coinciding with the number of apartments on a floor or some multiple thereof. Recall that sampling error is reduced by two Then the samples might include only northwest- factors in the sample design. First, a large sample corner apartments or only apartments near the produces a smaller sampling error than a small elevator. If these types of apartments have some sample does. Second, a homogeneous population other particular characteristic in common (for ex- produces samples with smaller sampling errors ample, higher rent), the sample will be biased. The than a heterogeneous population does. If 99 per- same danger would appear in a systematic sample cent of the population agrees with a certain state- of houses in a subdivision arranged with the same ment, it’s extremely unlikely that any probability number of houses on a block. sample will greatly misrepresent the extent of agreement. If the population is split 50–50 on the In considering a systematic sample from a list, statement, then the sampling error will be much then, you should carefully examine the nature greater. of that list. If the elements are arranged in any par- ticular order, you should figure out whether that Stratified sampling is based on this second order will bias the sample to be selected, then you factor in sampling theory. Rather than selecting should take steps to counteract any possible bias a sample from the total population at large, the (for example, take a simple random sample from researcher ensures that appropriate numbers of cyclical portions). elements are drawn from homogeneous subsets of that population. To get a stratified sample of Usually, however, systematic sampling is supe- university students, for example, you would first rior to simple random sampling, in convenience if organize your population by college class and then nothing else. Problems in the ordering of elements draw appropriate numbers of freshmen, sopho- in the sampling frame can usually be remedied mores, juniors, and seniors. In a nonstratified sam- quite easily. ple, representation by class would be subjected to the same sampling error as other variables would. Stratified Sampling In a sample stratified by class, the sampling error on this variable is reduced to zero. So far we’ve discussed two methods of sample selection from a list: random and systematic. More-complex stratification methods are also possible. In addition to stratifying by class, you stratification The grouping of the units composing might also stratify by sex, by GPA, and so forth. In a population into homogeneous groups (or strata) this fashion you might be able to ensure that your before sampling. This procedure, which may be used sample would contain the proper numbers of male in conjunction with simple random, systematic, or sophomores with a 3.5 average, of female sopho- cluster sampling, improves the representativeness mores with a 4.0 average, and so forth. of a sample, at least in terms of the stratification variables. The ultimate function of stratification, then, is to organize the population into homogeneous subsets (with heterogeneity between subsets)
Types of Sampling Designs ■ 151 and to select the appropriate number of elements The other method is to group students as de- from each. To the extent that the subsets are scribed and then put those groups together in a homogeneous on the stratification variables, continuous list, beginning with all freshmen men they may be homogeneous on other variables with a 4.0 average and ending with all senior as well. Because age is related to college class, women with a 1.0 or below. You would then select a sample stratified by class will be more repre- a systematic sample, with a random start, from sentative in terms of age as well, compared with the entire list. Given the arrangement of the list, an unstratified sample. Because occupational a systematic sample would select proper num- a spirations still seem to be related to sex, a s ample bers (within an error range of 1 or 2) from each stratified by sex will be more representative in subgroup. (Note: A simple random sample drawn terms of occupational aspirations. from such a composite list would cancel out the stratification.) The choice of stratification variables typically depends on what variables are available. Sex can Figure 5-12 offers a graphic illustration of often be determined in a list of names. U niversity stratified, systematic sampling. As you can see, we lists are typically arranged by class. Lists of fac- lined up our micropopulation according to sex and ulty members may indicate their departmental race. Then, beginning with a random start of “3,” affiliation. Government agency files may be ar- we’ve taken every tenth person thereafter: 3, 13, ranged by geographic region. Voter registration lists 23, . . . , 93. are arranged according to precinct. Stratified sampling ensures the proper repre- In selecting stratification variables from among sentation of the stratification variables; this, in turn, those available, however, you should be concerned enhances the representation of other variables primarily with those that are presumably related to related to them. Taken as a whole, then, a stratified variables you want to represent accurately. Because sample is more likely than a simple random sample sex is related to many variables and is often avail- to be more representative on several variables. able for stratification, it’s often used. Education is Although the simple random sample is still re- related to many variables, but it’s often not avail- garded as somewhat sacred, it should now be clear able for stratification. Geographic location within that you can often do better. a city, state, or nation is related to many things. Within a city, stratification by geographic loca- Implicit Stratification tion usually increases representativeness in social in Systematic Sampling class, ethnic group, and so forth. Within a nation, it increases representativeness in a broad range of I mentioned that systematic sampling can, under attitudes as well as in social class and ethnicity. certain conditions, be more accurate than simple random sampling. This is the case whenever the ar- When you’re working with a simple list of rangement of the list creates an implicit stratification. all elements in the population, two methods of As already noted, if a list of university students is ar- stratification predominate. In one method, you sort ranged by class, then a systematic sample provides a the population elements into discrete groups based stratification by class where a simple random sample on whatever stratification variables are being used. would not. On the basis of the relative proportion of the popu- lation represented by a given group, you select— In a study of students at the University of Ha- randomly or systematically—several elements from waii, after stratification by school class, the students that group constituting the same proportion of were arranged by their student identification your desired sample size. For example, if sopho- numbers. These numbers, however, were their so- more men with a 4.0 average compose 1 percent cial security numbers. The first three digits of the of the student population and you desire a sample social security number indicate the state in which of 1,000 students, you would select 10 sophomore the number was issued. As a result, within a class, men with a 4.0 average. students were arranged by the state in which they
152 ■ Chapter 5: Sampling Logic FIGURE 5-12 A Stratified, Systematic Sample with a Random Start. A stratified, systematic sample involves two stages. First the members of the population are gathered into homogeneous strata; this simple example merely uses sex and race as a stratification variable, but more could be used. Then every kth (in this case, every tenth) person in the stratified arrangement is selected into the sample. were issued a social security number, providing a sample of university students. The purpose of the rough stratification by geographic origin. study was to survey, with a mail-out question- naire, a representative cross section of students An ordered list of elements, therefore, may be attending the main campus of the University of more useful to you than an unordered, randomized Hawaii. The following sections describe the steps list. I’ve stressed this point in view of the unfortu- and decisions involved in selecting that sample. nate belief that lists should be randomized before systematic sampling. Only if the arrangement pres- Study Population and Sampling Frame ents the problems discussed earlier should the list be rearranged. The obvious sampling frame available for use in this sample selection was the computerized file Illustration: Sampling maintained by the university administration. The University Students file contained students’ names, local and perma- Let’s put these principles into practice by looking nent addresses, and social security numberCs,easnwgeallg e L e a r n i n g at an actual sampling design used to select a as a variety of other information such as fiBealdbobfie: The Practice of study, class, age, and sex.
Multistage Cluster Sampling ■ 153 The computer database, however, contained were systematically selected (with a random start) entries on all people who could, by any conceiv- for exclusion from the sample. The final sample for able definition, be called students, many of whom the study was thereby reduced to 733 students. seemed inappropriate for the purposes of the study. As a result, researchers needed to define the study I mention this modification in order to il- population in a somewhat more restricted fashion. lustrate the frequent need to alter a study plan in The final definition included those 15,225 day- midstream. Because the excluded students were program degree candidates who were registered systematically omitted from the initial systematic for the fall semester on the Manoa campus of the sample, the remaining 733 students could still be university, including all colleges and departments, taken as reasonably representing the study popula- both undergraduate and graduate students, and tion. The reduction in sample size did, of course, both U.S. and foreign students. The computer pro- increase the range of sampling error. gram used for sampling then limited consideration to students fitting this definition. Multistage Cluster Sampling Stratification The preceding sections have dealt with reason- ably simple procedures for sampling from lists of The sampling program also permitted stratification elements. Such a situation is ideal. Unfortunately, of students before sample selection. The researchers however, much interesting social research requires decided that stratification by college class would be the selection of samples from populations that sufficient, although the students might have been cannot easily be listed for sampling purposes: the further stratified within class, if desired, by sex, population of a city, state, or nation; all university c ollege, major, and so forth. students in the United States; and so forth. In such cases, the sample design must be much more com- Sample Selection plex. Such a design typically involves the initial sampling of groups of elements—clusters—followed Once the students had been arranged by class, a by the selection of elements within each of the systematic sample was selected across the entire selected clusters. rearranged list. The sample size for the study was initially set at 1,100. To achieve this sample, the Cluster sampling may be used when it’s sampling program was set for a 1⁄14 sampling ratio. either impossible or impractical to compile an ex- The program generated a random number between haustive list of the elements composing the target 1 and 14; the student having that number and population, such as all church members in the every 14th student thereafter was selected in the United States. Often, however, the population ele- sample. ments are already grouped into subpopulations, and a list of those subpopulations either exists or Once the sample had been selected, the com- can be created practically. For example, church puter was instructed to print each student’s name members in the United States belong to discrete and mailing address on self-adhesive mailing labels. churches, which are either listed or could be. Fol- These labels were then simply transferred to enve- lowing a cluster sample format, then, researchers lopes for mailing the questionnaires. cluster sampling A multistage sampling in which Sample Modification natural groups (clusters) are sampled initially, with the members of each selected group being sub- This initial design of the sample had to be modified. sampled afterward. For example, you might select Before the mailing of questionnaires, the research- a sample of U.S. colleges and universities from a ers discovered that, because of unexpected ex- directory, get lists of the students at all the selected penses in the production of the questionnaires, schools, then draw samples of students from each. they couldn’t cover the costs of mailing to all 1,100 students. As a result, one-third of the mailing labels
154 ■ Chapter 5: Sampling Logic Research in Real Life Sampling Iran 6. The eastern provinces including Khorasan and Semnan Whereas most of the examples given in this textbook are taken from 7. The northern provinces including Gilan, Mazandran, and Golestan its country of origin, the United States, the basic methods of sampling would apply in other national settings as well. At the same time, re 8. Systan searchers may need to make modifications appropriate to local condi- tions. In selecting a national sample of Iran, for example, Hamid Abdol- 9. Kurdistan lahyan and Taghi Azadarmaki (2000: 21) from the University of Tehran began by stratifying the nation on the basis of cultural differences, Within each of these cultural areas, the researchers selected dividing the country into nine cultural zones as follows: samples of census blocks and, on each selected block, a sample of households. Their sample design made provisions for getting the proper 1. Tehran numbers of men and women as respondents within households and provisions for replacing those households where no one was at home. 2. Central region including Isfahan, Arak, Qum, Yazd, and Kerman Though the United States and Iran are politically and culturally 3. The southern provinces including Hormozgan, Khuzistan, Bushehr, quite different, the sampling methods appropriate for selecting a repre- and Fars sentative sample of populations are the same. Later in this chapter, when you review a detailed description of sampling the household population 4. The marginal western region including Lorestan, Charmahal and of Oakland, California, you will find it strikingly similar to the methods Bakhtiari, Kogiluyeh and Eelam used in Iran by Abdollahyan and Azadarmaki. 5. The western provinces including western and eastern Azarbaijan, Source: Hamid Abdollahyan and Taghi Azadarmaki. 2000. “Sampling Design in a Zanjan, Ghazvin, and Ardebil Survey Research: The Sampling Practice in Iran.” Paper presented to the meetings of the American Sociological Association, August 12–16. Washington, DC. could sample the list of churches in some manner Multistage cluster sampling, then, involves (for example, a stratified, systematic sample). Next, the repetition of two basic steps: listing and sam- they would obtain lists of members from each of pling. The list of primary sampling units (churches, the selected churches. Each of the lists would then blocks) is compiled and, perhaps, stratified for sam- be sampled, to provide samples of church members pling. Then a sample of those units is selected. The for study. selected primary sampling units are then listed and perhaps stratified. The list of secondary sampling Another typical situation concerns sampling units is then sampled, and so forth. among population areas such as a city. Although there is no single list of a city’s population, citizens The listing of households on even the selected reside on discrete city blocks or census blocks. Re- blocks is, of course, a labor-intensive and costly searchers can, therefore, select a sample of blocks activity—one of the elements making face-to-face, initially, create a list of people living on each of the household surveys quite expensive. Vincent Ian- selected blocks, and take a subsample of the people nacchione, Jennifer Staab, and David Redden on each block. (2003) report some initial success using postal mailing lists for this purpose. Although the lists are In a more complex design, researchers might not perfect, they may be close enough to warrant sample blocks, list the households on each selected the significant savings in cost. block, sample the households, list the people resid- ing in each household, and, finally, sample the peo- Multistage cluster sampling makes possible ple within each selected household. This multistage those studies that would otherwise be impossible. sample design leads ultimately to a selection of a Specific research circumstances often call for special sample of individuals but does not require the ini- designs, as the feature Research in Real Life: “Sam- tial listing of all individuals in the city’s population. pling Iran” demonstrates.
Multistage Cluster Sampling ■ 155 Multistage Designs increased at the expense of more poorly represent- and Sampling Error ing the elements composing each cluster, or vice versa. Fortunately, homogeneity can be used to Although cluster sampling is highly efficient, the ease this dilemma. price of that efficiency is a less-accurate sample. A simple random sample drawn from a popula- Typically, the elements composing a given tion list is subject to a single sampling error, but a natural cluster within a population are more ho- two-stage cluster sample is subject to two sampling mogeneous than all elements composing the total errors. First, the initial sample of clusters will repre- population are. The members of a given church sent the population of clusters only within a range are more alike than all members of the denomi- of sampling error. Second, the sample of elements nation are; the residents of a given city block are selected within a given cluster will represent all more alike than the residents of a whole city are. the elements in that cluster only within a range As a result, relatively few elements may be needed of sampling error. Thus, for example, a researcher to represent a given natural cluster adequately, runs a certain risk of selecting a sample of dispro- although a larger number of clusters may be portionately wealthy city blocks, plus a sample needed to adequately represent the diversity of disproportionately wealthy households within found among the clusters. This fact is most clearly those blocks. The best solution to this problem lies seen in the extreme case of very different clusters in the number of clusters selected initially and the composed of identical elements within each. In number of elements within each cluster. such a situation, a large number of clusters would adequately represent all its members. Although Typically, researchers are restricted to a total this extreme situation never exists in reality, it’s sample size; for example, you may be limited to closer to the truth in most cases than its opposite: conducting 2,000 interviews in a city. Given this identical clusters composed of grossly divergent broad limitation, however, you have several op- elements. tions in designing your cluster sample. At the extremes you could choose one cluster and select The general guideline for cluster design, then, is 2,000 elements within that cluster, or you could to maximize the number of clusters selected while select 2,000 clusters with one element selected decreasing the number of elements within each within each. Of course, neither approach is advis- cluster. However, this scientific guideline must be able, but a broad range of choices lies between balanced against an administrative constraint. The them. Fortunately, the logic of sampling distribu- efficiency of cluster sampling is based on the abil- tions provides a general guideline for this task. ity to minimize the listing of population elements. By initially selecting clusters, you need only list the Recall that sampling error is reduced by elements composing the selected clusters, not all two factors: an increase in the sample size and elements in the entire population. Increasing the increased homogeneity of the elements being number of clusters, however, goes directly against sampled. These factors operate at each level of a this efficiency factor. A small number of clusters multistage sample design. A sample of clusters will may be listed more quickly and more cheaply than best represent all clusters if a large number are a large number. (Remember that all the elements selected and if all clusters are very much alike. A in a selected cluster must be listed even if only a sample of elements will best represent all elements few are to be chosen in the sample.) in a given cluster if a large number are selected from the cluster and if all the elements in the clus- The final sample design will reflect these two ter are very much alike. constraints. In effect, you’ll probably select as many clusters as you can afford. Lest this issue be left too With a given total sample size, however, if the open-ended at this point, here’s one general guide- number of clusters is increased, the number of ele- line. Population researchers conventionally aim at ments within a cluster must be decreased. In this the selection of 5 households per census block. If respect, the representativeness of the clusters is a total of 2,000 households are to be interviewed,
156 ■ Chapter 5: Sampling Logic FIGURE 5-13 Multistage Cluster Sampling. In multistage cluster sampling, we begin by selecting a sample of the clusters (in this case, city blocks). Then, we make a list of the elements (households, in this case) and select a sample of elements from each of the selected clusters. you would aim at 400 blocks with 5 household a multistage sample design is subject to a sampling interviews on each. Figure 5-13 presents a graphic error at each stage. Because the sample size is nec- overview of this process. essarily smaller at each stage than the total sample size, the sampling error at each stage will be greater Before we turn to other, more-detailed proce- than would be the case for a single-stage random dures available to cluster sampling, let me reiterate sample of elements. Second, sampling error is esti- that this method almost inevitably involves a loss of mated on the basis of observed variance among the accuracy. The manner in which this appears, how-
Multistage Cluster Sampling ■ 157 from among relatively homogeneous clusters, the selected. Notice that this produces an overall sam- estimated sampling error will be too optimistic and pling scheme in which every element in the whole must be corrected in the light of the cluster sample population has the same probability of selection. design. Let’s say we’re selecting households within a Stratification in Multistage city. If there are 1,000 city blocks and we initially Cluster Sampling select a sample of 100, that means that each block has a 100⁄1,000 or 0.1 chance of being selected. If Thus far, we’ve looked at cluster sampling as we next select 1 household in 10 from those resid- though a simple random sample were selected ing on the selected blocks, each household has a at each stage of the design. In fact, stratification 0.1 chance of selection within its block. To calculate techniques can be used to refine and improve the the overall probability of a household being se- sample being selected. lected, we simply multiply the probabilities at the individual steps in sampling. That is, each house- The basic options here are essentially the same hold has a 1⁄10 chance of its block being selected as those in single-stage sampling from a list. In se- and a 1⁄10 chance of that specific household being lecting a national sample of churches, for example, selected if the block is one of those chosen. Each you might initially stratify your list of churches by household, in this case, has a 1⁄10 × 1⁄10 = 1⁄100 denomination, geographic region, size, rural or chance of selection overall. Because each house- urban location, and perhaps by some measure of hold would have the same chance of selection, the social class. sample so selected should be representative of all households in the city. Once the primary sampling units (churches, blocks) have been grouped according to the rel- There are dangers in this procedure, however. evant, available stratification variables, either In particular, the variation in the size of blocks simple random or systematic-sampling techniques (measured in numbers of households) presents a can be used to select the sample. You might select a problem. Let’s suppose that half the city’s popula- specified number of units from each group, or stra- tion resides in 10 densely packed blocks filled with tum, or you might arrange the stratified clusters in high-rise apartment buildings, and suppose that the a continuous list and systematically sample that list. rest of the population lives in single-family dwell- ings spread out over the remaining 900 blocks. To the extent that clusters are combined in When we first select our sample of 1⁄10 of the to homogeneous strata, the sampling error at blocks, it’s quite possible that we’ll miss all of the this stage will be reduced. The primary goal of 10 densely packed high-rise blocks. No matter stratification, as before, is homogeneity. what happens in the second stage of sampling, our final sample of households will be grossly unrepre- There’s no reason why stratification couldn’t sentative of the city, comprising only single-family take place at each level of sampling. The elements dwellings. listed within a selected cluster might be stratified before the next stage of sampling. Typically, how- Whenever the clusters sampled are of greatly ever, this is not done. (Recall the assumption of differing sizes, it’s appropriate to use a modified relative homogeneity within clusters.) sampling design called PPS (probability p roportionate to size). This design guards against Probability Proportionate to Size (PPS) Sampling PPS (probability proportionate to size) This refers to a type of multistage cluster sample in which This section introduces you to a more sophisticated clusters are selected, not with equal probabilities form of cluster sampling, one that is used in many (see EPSEM) but with probabilities proportionate to large-scale survey-sampling projects. In the preced- their sizes—as measured by the number of units to ing discussion, I talked about selecting a random or systematic sample of clusters and then a random or
158 ■ Chapter 5: Sampling Logic the problem I’ve just described and still produces a Further refinements to this design make it final sample in which each element has the same a very efficient and effective method for select- chance of selection. ing large cluster samples. For now, however, it’s enough to understand the basic logic involved. As the name suggests, each cluster is given a chance of selection proportionate to its size. Thus, Disproportionate Sampling a city block with 200 households has twice the and Weighting chance of selection as one with only 100 house- holds. Within each cluster, however, a fixed num- Ultimately, a probability sample is representative of ber of elements is selected, say, 5 households per a population if all elements in the population have block. Notice how this procedure results in each an equal chance of selection in that sample. Thus, household having the same probability of selection in each of the preceding discussions, we’ve noted overall. that the various sampling procedures result in an equal chance of selection—even though the ulti- Let’s look at households of two different city mate selection probability is the product of several blocks. Block A has 100 households; Block B has partial probabilities. only 10. In PPS sampling, we would give Block A ten times as good a chance of being selected as More generally, however, a probability s ample Block B. So if, in the overall sample design, Block A is one in which each population element has a has a 1⁄20 chance of being selected, that means known nonzero probability of selection—even Block B would only have a 1⁄200 chance. Notice though different elements may have different that this means that all the households on Block probabilities. If controlled probability sampling A would have a 1⁄20 chance of having their block procedures have been used, any such sample may selected; Block B households have only a 1⁄200 be representative of the population from which chance. it is drawn if each sample element is assigned a weight equal to the inverse of its probability of If Block A is selected and we’re taking 5 house- selection. Thus, where all sample elements have holds from each selected block, then the house- had the same chance of selection, each is given holds on Block A have a 5⁄100 chance of being the same weight: 1. This is called a self-weighting selected into the block’s sample. Because we can sample. multiply probabilities in a case like this, we see that every household on Block A has an overall Sometimes it’s appropriate to give some chance of selection equal to 1⁄ 20 × 5⁄100 = cases more weight than others, a process called 5⁄ 2,000 = 1⁄400. w eighting. Disproportionate sampling and weighting come into play in two basic ways. First, If Block B happens to be selected, on the other you may sample subpopulations disproportion- hand, its households stand a much better chance of ately to ensure sufficient numbers of cases from being among the 5 chosen there: 5⁄10. When this each for analysis. For example, a given city may is combined with their relatively poorer chance of have a suburban area containing one-fourth of having their block selected in the first place, how- its total population. Yet you might be especially ever, they end up with the same chance of selec- interested in a detailed analysis of households tion as those on Block A: 1⁄ 200 × 5⁄10 = 5⁄2,000 = in that area and may feel that one-fourth of this 1⁄400. total sample size would be too few. As a result, you might decide to select the same number of weighting Assigning different weights to cases that households from the suburban area as from the were selected into a sample with different probabili- remainder of the city. Households in the suburban ties of selection. In the simplest scenario, each case is area, then, are given a disproportionately better given a weight equal to the inverse of its probability chance of selection than those located elsewhere of selection. When all cases have the same chance of in the city are. selection, no weighting is necessary.
Multistage Cluster Sampling ■ 159 As long as you analyze the two area samples response from women, and oversampling is a per- separately or comparatively, you need not worry fectly acceptable way of accomplishing that. about the differential sampling. If you want to combine the two samples to create a composite By sampling more women than a straightfor- picture of the entire city, however, you must take ward probability sample would have produced, the the disproportionate sampling into account. If n authors were able to “select” enough women (812) is the number of households selected from each to compare with the men (960). Thus, when they area, then the households in the suburban area report, for example, that 32 percent of the women had a chance of selection equal to n divided by and 66 percent of the men agree that “the amount one-fourth of the total city population. Because of sexual harassment at work is greatly exagger- the total city population and the sample size are ated,” we know that the female response is based the same for both areas, the suburban-area house- on a substantial number of cases. That’s good. holds should be given a weight of 1/4 n, and the There are problems, however. remaining households should be given a weight of 3/4 n. This weighting procedure could be sim To begin with, subscriber surveys are always plified by merely giving a weight of 3 to each of problematic. In this case, the best the research- the households selected outside the suburban ers can hope to talk about is “what subscribers to area. H arvard Business Review think.” In a loose way, it might make sense to think of that population as Here’s an example of the problems that can representing the more sophisticated portion of be created when disproportionate sampling is not corporate management. Unfortunately, the overall accompanied by a weighting scheme. When the response rate was 25 percent. Although that’s quite Harvard Business Review decided to survey its sub- good for subscriber surveys, it’s a low response rate scribers on the issue of sexual harassment at work, in terms of generalizing from probability samples. it seemed appropriate to oversample women be- cause female subscribers were vastly outnumbered Beyond that, however, the disproportionate by male subscribers. Here’s how G. C. Collins and sample design creates another problem. When the Timothy Blodgett explained the matter: authors state that 73 percent of respondents favor company policies against harassment (Collins and We also skewed the sample another way: to Blodgett 1981: 78), that figure is undoubtedly too ensure a representative response from women, high, because the sample contains a disproportion- we mailed a questionnaire to virtually every ately high percentage of women—who are more female subscriber, for a male/female ratio of likely than men to favor such policies. And, when 68% to 32%. This bias resulted in a response of the researchers report that top managers are more 52% male and 44% female (and 4% who gave likely to feel that claims of sexual harassment are no indication of gender)—compared to HBR’s exaggerated than are middle- and lower-level man- U.S. subscriber proportion of 93% male and agers (1981: 81), that finding is also suspect. As the 7% female. researchers report, women are disproportionately represented in lower management. That alone (1981: 78) might account for the apparent differences among levels of management. In short, the failure to take Notice a couple of things in this excerpt. First, account of the oversampling of women confounds it would be nice to know a little more about what all survey results that don’t separate the findings by “virtually every female” means. Evidently, the sex. The solution to this problem would have been authors of the study didn’t send questionnaires to to weight the responses by sex, as described earlier all female subscribers, but there’s no indication of in this section. who was omitted and why. Second, they didn’t use the term representative with its normal social In recent election campaign polling, survey science usage. What they mean, of course, is that weighting has become a controversial topic, as some they wanted to get a substantial or “large enough” polling agencies weight their results on the basis of party affiliation and other variables, whereas others
160 ■ Chapter 5: Sampling Logic do not. Weighting in this instance involves assump- we proceed through the book, we’ll see in greater tions regarding the differential participation of Re- detail how social researchers have found ways to publicans and Democrats in opinion polls and on deal with this issue. election day—plus a determination of how many Republicans and Democrats there are. This is likely The Ethics of Sampling to be a topic of debate among pollsters and politi- cians in the years to come. Alan Reifman has cre- The key purpose of the sampling techniques dis- ated a website devoted to a discussion of this topic cussed in this chapter is to allow researchers to (link to it on your Sociology CourseMate at www make relatively few observations but gain an ac- .cengagebrain.com). curate picture of a much larger population. In the case of quantitative studies using probability sam- Probability Sampling in Review pling, the result should be a statistical profile, based on the sample, that closely mirrors the profile Much of this chapter has been devoted to the that would have been gained from observing the key sampling method used in controlled survey whole population. In addition to using legitimate research: probability sampling. In each of the sampling techniques, researchers should be care- variations examined, we’ve seen that elements ful to point out the possibility of errors: sampling are chosen for study from a population on a error, flaws in the sampling frame, nonresponse basis of random selection with known nonzero error, or anything else that might make the results probabilities. misleading. Depending on the field situation, probability Sometimes, more typically in qualitative stud- sampling can be either very simple or extremely ies, the purpose of sampling may be to tap into the difficult, time-consuming, and expensive. What- breadth of variation within a population rather ever the situation, however, it remains the most ef- than to focus on the “average” or “typical” member fective method for the selection of study elements. of that population. While this is a legitimate and There are two reasons for this. valuable approach, it poses the risk that readers may mistake the display of differences to reflect the First, probability sampling avoids research- distribution of characteristics in the population. In ers’ conscious or unconscious biases in element such a case, the researcher should make sure that selection. If all elements in the population have the reader is not misled. an equal (or unequal and subsequently weighted) chance of selection, there is an excellent chance Main Points that the sample so selected will closely represent the population of all elements. Introduction Second, probability sampling permits estimates • Social researchers must select observations that of sampling error. Although no probability sample will be perfectly representative in all respects, con- will allow them to generalize to people and events trolled selection methods permit the researcher to not observed. Often this involves sampling a selec- estimate the degree of expected error. tion of people to observe. In this lengthy chapter, we’ve taken on a basic • Understanding the logic of sampling is essential to issue in much social research: selecting observa- tions that will tell us something more general than doing social research. the specifics we’ve actually observed. This issue A Brief History of Sampling confronts field researchers, who face more action and more actors than they can observe and record • Sometimes you can and should select probability fully, as well as political pollsters who want to pre- dict an election but can’t interview all voters. As samples using precise statistical techniques, but other times nonprobability techniques are more appropriate.
Key Terms ■ 161 Nonprobability Sampling • Stratification, the process of grouping the mem- • Nonprobability sampling techniques include rely- bers of a population into relatively homogeneous strata before sampling, improves the represen- ing on available subjects, purposive or judgmental tativeness of a sample by reducing the degree of sampling, snowball sampling, and quota sampling. sampling error. In addition, researchers studying a social group may make use of informants. Each of these tech- Multistage Cluster Sampling niques has its uses, but none of them ensures that the resulting sample will be representative of the • Multistage cluster sampling is a relatively complex population being sampled. sampling technique that frequently is used when The Theory and Logic a list of all the members of a population does of Probability Sampling not exist. Typically, researchers must balance the number of clusters and the size of each cluster • Probability-sampling methods provide an excel- to achieve a given sample size. Stratification can be used to reduce the sampling error involved in lent way of selecting representative samples multistage cluster sampling. from large, known populations. These methods counter the problems of conscious and uncon- • Probability proportionate to size (PPS) is a special, scious sampling bias by giving each element in the population a known (nonzero) probability efficient method for multistage cluster sampling. of selection. • If the members of a population have unequal • Random selection is often a key element in prob- probabilities of selection into the sample, research- ability sampling. ers must assign weights to the different observa- tions made, in order to provide a representative • The most carefully selected sample will never picture of the total population. The weight as- signed to a particular sample member should be provide a perfect representation of the population the inverse of its probability of selection. from which it was selected. There will always be some degree of sampling error. Probability Sampling in Review • By predicting the distribution of samples with re- • Probability sampling remains the most effective spect to the target parameter, probability-sampling method for the selection of study elements for methods make it possible to estimate the amount two reasons: it avoids researcher bias in element of sampling error expected in a given sample. selection and it permits estimates of sampling error. • The expected error in a sample is expressed The Ethics of Sampling in terms of confidence levels and confidence intervals. • Because probability sampling always carries a risk Populations and Sampling Frames of error, the researcher must inform readers of any errors that might make results misleading. • A sampling frame is a list or quasi list of the mem- • Sometimes, nonprobability sampling methods are bers of a population. It is the resource used in the selection of a sample. A sample’s representative- used to obtain the breadth of variations in a popu- ness depends directly on the extent to which a lation. In this case, the researcher must ensure sampling frame contains all the members of the that readers do not confuse variations with what’s total population that the sample is intended to typical in the population. represent. K e y T e rm s Types of Sampling Designs The following terms are defined in context in the • Several sampling designs are available to chapter and at the bottom of the page where the term is introduced, as well as in the comprehensive glossary researchers. at the back of the book. • Simple random sampling is logically the most fun- cluster sampling element confidence interval EPSEM damental technique in probability sampling, but it confidence level informant is seldom used in practice. • Systematic sampling involves the selection of every kth member from a sampling frame. This method is more practical than simple random sampling; with a few exceptions, it is functionally equivalent.
162 ■ Chapter 5: Sampling Logic nonprobability sampling sampling frame concerning sampling frames, representativeness, parameter sampling interval and the like? Do you see any solutions? population sampling ratio 5. Using InfoTrac College Edition on your Sociology PPS sampling unit CourseMate at www.cengagebrain.com, locate probability sampling simple random sampling studies using (1) a quota sample, (2) a multistage purposive (judgmental) snowball sampling cluster sample, and (3) a systematic sample. Write sampling statistic a brief description of each study. quota sampling stratification random selection study population S P SS E x e r c i s e s representativeness systematic sampling sampling error weighting See the booklet that accompanies your text for ex- ercises using SPSS (Statistical Package for the Social P r o p o s i n g S o c i a l R e s e a r c h: S a mp l i n g Sciences). There are exercises offered for each chapter, and you’ll also find a detailed primer on using SPSS. In this portion of the proposal, you’ll describe how you’ll select from among all the possible observations Online Study Resources you might make. Depending on the data-collection method you plan to employ, either probability or Access the resources your instructor has assigned. For nonprobability sampling may be more appropriate this book, you can access: to your study. Similarly, this aspect of your proposal may involve the sampling of subjects or informants, CourseMate for The or it could involve the sampling of corporations, cities, Practice of Social Research books, and so forth. Login to CengageBrain.com to access chapter-specific Your proposal, then, must specify what units learning tools including Learning Objectives, Practice you’ll be sampling among, the data you’ll use (such as Quizzes, Videos, Internet Exercises, Flash Cards, Glossaries, a sampling frame) for purposes of your sample selec- Web Links, and more from your Sociology CourseMate. tion, and the actual sampling methods you’ll use. If your professor has assigned Aplia homework: R e v i e w Q u e s ti o n s a n d E x e r c i s e s 1. Sign into your account. 2. After you complete each page of questions, click 1. Review the discussion of the 1948 Gallup Poll that predicted that Thomas Dewey would defeat Harry “Grade It Now” to see detailed explanations of Truman for president. What are some ways Gallup every answer. could have modified his quota sample design to 3. Click “Try Another Version” for an opportunity to avoid the error? improve your score. Visit www.cengagebrain.com to access your account 2. Using Appendix C of this book, select a simple and purchase materials. random sample of 10 numbers in the range of 1 to 9,876. What is each step in the process? 3. What are the steps involved in selecting a multi- stage cluster sample of students taking first-year English in U.S. colleges and universities? 4. In Chapter 8 we’ll discuss surveys conducted on the Internet. Can you anticipate possible problems
CHAPTER 6 From Concept to Measurement chapter o v er v i e w Introduction A Note on Dimensions Defining Variables and Attributes The interrelated steps of Measuring Anything That Exists Levels of Measurement conceptualization, operationalization, Conceptions, Concepts, and Reality Single or Multiple Indicators and measurement allow research- Concepts as Constructs Some Illustrations of ers to turn a general idea for a research topic into useful and valid Conceptualization Operationalization Choices measurements in the real world. Indicators and Dimensions Operationalization Goes On and An essential part of this process The Interchangeability of Indicators involves transforming the relatively Real, Nominal, and Operational On vague terms of ordinary language Definitions into precise objects of study with Creating Conceptual Order Criteria of Measurement Quality well-defined and measurable An Example of Conceptualization: Precision and Accuracy meanings. The Concept of Anomie Reliability Validity Definitions in Descriptive Who Decides What’s Valid? and Explanatory Studies Tension between Reliability and Validity Operationalization Choices Range of Variation The Ethics of Measurement Variations between the Extremes Aplia for The Practice of Social Research After reading, go to “Online Study Resources” at the end of this chapter for
164 ■ Chapter 6: From Concept to Measurement Introduction use the term measurement instead, meaning careful, deliberate observations of the real world for the This chapter and the next one deal with how purpose of describing objects and events in terms of researchers move from a general idea about what the attributes composing a variable. they want to study to effective and well-defined measurements in the real world. This chapter You may have some reservations about the discusses the interrelated processes of conceptu- ability of science to measure the really important alization, operationalization, and measurement. aspects of human social existence. If you’ve read Chapter 7 builds on this foundation to discuss types research reports dealing with something like liber- of measurements that are more complex. alism or religion or prejudice, you may have been dissatisfied with the way the researchers measured Consider a notion such as “satisfaction with col- whatever they were studying. You may have felt lege.” I’m sure you know some people who are very that they were too superficial, that they missed the satisfied, some who are very dissatisfied, and many aspects that really matter most. Maybe they mea- who are between those extremes. Moreover, you can sured religiosity as the number of times a person probably place yourself somewhere along that satis- went to religious services, or maybe they measured faction spectrum. While this probably makes sense liberalism by how people voted in a single election. to you as a general matter, how would you go about Your dissatisfaction would surely have increased if measuring how different students were in this re- you had found yourself being misclassified by the gard, so you could place them along that spectrum? measurement system. There are some comments students make in Your feeling of dissatisfaction reflects an im- conversations (such as “This place sucks”) that portant fact about social research: Most of the var- would tip you off as to where they stood. Or, in iables we want to study don’t actually exist in the a more active effort, you can probably think of way that rocks exist. Indeed, they are made up. questions you might ask students to learn about Moreover, they seldom have a single, unambigu- their satisfaction (such as “How satisfied are you ous meaning. with . . . ?”). Perhaps there are certain behaviors (class attendance, use of campus facilities, setting To see what I mean, suppose we want to study the dean’s office on fire) that would suggest differ- political party affiliation. To measure this variable, ent levels of satisfaction. As you think about ways we might consult the list of registered voters to of measuring satisfaction with college, you are en- note whether the people we were studying were gaging in the subject matter of this chapter. registered as Democrats or Republicans and take that as a measure of their party affiliation. But we We begin by confronting the hidden concern could also simply ask someone what party they people sometimes have about whether it’s truly identify with and take their response as our mea- possible to measure the stuff of life: love, hate, sure. Notice that these two different measurement prejudice, religiosity, radicalism, alienation. The possibilities reflect somewhat different definitions answer is yes, but it will take a few pages to see of political party affiliation. They might even pro- how. Once we establish that researchers can mea- duce different results: Someone may have reg- sure anything that exists, we’ll turn to the steps istered as a Democrat years ago but gravitated involved in doing just that. more and more toward a Republican philosophy over time. Or someone who is registered with Measuring Anything That Exists neither political party may, when asked, say she is affiliated with the one she feels the most kinship Earlier in this book, I said that one of the two pil- with. lars of science is observation. Because this word can suggest a casual, passive activity, scientists often Similar points apply to religious affiliation. Some- times this variable refers to official membership in
Measuring Anything That Exists ■ 165 a particular church, temple, mosque, and so forth; With additional experience, we notice some- other times it simply means whatever religion, if thing more. A lot of the people who call African any, you identify yourself with. Perhaps to you it Americans ugly names also seem to want women to means something else, such as attendance at reli- “stay in their place.” They are also likely to think mi- gious services. norities are inferior to the majority and that women are inferior to men. These several tendencies often The truth is that neither party affiliation nor appear together in the same people and also have religious affiliation has any real meaning, if by “real” something in common. At some point, someone we mean corresponding to some objective aspect of had a bright idea: “Let’s use the word prejudiced as a reality. These variables do not exist in nature. They shorthand notation for people like that. We can use are merely terms we’ve made up and assigned the term even if they don’t do all those things—as specific meanings to for some purpose, such as long as they’re pretty much like that.” doing social research. Being basically agreeable and interested in But, you might object, political affiliation and efficiency, we went along with the system. That’s religious affiliation—and a host of other things social where “prejudice” came from. We never observed researchers are interested in, such as prejudice or it. We just agreed to use it as a shortcut, a name compassion—have some reality. After all, research- that represents a collection of apparently related ers make statements about them, such as “In phenomena that we’ve each observed in the course H appytown, 55 percent of the adults affiliate with of life. In short, we made it up. the Republican Party, and 45 percent of them are Episcopalians. Overall, people in Happytown are Here’s another clue that prejudice isn’t some- low in prejudice and high in compassion.” Even thing that exists apart from our rough agreement ordinary people, not just social researchers, have to use the term in a certain way. Each of us devel- been known to make statements like that. If these ops our own mental image of what the set of real things don’t exist in reality, what is it that we’re phenomena we’ve observed represents in general measuring and talking about? and what these phenomena have in common. When I say the word prejudice, it evokes a mental What indeed? Let’s take a closer look by image in your mind, just as it evokes one in mine. c onsidering a variable of interest to many social It’s as though file drawers in our minds contained researchers (and many other people as thousands of sheets of paper, with each sheet of well)—prejudice. paper labeled in the upper right-hand corner. A sheet of paper in each of our minds has the term Conceptions, Concepts, prejudice on it. On your sheet are all the things and Reality you’ve been told about prejudice and everything you’ve observed that seems to be an example of it. As we wander down the road of life, we observe My sheet has what I’ve been told about it plus all a lot of things and know they are real through the things I’ve observed that seem examples of it— our observations, and we hear reports from other and mine isn’t the same as yours. people that seem real. For example: The technical term for those mental images, • We personally hear people say nasty things those sheets of paper in our mental file drawers, is conception. That is, I have a conception of preju- about minority groups. dice, and so do you. We can’t communicate these mental images directly, so we use the terms written • We hear people say that women are inferior in the upper right-hand corner of our own mental sheets of paper as a way of communicating about to men. our conceptions and the things we observe that are related to those conceptions. These terms make • We read that women and minorities earn less it possible for us to communicate and eventually for the same work. • We learned about “ethnic cleansing” and wars in which one ethnic group tries to eradicate another.
166 ■ Chapter 6: From Concept to Measurement Research in Real Life Gender and Race in City Streets You can read excerpts of the book online and can hear Anderson discuss the book in an interview with BBC’s Laurie Taylor at the links on In the early 1970s, Elijah Anderson spent three years observing life in a black, your Sociology CourseMate at www.cengagebrain.com. working-class neighborhood in South Chicago, focusing on Jelly’s, a combina- tion bar and liquor store.While some people still believe that impoverished Elijah Anderson, A Place on the Corner: A Study of Black Street Corner Men neighborhoods in the inner city are socially chaotic and disorganized, (University of Chicago Press, 2004). Anderson’s study and others like it have clearly demonstrated a definite social structure there that guides the behavior of its participants. Much of his interest centered on systems of social status and how the 55 or so regulars at Jelly’s worked those systems to establish themselves among their peers. In the second edition of this classic study of urban life, Elijah Anderson returned to Jelly’s and the surrounding neighborhood. There he found several changes, largely due to the outsourcing of manufactur- ing jobs overseas that has brought economic and mental depression to many of the residents. These changes, in turn, had also altered the nature of social organization. For a research methods student, the book offers many insights into the process of establishing rapport with people being observed in their natural surroundings. Further, Anderson offers excellent examples of how concepts are established in qualitative research. agree on what we specifically mean by those words—ranging from 1 to 400. Several sources, terms. In social research, the process of coming moreover, will suggest that if the Inuit have sev- to an agreement about what terms mean is eral words for snow, so does English. Cecil Adams, conceptualization, and the result is called a for example, lists “snow, slush, sleet, hail, powder, concept. See “Research in Real Life” for a glimpse at hard pack, blizzard, flurries, flake, dusting, crust, a project that reveals a lot about conceptualization. avalanche, drift, frost, and iceberg” (Straight Dope 2001). This illustrates the ambiguities in the field Perhaps you’ve heard some reference to the with regard to the concepts and words that we use many words Eskimos have for snow, as an example in everyday communications and that also serve as of how environment can shape language. Here’s the grounding for social research. an exercise you might enjoy when you’re ready to take a break from reading. Search the web Let’s take another example of a conception. for “Eskimo words for snow.” You may be sur- S uppose that I’m going to meet someone named prised by what you find. You’re likely to discover Pat, whom you already know. I ask you what Pat is wide disagreement on the number of, say, Inuit, like. Now suppose that you’ve seen Pat help lost chil- dren find their parents and put a tiny bird back in its conceptualization The mental process whereby nest. Pat got you to take turkeys to poor families on fuzzy and imprecise notions (concepts) are made Thanksgiving and to visit a children’s hospital on more specific and precise. So you want to study Christmas. You’ve seen Pat weep through a movie prejudice. What do you mean by “prejudice”? Are about a mother overcoming adversities to save and there different kinds of prejudice? What are they? protect her child. As you search through your mental files, you may find all or most of those phenomena recorded on a single sheet labeled “compassionate.”
Measuring Anything That Exists ■ 167 You look over the other entries on the page, and you his or her mental images to correspond with such find they seem to provide an accurate description of agreements. But because all of us have different Pat. So you say, “Pat is compassionate.” experiences and observations, no two people end up with exactly the same set of entries on any Now I leaf through my own mental file drawer sheet in their file systems. If we want to measure until I find a sheet marked “compassionate.” I then “prejudice” or “compassion,” we must first stipulate look over the things written on my sheet, and I say, what, exactly, counts as prejudice or compassion “Oh, that’s nice.” I now feel I know what Pat is like, for our purposes. but my expectations reflect the entries on my file sheet, not yours. Later, when I meet Pat, I happen Returning to the assertion made at the outset to find that my own experiences correspond to the of this chapter, we can measure anything that’s entries I have on my “compassionate” file sheet, real. We can measure, for example, whether Pat and I say that you sure were right. actually puts the little bird back in its nest, visits the hospital on Christmas, weeps at the movie, or But suppose my observations of Pat contradict refuses to contribute to saving the whales. All of the things I have on my file sheet. I tell you that those behaviors exist, so we can measure them. I don’t think Pat is very compassionate, and we But is Pat really compassionate? We can’t answer begin to compare notes. that question; we can’t measure compassion in any objective sense, because compassion doesn’t exist You say, “I once saw Pat weep through a movie in the way that those things I just described exist. about a mother overcoming adversity to save and Compassion exists only in the form of the agree- protect her child.” I look at my “compassionate ments we have about how to use the term in sheet” and can’t find anything like that. Looking communicating about things that are real. elsewhere in my file, I locate that sort of phenom- enon on a sheet labeled “sentimental.” I retort, Concepts as Constructs “That’s not compassion. That’s just sentimentality.” If you recall the discussions of postmodernism in To further strengthen my case, I tell you that Chapter 3, you’ll recognize that some people would I saw Pat refuse to give money to an organiza- object to the degree of “reality” I’ve allowed in tion dedicated to saving whales from extinction. the preceding comments. Did Pat “really” visit the “That represents a lack of compassion,” I argue. hospital on Christmas? Does the hospital “really” You search through your files and find saving the exist? Does Christmas? Though we aren’t going whales on two sheets—“environmental activism” to be radically postmodern in this chapter, I think and “cross-species dating”—and you say so. Even- you’ll recognize the importance of an intellectually tually, we set about comparing the entries we have tough view of what’s real and what’s not. (When on our respective sheets labeled “compassionate.” the intellectual going gets tough, the tough become We then discover that many of our mental images social scientists.) corresponding to that term differ. In this context, Abraham Kaplan (1964) dis- In the big picture, language and communica- tinguishes three classes of things that scientists tion work only to the extent that you and I have measure. The first class is direct observables: those considerable overlap in the kinds of entries we things we can observe rather simply and directly, have on our corresponding mental file sheets. The like the color of an apple or the check mark on a similarities we have on those sheets represent the questionnaire. The second class, indirect observables, agreements existing in our society. As we grow up, require “relatively more subtle, complex, or indi- we’re told approximately the same thing when rect observations” (1964: 55). We note a person’s we’re first introduced to a particular term, though check mark beside “female” in a questionnaire and our nationality, gender, race, ethnicity, region, have indirectly observed that person’s sex. H istory language, or other cultural factors may shade our books or minutes of corporate board meetings understanding of concepts. Dictionaries formalize the agreements our soci- ety has about such terms. Each of us, then, shapes
168 ■ Chapter 6: From Concept to Measurement Table 6-1 What Social Scientists Measure Direct observables Examples Indirect observables Constructs Physical characteristics (sex, height, skin color) of a person being observed and/or interviewed Characteristics of a person as indicated by answers given in a self-administered questionnaire Level of alienation, as measured by a scale that is created by combining several direct and/or indirect observables provide indirect observations of past social actions. Usually, however, we fall into the trap of be- Finally, the third class of observables consists of lieving that terms for constructs do have intrinsic constructs—theoretical creations that are based on meaning, that they name real entities in the world. observations but that cannot be observed directly That danger seems to grow stronger when we begin or indirectly. A good example is intelligence quo- to take terms seriously and attempt to use them tient, or IQ. It is constructed mathematically from precisely. Further, the danger is all the greater in observations of the answers given to a large num- the presence of experts who appear to know more ber of questions on an IQ test. No one can directly than we do about what the terms really mean: It’s or indirectly observe IQ. It is no more a “real” char- easy to yield to authority in such a situation. acteristic of people than is compassion or prejudice. See Table 6-1 for more examples of what social Once we assume that terms like prejudice and scientists measure. compassion have real meanings, we begin the tor- tured task of discovering what those real meanings Kaplan (1964: 49) defines concept as a “family are and what constitutes a genuine measurement of conceptions.” A concept is, as Kaplan notes, a of them. Regarding constructs as real is called construct, something we create. Concepts such as reification. The reification of concepts in day-to-day compassion and prejudice are constructs created life is quite common. In science, we want to be quite from your conception of them, my conception of clear about what it is we are actually measuring, but them, and the conceptions of all those who have this aim brings a pitfall with it. Settling on the “best” ever used these terms. They cannot be observed way of measuring a variable in a particular study directly or indirectly, because they don’t exist. We may imply that we’ve discovered the “real” mean- made them up. ing of the concept involved. In fact, concepts have no real, true, or objective meanings—only those we To summarize, concepts are constructs derived agree are best for a particular purpose. by mutual agreement from mental images (con- ceptions). Our conceptions summarize collections Does this discussion imply that compassion, of seemingly related observations and experiences. prejudice, and similar constructs can’t be mea- Although the observations and experiences are sured? Interestingly, the answer is no. (And a good real, at least subjectively, conceptions, and the con- thing, too, or a lot of us social researcher types cepts derived from them, are only mental creations. would be out of work.) I’ve said that we can mea- The terms associated with concepts are merely sure anything that’s real. Constructs aren’t real devices created for the purposes of filing and com- in the way that trees are real, but they do have munication. A term such as prejudice is, objectively another important virtue: They are useful. That speaking, only a collection of letters. It has no in- is, they help us organize, communicate about, trinsic reality beyond that. Is has only the meaning and understand things that are real. They help us we agree to give it. make predictions about real things. Some of those
Conceptualization ■ 169 predictions even turn out to be true. Constructs presence or absence of the concept we’re studying. can work this way because, although not real or Here’s an example. observable in themselves, they have a definite rela- tionship to things that are real and observable. The We might agree that visiting children’s hospitals bridge from direct and indirect observables to useful during Christmas and Hanukkah is an indicator of constructs is the process called conceptualization. compassion. Putting little birds back in their nests might be agreed on as another indicator, and so Conceptualization forth. If the unit of analysis for our study is the individual person, we can then observe the pres- As we’ve seen, day-to-day communication usually ence or absence of each indicator for each person occurs through a system of vague and general under study. Going beyond that, we can add up agreements about the use of terms. Although the number of indicators of compassion observed you and I do not agree completely about the use of for each individual. We might agree on ten specific the term compassionate, I’m probably safe in assum- indicators, for example, and find six present in our ing that Pat won’t pull the wings off flies. A wide study of Pat, three for John, nine for Mary, and so range of misunderstandings and conflict—from the forth. interpersonal to the international—is the price we pay for our imprecision, but somehow we muddle Returning to our question about whether men through. Science, however, aims at more than or women are more compassionate, we might muddling; it cannot operate in a context of such calculate that the women we studied displayed an imprecision. average of 6.5 indicators of compassion, the men an average of 3.2. On the basis of our quantitative The process through which we specify what analysis of group difference, we might therefore we mean when we use particular terms in research conclude that women are, on the whole, more is called conceptualization. Suppose we want to compassionate than men. find out, for example, whether women are more compassionate than men. I suspect many people Usually, though, it’s not that simple. Imagine assume this is the case, but it might be interest- you’re interested in understanding a small fun- ing to find out if it’s really so. We can’t meaning- damentalist religious cult, particularly their harsh fully study the question, let alone agree on the views on various groups: gays, nonbelievers, answer, without some working agreements about feminists, and others. In fact, they suggest that the meaning of compassion. They are “working” anyone who refuses to join their group and abide agreements in the sense that they allow us to work by its teachings will “burn in hell.” In the context on the question. We don’t need to agree or even of your interest in compassion, they don’t seem pretend to agree that a particular specification is to have much. And yet, the group’s literature ultimately the best one. often speaks of their compassion for others. You want to explore this seeming paradox. Conceptualization, then, produces a specific, agreed-on meaning for a concept for the purposes To pursue this research interest, you might of research. This process of specifying exact mean- arrange to interact with cult members, getting to ing involves describing the indicators we’ll be using know them and learning more about their views. to measure our concept and the different aspects of You could tell them you were a social researcher the concept, called dimensions. interested in learning about their group, or per- haps you would just express an interest in learning Indicators and Dimensions more, without saying why. Conceptualization gives definite meaning to a con- indicator An observation that we choose to con- cept by specifying one or more indicators of what sider as a reflection of a variable we wish to study. we have in mind. An indicator is a sign of the Thus, for example, attending religious services might be considered an indicator of religiosity.
170 ■ Chapter 6: From Concept to Measurement In the course of your conversations with group dimension” of compassion and the “action di- members and perhaps attendance of religious mension” of compassion. In a different grouping services, you would put yourself in situations scheme, we might distinguish “compassion for where you could come to understand what the humans” from “compassion for animals.” Or we cult members mean by compassion. You might might see compassion as helping people have learn, for example, that members of the group what we want for them versus what they want were so deeply concerned about sinners burning in for themselves. Still differently, we might distin- hell that they were willing to be aggressive, even guish compassion as forgiveness from compas- violent, to make people change their sinful ways. sion as pity. Within their own paradigm, then, cult members would see beating up gays, prostitutes, and abor- Thus, we could subdivide compassion into sev- tion doctors as acts of compassion. eral clearly defined dimensions. A complete con- ceptualization involves both specifying dimensions Social researchers focus their attention on and identifying the various indicators for each. the meanings that the people under study give to words and actions. Doing so can often clarify the When Jonathan Jackson (2005: 301) set out behaviors observed: At least now you understand to measure “fear of crime,” he considered seven how the cult can see violent acts as compassionate. different dimensions: On the other hand, paying attention to what words and actions mean to the people under study almost • The frequency of worry about becom- always complicates the concepts researchers are interested in. (We’ll return to this issue when we ing a victim of three personal crimes and discuss the validity of measures, toward the end of two property crimes in the immediate this chapter.) n eighbourhood . . . Whenever we take our concepts seriously and • Estimates of likelihood of falling victim to each set about specifying what we mean by them, we discover disagreements and inconsistencies. Not crime locally only do you and I disagree, but each of us is likely to find a good deal of muddiness within our own • Perceptions of control over the possibility of mental images. If you take a moment to look at what you mean by compassion, you’ll probably becoming a victim of each crime locally find that your image contains several kinds of compassion. That is, the entries on your mental file • Perceptions of the seriousness of the conse- sheet can be combined into groups and subgroups, say, compassion toward friends, co-religionists, quences of each crime humans, and birds. You may also find several dif- ferent strategies for making combinations. For • Beliefs about the incidence of each crime example, you might group the entries into feelings and actions. locally The technical term for such groupings is • Perceptions of the extent of social physical inci- dimension, a specifiable aspect of a concept. For instance, we might speak of the “feeling vilities in the neighbourhood dimension A specifiable aspect of a concept. “Reli- • Perceptions of community cohesion, including giosity,” for example, might be specified in terms of a belief dimension, a ritual dimension, a devotional informal social control and trust/social capital dimension, a knowledge dimension, and so forth. Sometimes conceptualization aimed at identi- fying different dimensions of a variable leads to a different kind of distinction. We may conclude that we’ve been using the same word for meaningfully distinguishable concepts. In the following example, the researchers find (1) that “violence” is not a sufficient description of “genocide” and (2) that the concept “genocide” itself comprises several distinct phenomena. Let’s look at the process they went through to come to this conclusion. When Daniel Chirot and Jennifer Edwards attempted to define the concept of “genocide,”
Conceptualization ■ 171 they found existing assumptions were not precise worries that the growing Albanian population enough for their purposes: of Kosovo was gaining political strength through numbers. Similarly, the Hutu attempt The United Nations originally defined it as to eradicate the Tutsis of Rwanda grew out an attempt to destroy “in whole or in part, a of a fear that returning Tutsi refugees would national, ethnic, racial, or religious group.” If seize control of the country. Often intergroup genocide is distinct from other types of vio- fears such as these grow out of long histories of lence, it requires its own unique explanation. atrocities, often inflicted in both directions. (2003: 14) 4. Purification: The Nazi Holocaust, probably the most publicized case of genocide, was intended Notice the final comment in this excerpt, as it as a purification of the “Aryan race.” While provides an important insight into why research- Jews were the main target, gypsies, homosexu- ers are so careful in specifying the concepts they als, and many other groups were also included. study. If genocide, such as the Holocaust, were Other examples include the Indonesian witch- simply another example of violence, like assaults hunt against Communists in 1965–1966 and and homicides, then what we know about violence the attempt to eradicate all non-Khmer Cam- in general might explain genocide. If it differs from bodians under Pol Pot in the 1970s. other forms of violence, then we may need a differ- ent explanation for it. So, the researchers began by No single theory of genocide could explain these suggesting that “genocide” was a concept distinct various forms of mayhem. Indeed, this act of con- from “violence” for their purposes. ceptualization suggests four distinct phenomena, each needing a different set of explanations. Then, as Chirot and Edwards examined histori- cal instances of genocide, they began concluding Specifying the different dimensions of a con- that the motivations for launching genocidal may- cept often paves the way for a more sophisticated hem differed sufficiently to represent four distinct understanding of what we’re studying. We might phenomena that were all called “genocide” (2003: observe, for example, that women are more com- 15–18). passionate in terms of feelings, and men more so in terms of actions—or vice versa. Whichever 1. Convenience: Sometimes the attempt to eradi- turned out to be the case, we would not be able to cate a group of people serves a function for the say whether men or women are really more com- eradicators, such as Julius Caesar’s attempt to passionate. Our research would have shown that eradicate tribes defeated in battle, fearing they there is no single answer to the question. That would be difficult to rule. Or when gold was alone represents an advance in our understanding discovered on Cherokee land in the Southeast- of reality. To get a better feel for concepts, vari- ern United States in the early nineteenth cen- ables, and indicators, go to the General Social tury, the Cherokee were forcibly relocated to Survey codebook and explore some of the ways Oklahoma in an event known as the “Trail of the researchers have measured various concepts Tears,” which ultimately killed as many as half (see the link at your Sociology CourseMate at of those forced to leave. www.cengagebrain.com). 2. Revenge: When the Chinese of Nanking bravely The Interchangeability resisted the Japanese invaders in the early of Indicators years of World War II, the conquerors felt they had been insulted by those they regarded as There is another way that the notion of indicators inferior beings. Tens of thousands were slaugh- can help us in our attempts to understand real- tered in the “Rape of Nanking” in 1937–1938. ity by means of “unreal” constructs. Suppose, for the moment, that you and I have compiled a list 3. Fear: The ethnic cleansing that recently oc- curred in the former Yugoslavia was at least partly motivated by economic competition and
172 ■ Chapter 6: From Concept to Measurement of 100 indicators of compassion and its various di- conceptualization by looking at some of the ways mensions. Suppose further that we disagree widely social researchers provide standards, consistency, on which indicators give the clearest evidence and commonality for the meanings of terms. of compassion or its absence. If we pretty much agree on some indicators, we could focus our at- Real, Nominal, tention on those, and we would probably agree on and Operational Definitions the answer they provided. We would then be able to say that some people are more compassionate As we have seen, the design and execution of social than others in some dimension. But suppose we research requires us to clear away the confusion don’t really agree on any of the possible indicators. over concepts and reality. To this end, logicians and Surprisingly, we can still reach an agreement on scientists have found it useful to distinguish three whether men or women are the more compas- kinds of definitions: real, nominal, and operational. sionate. How we do that has to do with the inter- changeability of indicators. The first of these reflects the reification of terms. As Carl Hempel cautions, The logic works like this. If we disagree totally on the value of the indicators, one solution would A “real” definition, according to traditional be to study all of them. Suppose that women turn logic, is not a stipulation determining the out to be more compassionate than men on all 100 meaning of some expression but a statement indicators—on all the indicators you favor and on of the “essential nature” or the “essential at- all of mine. Then we would be able to agree that tributes” of some entity. The notion of essential women are more compassionate than men, even nature, however, is so vague as to render this though we still disagree on exactly what compas- characterization useless for the purposes of rig- sion means in general. orous inquiry. The interchangeability of indicators means (1952: 6) that if several different indicators all represent, to some degree, the same concept, then all of them In other words, trying to specify the “real” meaning will behave the same way that the concept would of concepts only leads to a quagmire: It mistakes a behave if it were real and could be observed. Thus, construct for a real entity. given a basic agreement about what “compas- sion” is, if women are generally more compas- The specification of concepts in scientific in- sionate than men, we should be able to observe quiry depends instead on nominal and operational that difference by using any reasonable measure definitions. A nominal definition is one that is of compassion. If, on the other hand, women are simply assigned to a term without any claim that more compassionate than men on some indica- the definition represents a “real” entity. Nominal tors but not on others, we should see if the two definitions are arbitrary—I could define compas- sets of indicators represent different dimensions of sion as “plucking feathers off helpless birds” if I compassion. wanted to—but they can be more or less useful. For most purposes, especially communication, that You have now seen the fundamental logic last definition of compassion would be pretty use- of conceptualization and measurement. The less. Most nominal definitions represent some con- discussions that follow are mainly refinements sensus, or convention, about how a particular term and extensions of what you’ve just read. Before is to be used. turning to a technical elaboration of measure- ment, however, we need to fill out the picture of An operational definition, as you may remem- ber from Chapter 4, specifies precisely how a specification The process through which concepts concept will be measured—that is, the operations are made more specific. we’ll perform. An operational definition is nomi- nal rather than real, but it has the advantage of achieving maximum clarity about what a concept means in the context of a given study. In the midst
Conceptualization ■ 173 of disagreement and confusion over what a term meaning of the text as it is anticipated. The “really” means, we can specify a working definition closer determination of the meaning of the for the purposes of an inquiry. Wishing to examine separate parts may eventually change the origi- socioeconomic status (SES) in a study, for example, nally anticipated meaning of the totality, which we may simply specify that we are going to treat again influences the meaning of the separate SES as a combination of income and educational parts, and so on. attainment. In this decision, we rule out other pos- sible aspects of SES: occupational status, money in (Kvale 1996: 47) the bank, property, lineage, lifestyle, and so forth. Our findings will then be interesting to the extent Consider the concept “prejudice.” Suppose you that our definition of SES is useful for our purpose. needed to write a definition of the term. You might start out thinking about racial/ethnic prejudice. At Creating Conceptual Order some point you would realize you should prob- ably allow for gender prejudice, religious prejudice, The clarification of concepts is a continuing pro- antigay prejudice, and the like in your definition. cess in social research. Catherine Marshall and Examining each of these specific types of prejudice Gretchen Rossman (1995: 18) speak of a “concep- would affect your overall understanding of the tual funnel” through which a researcher’s general concept. As your general understanding interest becomes increasingly focused. Thus, a changed, however, you would likely see each of general interest in social activism could narrow to the individual forms somewhat differently. “individuals who are committed to empowerment and social change” and further focus on discovering The continual refinement of concepts occurs “what experiences shaped the development of fully in all social research methods. Often you will find committed social activists.” This focusing process is yourself refining the meaning of important con- inescapably linked to the language we use. cepts even as you write up your final report. In some forms of qualitative research, the Although conceptualization is a continuing clarification of concepts is a key element in the process, it is vital to address it specifically at the collection of data. Suppose you were conducting beginning of any study design, especially rigorously interviews and observations of a radical political structured research designs such as surveys and group devoted to combating oppression in U.S. experiments. In a survey, for example, operational- society. Imagine how the meaning of oppression ization results in a commitment to a specific set of would shift as you delved more and more deeply questionnaire items that will represent the concepts into the members’ experiences and worldviews. under study. Without that commitment, the study For example, you might start out thinking of op- could not proceed. pression in physical and perhaps economic terms. The more you learned about the group, however, Even in less-structured research methods, the more you might appreciate the possibility of however, it’s important to begin with an initial set psychological oppression. of anticipated meanings that can be refined during data collection and interpretation. No one seriously The same point applies even to contexts where believes we can observe life with no preconcep- meanings might seem more fixed. In the analysis tions; for this reason, scientific observers must be of textual materials, for example, social researchers conscious of and explicit about these conceptual sometimes speak of the “hermeneutic circle,” a starting points. c yclical process of ever-deeper understanding. Let’s explore initial conceptualization the way The understanding of a text takes place it applies to structured inquiries such as surveys through a process in which the meaning of the and experiments. Though specifying nominal separate parts is determined by the global definitions focuses our observational strategy, it does not allow us to observe. As a next step we must specify exactly what we are going to observe, how we will do it, and what interpretations we are
174 ■ Chapter 6: From Concept to Measurement Table 6-2 Progression of Measurement Measurement Step Example: Social Class Conceptualization What are the different meanings and dimensions of the concept“social class”? Nominal definition For our study, we will define“social class”as representing economic differences: specifically, income. Operational definition We will measure economic differences via responses to the survey question“What was your a nnual income, before taxes, last year?” Measurements in the real world The interviewer will ask,“What was your annual income, before taxes, last year?” going to place on various possible observations. means to specific measurements in a fully struc- All these further specifications make up the opera- tured scientific study. tional definition of the concept. An Example of Conceptualization: In the example of socioeconomic status, we The Concept of Anomie might decide to ask survey respondents two questions, corresponding to the decision to mea- To bring this discussion of conceptualization in sure SES in terms of income and educational research together, let’s look briefly at the history attainment: of a specific social science concept. Researchers studying urban riots are often interested in the part 1. What was your total family income during the played by feelings of powerlessness. Social scientists past 12 months? sometimes use the word anomie in this context. This term was first introduced into social science by 2. What is the highest level of school you Emile Durkheim, the great French sociologist, in completed? his classic 1897 study, Suicide. To organize our data, we’d probably want Using only government publications on to specify a system for categorizing the answers suicide rates in different regions and countries, people give us. For income, we might use catego- Durkheim produced a work of analytic genius. ries such as “under $5,000,” “$5,000 to $10,000,” To determine the effects of religion on suicide, and so on. Educational attainment might be simi- he compared the suicide rates of predominantly larly grouped in categories: less than high school, Protestant countries with those of predominantly high school, college, graduate degree. Finally, we Catholic ones, Protestant regions of Catholic coun- would specify the way a person’s responses to these tries with Catholic regions of Protestant countries, two questions would be combined in creating a and so forth. To determine the possible effects of measure of SES. the weather, he compared suicide rates in north- ern and southern countries and regions, and he In this way we would create a working and examined the different suicide rates across the workable definition of SES. Although others might months and seasons of the year. Thus, he could disagree with our conceptualization and operation- draw conclusions about a supremely individualistic alization, the definition would have one essential and personal act without having any data about scientific virtue: It would be absolutely specific and the individuals engaging in it. unambiguous. Even if someone disagreed with our definition, that person would have a good idea At a more general level, Durkheim suggested how to interpret our research results, because what that suicide also reflects the extent to which a we meant by SES—reflected in our analyses and society’s agreements are clear and stable. Noting conclusions—would be precise and clear. that times of social upheaval and change often Table 6-2 shows the progression of measure- ment steps from our vague sense of what a term
Conceptualization ■ 175 present individuals with grave uncertainties about Powell went on to suggest there were two dis- what is expected of them, Durkheim suggested tinct kinds of anomia and to examine how the two that such uncertainties cause confusion, anxiety, rose out of different occupational experiences to re- and even self-destruction. To describe this societal sult at times in suicide. In his study, however, Powell condition of normlessness, Durkheim chose the did not measure anomia per se; he studied the re- term anomie. Durkheim did not make this word up. lationship between suicide and occupation, making Used in both German and French, it literally meant inferences about the two kinds of anomia. Thus, the “without law.” The English term anomy had been study did not provide an operational definition of used for at least three centuries before Durkheim to anomia, only a further conceptualization. mean disregard for divine law. However, Durkheim created the social science concept of anomie. Although many researchers have offered oper- ational definitions of anomia, one name stands out In the years that have followed the publica- over all. Two years before Powell’s article appeared, tion of Suicide, social scientists have found anomie Leo Srole (1956) published a set of questionnaire a useful concept, and many have expanded on items that he said provided a good measure of Durkheim’s use. Robert Merton, in a classic article anomia as experienced by individuals. It consists of entitled “Social Structure and Anomie” (1938), five statements that subjects were asked to agree or concluded that anomie results from a disparity be- disagree with: tween the goals and means prescribed by a society. Monetary success, for example, is a widely shared 1. In spite of what some people say, the lot of goal in our society, yet not all individuals have the the average man is getting worse. resources to achieve it through acceptable means. An emphasis on the goal itself, Merton suggested, 2. It’s hardly fair to bring children into the produces normlessness, because those denied the world with the way things look for the traditional avenues to wealth go about getting future. it through illegitimate means. Merton’s discussion, then, could be considered a further conceptualiza- 3. Nowadays a person has to live pretty much tion of the concept of anomie. for today and let tomorrow take care of itself. Although Durkheim originally used the con- 4. These days a person doesn’t really know cept of anomie as a characteristic of societies, as did who he can count on. Merton after him, other social scientists have used it to describe individuals. To clarify this distinction, 5. There’s little use writing to public officials some scholars have chosen to use anomie in refer- because they aren’t really interested in the ence to its original, societal meaning and to use the problems of the average man. term anomia in reference to the individual charac- teristic. In a given society, then, some individuals ex- (1956: 713) perience anomia, and others do not. Elwin Powell, writing 20 years after Merton, provided the follow- In the half-century following its publication, ing conceptualization of anomia (though using the the Srole scale has become a research staple for term anomie) as a characteristic of individuals: social scientists. You’ll likely find this particular operationalization of anomia used in many of the When the ends of action become contradic- research projects reported in academic journals. tory, inaccessible or insignificant, a condition of anomie arises. Characterized by a general loss This abbreviated history of anomie and anomia of orientation and accompanied by feelings of as social science concepts illustrates several points. “emptiness” and apathy, anomie can be simply First, it’s a good example of the process through conceived as meaninglessness. which general concepts become operationalized measurements. This is not to say that the issue of (1958: 132) how to operationalize anomie/anomia has been resolved once and for all. Scholars will surely con- tinue to reconceptualize and reoperationalize these concepts for years to come, continually seeking more-useful measures.
Search
Read the Text Version
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
- 18
- 19
- 20
- 21
- 22
- 23
- 24
- 25
- 26
- 27
- 28
- 29
- 30
- 31
- 32
- 33
- 34
- 35
- 36
- 37
- 38
- 39
- 40
- 41
- 42
- 43
- 44
- 45
- 46
- 47
- 48
- 49
- 50
- 51
- 52
- 53
- 54
- 55
- 56
- 57
- 58
- 59
- 60
- 61
- 62
- 63
- 64
- 65
- 66
- 67
- 68
- 69
- 70
- 71
- 72
- 73
- 74
- 75
- 76
- 77
- 78
- 79
- 80
- 81
- 82
- 83
- 84
- 85
- 86
- 87
- 88
- 89
- 90
- 91
- 92
- 93
- 94
- 95
- 96
- 97
- 98
- 99
- 100
- 101
- 102
- 103
- 104
- 105
- 106
- 107
- 108
- 109
- 110
- 111
- 112
- 113
- 114
- 115
- 116
- 117
- 118
- 119
- 120
- 121
- 122
- 123
- 124
- 125
- 126
- 127
- 128
- 129
- 130
- 131
- 132
- 133
- 134
- 135
- 136
- 137
- 138
- 139
- 140
- 141
- 142
- 143
- 144
- 145
- 146
- 147
- 148
- 149
- 150
- 151
- 152
- 153
- 154
- 155
- 156
- 157
- 158
- 159
- 160
- 161
- 162
- 163
- 164
- 165
- 166
- 167
- 168
- 169
- 170
- 171
- 172
- 173
- 174
- 175
- 176
- 177
- 178
- 179
- 180
- 181
- 182
- 183
- 184
- 185
- 186
- 187
- 188
- 189
- 190
- 191
- 192
- 193
- 194
- 195
- 196
- 197
- 198
- 199
- 200
- 201
- 202
- 203
- 204
- 205
- 206
- 207
- 208
- 209
- 210
- 211
- 212
- 213
- 214
- 215
- 216
- 217
- 218
- 219
- 220
- 221
- 222
- 223
- 224
- 225
- 226
- 227
- 228
- 229
- 230
- 231
- 232
- 233
- 234
- 235
- 236
- 237
- 238
- 239
- 240
- 241
- 242
- 243
- 244
- 245
- 246
- 247
- 248
- 249
- 250
- 251
- 252
- 253
- 254
- 255
- 256
- 257
- 258
- 259
- 260
- 261
- 262
- 263
- 264
- 265
- 266
- 267
- 268
- 269
- 270
- 271
- 272
- 273
- 274
- 275
- 276
- 277
- 278
- 279
- 280
- 281
- 282
- 283
- 284
- 285
- 286
- 287
- 288
- 289
- 290
- 291
- 292
- 293
- 294
- 295
- 296
- 297
- 298
- 299
- 300
- 301
- 302
- 303
- 304
- 305
- 306
- 307
- 308
- 309
- 310
- 311
- 312
- 313
- 314
- 315
- 316
- 317
- 318
- 319
- 320
- 321
- 322
- 323
- 324
- 325
- 326
- 327
- 328
- 329
- 330
- 331
- 332
- 333
- 334
- 335
- 336
- 337
- 338
- 339
- 340
- 341
- 342
- 343
- 344
- 345
- 346
- 347
- 348
- 349
- 350
- 351
- 352
- 353
- 354
- 355
- 356
- 357
- 358
- 359
- 360
- 361
- 362
- 363
- 364
- 365
- 366
- 367
- 368
- 369
- 370
- 371
- 372
- 373
- 374
- 375
- 376
- 377
- 378
- 379
- 380
- 381
- 382
- 383
- 384
- 385
- 386
- 387
- 388
- 389
- 390
- 391
- 392
- 393
- 394
- 395
- 396
- 397
- 398
- 399
- 400
- 401
- 402
- 403
- 404
- 405
- 406
- 407
- 408
- 409
- 410
- 411
- 412
- 413
- 414
- 415
- 416
- 417
- 418
- 419
- 420
- 421
- 422
- 423
- 424
- 425
- 426
- 427
- 428
- 429
- 430
- 431
- 432
- 433
- 434
- 435
- 436
- 437
- 438
- 439
- 440
- 441
- 442
- 443
- 444
- 445
- 446
- 447
- 448
- 449
- 450
- 451
- 452
- 453
- 454
- 455
- 456
- 457
- 458
- 459
- 460
- 461
- 462
- 463
- 464
- 465
- 466
- 467
- 468
- 469
- 470
- 471
- 472
- 473
- 474
- 475
- 476
- 477
- 478
- 479
- 480
- 481
- 482
- 483
- 484
- 485
- 486
- 487
- 488
- 489
- 490
- 491
- 492
- 493
- 494
- 495
- 496
- 497
- 498
- 499
- 500
- 501
- 502
- 503
- 504
- 505
- 506
- 507
- 508
- 509
- 510
- 511
- 512
- 513
- 514
- 515
- 516
- 517
- 518
- 519
- 520
- 521
- 522
- 523
- 524
- 525
- 526
- 527
- 528
- 529
- 530
- 531
- 532
- 533
- 534
- 535
- 536
- 537
- 538
- 539
- 540
- 541
- 542
- 543
- 544
- 545
- 546
- 547
- 548
- 549
- 550
- 551
- 552
- 553
- 554
- 555
- 556
- 557
- 558
- 559
- 560
- 561
- 562
- 563
- 564
- 565
- 566
- 567
- 568
- 569
- 570
- 571
- 572
- 573
- 574
- 575
- 576
- 577
- 578
- 579
- 580
- 581
- 582
- 583
- 584
- 585
- 586
- 587
- 588
- 589
- 590
- 591
- 592
- 593
- 594
- 595
- 596
- 597
- 598
- 599
- 600
- 601
- 602
- 603
- 604
- 605
- 606
- 607
- 608
- 609
- 1 - 50
- 51 - 100
- 101 - 150
- 151 - 200
- 201 - 250
- 251 - 300
- 301 - 350
- 351 - 400
- 401 - 450
- 451 - 500
- 501 - 550
- 551 - 600
- 601 - 609
Pages:
