Power and Society_ An Introduction to the Social Sciences (Brigid C. Harrison, Ch2)

SOCIAL CHAPTER SCIENCES AND THE SCIENTIFIC 2 METHOD WHAT IS THE PURPOSE OF THE SCIENTIFIC METHOD? WHAT BASIC TERMS DO SOCIAL SCIENTISTS USE IN CONDUCTING RESEARCH? WHAT ARE SOME OF THE DIFFICULTIES THAT SOCIAL SCIENTISTS HAVE IN APPLYING THE SCIENTIFIC METHOD TO THE STUDY OF SOCIAL PROBLEMS? WHAT METHODS DO SOCIAL SCIENTISTS USE TO CONDUCT RESEARCH? WHAT ARE THE PROS AND CONS OF EACH? Science and the Scientific Method A science may be broadly defined as any organized body of knowledge, or it may be science more narrowly defined as a discipline that employs the scientific method. If we use broadly defined, any the broad definition, we can safely say that all the social sciences are indeed sci- organized body of ences. However, if we narrow our definition to only those disciplines that employ knowledge the scientific method, then some questions arise about whether the social sciences are really scientific. In other words, if science is defined as a method of study, rather scientific method than a body of knowledge, then not all studies in the social sciences are truly scientific. a method of explanation that develops and tests The scientific method develops and tests theories about how observable facts theories about how or events are related in order to explain them. What does this definition really observable facts or mean? How is this method of study actually applied in the social sciences? To an- events are related swer these questions, let’s examine each aspect of the scientific method separately. hypothesis Explaining Relationships a tentative statement about a relationship The goal of the scientific method is explanation. When using this method, we seek between observable facts to answer why. Any scientific inquiry must begin by observing and classifying or events things. Just as biology begins with the careful observation, description, and classification of thousands upon thousands of different forms of life, the social sciences also must begin with the careful observation, description, and classifi- cation of various forms of human behavior. But the goal is explanation, not just description. Just as biology seeks to develop theories of evolution and genetics to explain the various forms of life upon the earth, the social sciences seek to develop theories to explain why human beings behave as they do. To answer the question of why, the scientific method searches for relationships. All scientific hypotheses assert some relationship between observable facts or events. The social sciences seek to find relationships that explain human behavior. The first question is whether two or more events or behaviors are related in any way—that is, do they occur together consistently? The second question is whether 19

20 Part 1 / The Nature and Study of Power FOCUS The Vocabulary of Social Science Social science researchers use many special terms in their work, some of which have already been defined. While reading social science research reports, it helps to understand the specific meanings given to the following terms: • THEORY: A causal explanation of relationship between observable facts or events. A good theory fits the facts, explains why they occur, and allows us to predict future events. • HYPOTHESIS: A tentative statement about a relationship between facts or events that should be derived from the theory and should be testable. Hypotheses typically are statements of relationships between variables. • VARIABLE: A characteristic that varies among different individuals or groups. • INDEPENDENT VARIABLE: Whatever is hypothesized to be the cause of something else. • DEPENDENT VARIABLE: Whatever is hypothesized to be the effect of something else. • SIGNIFICANT: Not likely to have occurred by chance. • CORRELATION: Significant relationships found in the data. • CAUSATION: A significant relationship wherein the presence of one variable (the independent variable) causes changes in another variable (dependent variable). • INFERENCE: A causal statement based on data showing a significant relationship. • SPURIOUS: Describing a relationship among facts or events that is not causal but is a product of the fact that both the independent and dependent variables are being caused by a third factor. significant either event or behavior causes the other. Social scientists first try to learn whether not likely to have human events have occurred together merely by chance or accident or whether occurred by chance they occur together so consistently that their relationship cannot be a mere coin- cidence. A relationship that is not likely to have occurred by chance is said to be significant. After observing a significant relationship, social scientists next ask whether there is a causal relationship between the phenomena (that is, whether the facts or events occurred together because one is the cause of the other) or whether both phenomena are being caused by some third factor. “Focus: The Vocabulary of Social Science” explains some of the terms used in scientific studies. Developing and Testing Hypotheses The scientific method seeks to develop statements (hypotheses) about how events or behaviors might be related and then determines the validity of these statements by care- ful, systematic, and logical tests. Scientific tests are really exercises in logic. For example, if we wanted to find out something about the relationship between peo- ple’s race and party preference in voting, we might collect and record data from a national sample of African American and white voters chosen at random.* If our data showed that all blacks voted Democratic and all whites Republican, it would * Throughout this book we use the term African American when referring to specific individ- uals or the racial group, but in text and tables that compare African Americans and whites, we use parallel terms, black and white.

Chapter 2 / Social Sciences and the Scientific Method 21 be obvious that there was a perfect correlation, or a significant statistical rela- correlation tionship, between race and voting. In contrast, if both blacks and whites voted a significant statistical Republican and Democratic in the same proportions, then it would be obvious relationship that there was no correlation. inference But in the social sciences, we rarely have such obvious, clear-cut results. a causal statement based Generally our data will show a mixed pattern. For example, in the 2008 presiden- on data showing a sig- tial election, with Barack Obama seeking to become the first African American nificant relationship president in U.S. history, 95 percent of all African American voters supported him, while 4 percent backed his opponent John McCain. Among white voters, 43 percent voted for Obama, while 57 percent supported McCain. And in the 2004 presidential election between Democrat John Kerry and Republican George W. Bush, polls indicated that 88 percent of African Americans voted Democratic and 11 percent voted Republican. In that same election, 58 percent of whites voted Republican and only 41 percent voted Democratic. If there had been no relation- ship between race and voting, then blacks and whites would have voted Demo- cratic and Republican in roughly the same proportions. But as we have just noted, blacks voted Democratic in far heavier proportions in both elections (95 percent and 88 percent) than whites (43 and 41 percent). This difference is not likely to have occurred by chance—thus, we consider it “significant.” The same pattern of heavy Democratic voting among African Americans can be observed in other elections (Table 2-1). So we can make the inference that race is related to voting. Note, however, that correlation (two events occurring together in a statisti- cally significant relationship) does not equal causation (that one event necessar- ily caused the other to occur). Rather, both events could be caused by a third, as yet unidentified, characteristic or event. We must employ additional logic to find out which fact or event caused the other, or whether both were caused by a third fact or event. We can eliminate as illogical the possibility that voting Democratic causes one to become an African American. That leaves us with two possibilities: Being African American may cause Democratic voting, or voting Democratic may be caused by some third condition shared by many African Americans. For ex- ample, the real causal relationship may be between lower household incomes and Democratic voting: People with lower household incomes tend to identify with the Democratic Party. And while of course not all African Americans have lower household incomes, in general, African Americans constitute a larger proportion of lower-income households than their proportion of the population as a whole. We can test this new hypothesis by looking at the voting behavior of both black and white low-income groups. It turns out that low-income black voters vote more heavily Democratic than low-income white voters, so we can reject the lower household income explanation. We may therefore infer that race is independently related to voting behavior. But there may be other possible alter- natives to our explanation of the relationship between race and voting behav- ior. For example, African American parents may socialize their children to be loyal Democrats. Social scientists must test as many alternative explanations as possible before asserting a causal relationship. Every time that we can reject an alternative explanation for the relationship we have observed, we increase our confidence that the relationship (as between race and voting behavior) is a causal one. Of course, in the areas of interest to social scientists, someone can always think of new alternative explanations, so

22 Part 1 / The Nature and Study of Power TA B L E 2-1 VOTING BY RACE IN PRESIDENTIAL ELECTIONS Testing the Hypothesis: African Americans Tend to Vote Democratic Election Year Candidates All (%)[TFTN]* Whites (%) Blacks (%) 2008 Republican McCain 47 57 4 2004 Democrat Obama 53 43 95 2000 1996 Republican Bush 51 58 11 Democrat Kerry 48 41 88 1992 Republican Bush 48 54 8 1988 Democrat Gore 48 42 90 1984 1980 Republican Dole 41 46 12 Democrat Clinton 49 43 84 1976 Independent Perot 8 94 1972 Republican Bush 38 41 11 Democrat Clinton 43 39 82 Independent Perot 19 20 7 Republic Bush 54 60 11 Democrat Dukakis 46 40 89 Republican Reagan 59 66 9 Democrat Mondale 41 34 90 Republican Reagan 51 56 10 Democrat Carter 41 36 86 Independent Anderson 7 72 Republican Ford 48 5 15 Democrat Carter 50 46 85 Republican Nixon 62 68 13 Democrat McGovern 38 32 87 * Figures are percentages of the vote won by each candidate. Percentages in each election may not add up to 100 because of voting for minor-party candidates. SOURCE: Data from the Gallup Opinion Poll surveys and CNN Exit Poll results (2008). empirical it is generally impossible to establish for certain that a causal relationship ex- referring to observable ists. Some social scientists react to the difficulties of proving “cause” by refus- facts and events; what is ing to say that the relationships they find are anything more than correlations. The decision whether or not to call a relationship “causal” is difficult. Statisti- normative cal techniques cannot guarantee that a relationship is causal; social scientists referring to values or must be prepared to deal with probabilities rather than absolutes. norms; what should be Dealing with Observable Phenomena The scientific method deals only with observable—empirical—facts and events. In other words, the scientific method deals with what is, rather than what should be. It cannot test the validity of values, norms, or feelings, except insofar as it can test for their existence in a society, group, or individual. For example, the scientific method can be employed to determine whether voting behavior is related to race, but it cannot determine whether voting behavior should be related to race. The latter question is a normative one (dealing with “ought” and “should”), rather than an empirical one (dealing with “is”). The scientific method is descriptive and explanatory, but not normative. The social sciences can explain many aspects of human behavior but cannot tell human beings how

Chapter 2 / Social Sciences and the Scientific Method 23 they ought to behave. For guidance in values and norms—for prescriptions about how people should live—we must turn to ethics, religion, or philosophy. Developing Theory The scientific method strives to develop a systematic body of theory. Science is more theory than crude empiricism—the listing of facts without any statement of relation- an explanation of facts ships among them. Of course, especially in the early stages of a science, research or events may consist largely of collecting data, but the ultimate goal of the scientific method is to develop verifiable statements about relationships among facts and events. It is the task of social scientists to find patterns and regularities in hu- man behavior, just as it is the task of physicists and chemists to find patterns and regularities in the behavior of matter and energy. The social scientist’s use of the scientific method, then, assumes that human behavior is not random, rather that it is regular and predictable. Social scientists’ development of theory, however, must be more normative than in the natural sciences. For example, underlying much theory in the social sciences are assumptions about human nature—whether it is inherently good or inherently selfish. These assumptions oftentimes form the building blocks of theory. Theories are developed at different levels of generality. Theories with low lev- els of generality explain only a small or narrow range of behaviors. For example, we might theorize that conservative Christian voters tend to vote Republican, and this theory has a fairly low-level generality about political behavior, that is, it is a narrow and quite specific theory about one group’s voting behavior. Theo- ries with higher levels of generality explain a greater or wider range of behavior. For example, the statement that religious differences cause political conflict has a higher level of generality. Strictly speaking, a theory is a set of interrelated con- cepts at a fairly high level of generality. Some social scientists concentrate on theory building rather than on empirical research; they try to develop sweeping social theories to explain all, or a large part, of human behavior. Still other social theo- rists provide insights, hunches, or vague notions that suggest possible explana- tions of human behavior, thus developing new hypotheses for empirical research. Maintaining a Scientific Attitude Perhaps more than anything else, the scientific method is an attitude of doubt or scientific attitude skepticism. It is an insistence on careful collection of data and systematic testing doubt or skepticism of ideas; a commitment to keep bias out of one’s work, to collect and record all about theories until they relevant facts, and to interpret them rationally regardless of one’s feelings. Ad- have been scientifically mittedly, it is difficult to maintain a truly scientific attitude when examining tested social behavior. (See Controversies in Social Science: “Can Social Science Be Scientific?”) For the social scientist, it is the determination to test explanations of human behavior by careful observations of real-world experiences. It is a recognition that any explanation is tentative and may be modified or disproved by careful investigation. Even the scientific theories that constitute the core knowledge in any discipline are not regarded as absolutes by the true social scientist; rather, they are regarded as probabilities or generalizations developed from what is known so far.

24 Part 1 / The Nature and Study of Power CONTROVERSIES IN SOCIAL SCIENCE Can Social Science Be Scientific? The scientific method was devised in the physical and biological sciences. There are many difficulties in applying this method to the study of individuals, groups, economies, classes, governments, nations, or whole societies. Let’s examine some of the obstacles to the devel- opment of truly scientific social sciences. Personal Bias Social science deals with subjective topics and must rely on interpretation of results. Social scientists are part of what they investigate—they belong to a family, class, race, gender, political party, interest group, profession, and nation. If the topic is an emotional one, the social scientist may find it much harder to suppress personal bias than does the investigator in the physical sciences: It is easier to conduct an unbiased study of migratory birds than of migrant workers. Researchers study what they think is important in society, and their personal values affect what they think is important. Moreover, researchers’ values are also frequently reflected in their per- ceptions of the data, in their statement of the hypotheses, in their design of the test for the hy- potheses, and in their interpretations of the findings. An extreme version of this criticism of the social sciences (sometimes referred to as poststructuralism) argues that no knowledge is free of the race, gender, and class bias of the researcher. Public Attitudes Another problem in the scientific study of human behavior centers on public attitudes toward social science. Few people would consider arguing with atomic physicists or biochemists about their respective fields, but most people believe they know something about social problems. Many people think they know exactly what should be done about juvenile delinquency, welfare dependency, and race relations. Very often their information is limited, and their view of the problem is simplistic. When a social scientist suggests that a problem is very complex, that it has many causes, and that information on the problem is incomplete, people may believe that the social scientist is simply obscuring matters that seem obvious. universal statement What Is a “Fact”? a statement that applies to every circumstance In the social sciences, very few statements can be made that apply to every circum- stance. We cannot say, for example, that “all evangelical/born-again Christians probabilistic vote Republican.” This is a universal statement covering every evangelical or statement born-again Christian, and universal statements are seldom true in the social sci- a statement that applies ences. Moreover, it would be difficult to examine the voting behavior of every to some proportion of evangelical or born-again Christian voter in the past and in the future to prove circumstances that the statement is true. A more accurate statement might be “most white evangelical/born-again Christians vote Republican.” This is a probabilistic statement; it does not exclude the possibility that some white evangelical/born-again Christians vote Democratic. An even more accurate statement would be that “74 percent of

Chapter 2 / Social Sciences and the Scientific Method 25 Social science sometimes develops explanations of human behavior that contradict established ideas. Of course, the physical and biological sciences have long faced this same problem: Galileo faced the opposition of the established church when he argued that the earth revolved around the sun, and Darwin’s theory of evolution continues to be a public issue. But social science generates even more intense feelings when it deals with poverty, crime, sexual behavior, race relations, and other sensitive topics. Limitations and Design of Social Science Research Another set of problems in social science centers on the limitations and design of social science research. It is not really possible to conduct some forms of controlled experiments on human beings. For example, we cannot deliberately subject people to poverty and deprivation just to see if it makes them violent. Instead, social researchers must find situations of poverty and deprivation in order to make the necessary observations about causes of violence. In a laboratory, we can control all or most of the factors that go into the experimental situation. But in real-world observations, we cannot control many factors; this makes it difficult to pinpoint what it is that causes the behavior that we are studying. Moreover, even where some experimentation is permitted, human beings frequently modify their behavior simply because they know they are being observed in a social science experiment. This phenomenon, known as the Hawthorne effect, makes it difficult to determine whether the observed behavior is a product of the stimulus being introduced or merely a product of the experimental situation itself. The Hawthorne effect is so named because it was identified for the first time by a group of research- ers led by Elton Mayo. The researchers were investigating the impact of various social and environ- mental conditions at the Western Electric Hawthorne plant. The researchers found that whatever they did (improve lighting, dim lighting) resulted in increases in productivity. It was later determined that the increases actually were due to the increased attention that the workers were getting from the researchers, rather than the changes to the social and environmental conditions in the workplace. white evangelical/born-again Christians cast their ballots for Republican can- didate John McCain in the 2008 presidential election.” This means there was a 74 percent probability of a white evangelical/born-again Christian voter casting his or her ballot for Republican John McCain. A probabilistic statement is a fact, just like a universal statement. Students in the physical sciences deal with many universal statements—for example, “Water boils at 100 degrees Celsius.” Water always does this. But social science students must learn to think in probabilities rather than in absolute terms. Social scientists must also beware of substituting individual cases for state- ments of probability. They must be careful about reasoning from one or two observed cases. A statement such as “I know an evangelical family who always votes Democratic” may be true, but it would be very dangerous to generalize about the voting habits of all evangelicals on the basis of this one case. We always

26 Part 1 / The Nature and Study of Power build tentative generalizations from our own world of experiences. However, as social scientists, we must ensure that our own experiences are typical. We should keep in mind that the “facts” of the social sciences are seldom absolute—they rarely cover the complexity of any aspect of human behavior. So we must be pre- pared to study probabilities. The Classic Scientific Research Design experiment An experiment is a scientific test that is controlled by the researcher and designed a scientific test controlled to observe the effect of a program or treatment. The classic scientific research design by the researcher to involves the comparison of specific changes in two or more carefully selected observe effects of a groups, both of which are identical in every way, except that one has been given specific program or the program or treatment under study while the other has not.1 treatment This design involves the following: experimental group • Identification of the goals of the study and the selection of specific hypotheses the group that will to be tested. participate in the program or undergo the • Selection of the groups to be compared—the experimental group, which treatment under study will participate in the program or undergo the treatment being studied, and control group the control group, which is similar to the experimental group in every way, a group, similar to the except that it will not participate in the program or undergo the treatment experimental group, being studied. that does not undergo treatment; used for • Measurement of the characteristics of both the experimental and control comparison groups before participation in the experiment. null hypothesis • Application of the program or treatment to the experimental group, but a statement that the not to the control group. (Members of the control group may be given a program or treatment placebo—some activity or program known to have no effect—to make them has no effect believe that they are participating in the experiment. Indeed, the scientific staff administering the experiment may not know which group is the real experimental group and which group is the control group. When neither the staff nor the group members themselves know who is really receiving the treatment, the experiment is called a double-blind experiment.) • Measurement of the condition of both the experimental and control groups after the program or treatment. If there are measurable differences between the experimental and control groups, the scientist can begin to infer that the program or treatment has a specific effect. If there are no measurable differences, then the scientist must accept the null hypothesis—the state- ment that the program or treatment has no effect. • Comparison of the preprogram/pretreatment status versus the post- program/post-treatment status in both groups. This is a check to see if the difference between the experimental and control groups occurred during the experiment (see Case Study: “An Experiment in Preventing Underage Drinking”). If there is no control group and only the experimental group is studied, this method is often called a “before–after” study. • A search for plausible explanations for differences after treatment between the control and experimental groups that might be due to factors other than the treatment itself.

Chapter 2 / Social Sciences and the Scientific Method 27 CASE STUDY An Experiment in Preventing Underage Drinking Let’s consider an example of applying the classic scientific research design to a specific social problem—underage drinking. A local government is considering creating recre- ation programs for young adults aged 18–21, which include coffee house–style gatherings, informal basketball and volleyball games, and battles of the bands, all of which would take place at night in local neighborhood elementary schools. The hypothesis is that the recre- ation programs will reduce underage-drinking rates. Before spending large sums of money to implement the recreation programs throughout the entire city without knowing whether the plan will work, the city council decides to put the program to a scientific test. The council selects several neighborhoods that have identical characteristics (underage-drinking rate, land use, population density, unemployment, population age, income, racial balance, and so forth). Some of the areas are randomly selected for the implementation of the recreation program. Under- age-drinking rates, as measured by arrests, are then carefully tracked both before the implementation of the recreation programs in those neighborhoods that will house the program and in those neigh- borhoods that were not randomly selected for the program (see the accompanying graph). After several months of administering the program, underage-drinking rates are again carefully measured in the experimental neighborhoods (which received the program) and the control neigh- borhoods (which did not). The results are compared. If a significant reduction in underage drinking occurred in the neighborhoods with the program but did not occur in the neighborhoods without the program and no other changes can be identified in the neighborhoods that might account for the differences, then the city can have some confidence that recreational programs reduce underage drinking. An expansion of the program to the rest of the city would then seem appropriate. A Scientific Research Design Preprogram observations Postprogram observations for Preventing Underage 30 Drinking 25 Underage drinking 20 Estimated 15 effect of program 10 5 0 J F MA J J ASO Months before Rec program Months after implemented Experimental neighborhoods Control neighborhoods (rec program implemented) (no rec program implemented)

28 Part 1 / The Nature and Study of Power The classic research design is not without its problems. Social scientists must be aware of the more difficult problems in applying this research design to social science research and must be prepared on occasion to change their procedures accordingly. These problems include the following: • Members of the experimental group may respond differently to a program if they know it is an experiment. Members of a control group are often told they are participating in an experiment, even though nothing is really being done to the control group. • If the experimental group is only one part of a larger city, state, or nation, the response to the experiment may be different from what it would have been had all parts of the city, state, or nation been receiving the program. For example, if only one part of a city receives streetlights, criminals may simply operate as usual (even with the lights), and total crime rates will be unaffected. • If persons are allowed to volunteer for the experiment, then experimental and control groups may not be representative of the population as a whole. • In some situations, political pressures may make it possible to provide one neighborhood or group with certain services while denying these same ser- vices to the rest of the city, state, or nation. If everyone thinks the program is beneficial before the experiment begins, no one will want to be in the control group. • It may be considered morally wrong to provide some groups or persons with services, benefits, or treatment while denying the same to other groups or persons (control groups) who are identical in their needs or problems. • Careful research is costly and time consuming. Public officials often need to make immediate decisions. They cannot spend time or money on research even if they understand the long-term benefits of careful investigation. Too often, politicians and other policy makers must operate on “short-run” rather than “long-run” considerations. Gathering Data: Survey Research sample How do social scientists go about observing the behaviors of individuals, groups, in survey research, the and societies? There are a variety of methods for gathering data (Table 2-2); people chosen to repre- some fields rely more heavily on one particular method than on another. The sent the opinions controlled experiment, described earlier, is often used in psychology; the survey is of a larger group frequently employed in political science and sociology; field research, or partici- pant observation, is a major source of data in anthropology; and secondary data universe analysis is employed in all social sciences.2 the whole group about which information is Survey Research desired Most surveys ask questions of a representative sample of the population rather than question the entire population. A selected number of people, the sample, is chosen in a way that ensures that this group is representative of the universe,

Chapter 2 / Social Sciences and the Scientific Method 29 TA B L E 2-2 GATHERING DATA IN THE SOCIAL SCIENCES Research Method Operationalization Example Classical scientific See Case Study: “An Experiment in experiment Uses the scientific method; includes Preventing Underage Drinking.” both an experimental and a control Survey research group; measures the impact of the ap- Gallup polls. plication of the program or treatment Field research: on both groups. Political scientist works as a cam- participant paign consultant while researching observation Uses the scientific method; asks opin- the decision-making structure in ions of a randomly selected sample of campaigns. Field research: the population. Sociologist researching children’s unobtrusive sex roles at play observes children observer Researchers both observe and partici- while going unnoticed. pate in the behavior being studied. Researchers observe the behavior being studied but try not to intrude or partake in the behavior. Field research: A systematic description of a society’s Anthropologist lives with, inter- ethnography customary behaviors, beliefs, and views, and observes people of attitudes. another culture. Secondary-source data Data used by social scientists that The Statistical Abstract of the have been collected by other organiza- United States (see p. 36). tions or governments or researchers. the whole group of people about which information is desired. To ensure that random sample the sample is representative of the universe, most surveys rely on random selec- a subset of the popula- tion. Random sampling means that each person in the universe has an equal tion in which there is chance of being selected for interviewing. Random sampling improves the an equal chance of each likelihood that the responses obtained from the sample would be the same as person in the universe those obtained from the universe if everyone were questioned. Hypothetically, being selected in the we must obtain a random sample of American voters by throwing every voter’s sample for interviewing name in a giant box and blindly picking out one thousand names to be inter- viewed. A more common method is to randomly select telephone area codes and sampling error then numbers from across the nation. the range of responses in which a 95 percent There is always the chance that the sample selected will not be representa- chance exists that the tive of the universe. But survey researchers can estimate this sampling error sample reflects the through the mathematics of probability. The sampling error is usually expressed universe as a range above and below the sample response, within which there is a 95 per- cent likelihood that the universe response would be found if the entire universe was questioned. For example, if 63 percent of the people questioned (the sample) say they approve of the way that the president is handling his job and the sam- pling error is calculated at Ϯ3 percent (said “plus or minus 3 percent”), then we can say that there is a 95 percent likelihood that the president’s approval rating among the whole population (the universe) stands somewhere between 60 and 66 percent (63 – 3 and 63 + 3).

30 Part 1 / The Nature and Study of Power Large samples are not really necessary to narrow the sampling error. Large samples are not much more accurate than small samples. A sample of a few thousand—even one thousand—is capable of reflecting the opinions of 1 million or 100 million voters fairly accurately. For example, a random sample of one thou- sand voters across the United States can produce a sampling error (plus or minus) of only 3 percent. Problems in Survey Research public opinion If a poll is constructed scientifically and thoughtfully, it can provide accurate the aggregate of opinions information about the opinions about a population. When well-constructed of individuals on topics polls are inaccurate, it is usually because public opinion is unformed, weakly in survey research held, or changing rapidly. If public opinion is really unformed on a topic, as may be the case in early presidential preference polls, people may choose a salient issue familiar name or a celebrity who is frequently mentioned in the news. As the an issue that most people campaign progresses and people learn about the candidates and form opinions have thoughts about and on them, candidates who were once unknown and rated only a few percent- hold strong and stable age points in early polls can emerge as front-runners. Weakly held opinions opinions about are more likely to change than strongly held opinions. Political commentators sometimes say a particular candidate’s support is “soft,” meaning that his or her supporters are not very intense in their commitment; therefore, the polls could swing quickly away from the candidate. Finally, widely reported news events may change public opinion very rapidly. A survey can measure opinions only at the time it is taken. A few days later, public opinion may change, especially if major events are receiving heavy television coverage. Some political pollsters conduct continuous surveys until election night in order to catch last-minute opinion changes. But today pollsters and public opinion researchers who rely on telephone polling face a large obstacle in the data collection process: Many people rely exclusively on cell phones and do not have landline telephones. No national polling organization regularly polls cell phone subscribers, meaning that those “celly-onlys” tend to be severely underrepresented in most national polls. Most data suggest that these “celly-onlys” are disproportionately young and oftentimes have less income than those with landlines, or those with both landlines and cell phones. Pollsters are attempting to remedy this prob- lem in several ways—some are conducting polls of celly-onlys (concerning the 2008 presidential election, for example). Others are attempting to rectify the underrepresentation of these individuals by including a larger number of landline-surveyed younger people and those with lower incomes polled in their samples. Assessing Public Opinion Public opinion in democracies is given a great deal of attention. Indeed, survey research on public opinion is a thriving industry. There are some “hot- button” issues about which virtually everyone has an opinion and many people feel very intensely. And survey results on these issues command the attention of politicians and the news media as well as social scientists. Salient issues are

Chapter 2 / Social Sciences and the Scientific Method 31 those that people think about most and about which they hold strong and stable push poll opinions (see Focus: “The Death Penalty: A Hot-Button Issue”). a survey that asks lead- ing questions in order In assessing public opinion polls, the construction of a poll merits scrutiny. to sway opinion for a It is important to realize that the wording or phrasing of public opinion ques- particular candidate or tions can often determine the outcome of a poll. Indeed, “loaded” or “leading” position questions are often asked by unprofessional pollsters simply to produce results favorable to their political candidate or to their side of an argument. Called halo effect push polls, these efforts really constituent more of a campaign tactic than a the tendency of scientific public opinion survey. A push poll might sound something like this: respondents to give “good-citizen” responses PUSH POLLER: In Tuesday’s upcoming election between Rose Fitzgerald and Douglas to pollsters Sparks, would you say that you are more likely to vote for Rose Fitzgerald or Doug- las Sparks for mayor? VOTER: Douglas Sparks. PUSH POLLER: If you were informed that Douglas Sparks had multiple arrests for dis- orderly conduct and lewd behavior, would you be more or less likely to vote for him? The purpose of a push poll, as you can see, is not to gauge public opinion about a candidate but to unscrupulously smear the name of a candidate, oftentimes with baseless charges. Note that the push poller is not claiming any of the statements are facts but is saying, “If they were . . . .” Nonetheless, the voter is left with the impression that the charges have merit. In scientific public opinion polls, ideally, questions should be clear and precise, easily understood by the respondents, and as neutral and unbiased as possible. But because all questions have a potential bias, it is usually better to examine changes over time in responses to identically worded questions. Even the most scientific surveys are not error free, however. We have al- ready noted that weakly held opinions can change rapidly. Thus, by the time poll results are reported in the media, those results may no longer reflect public opinion. Also, many opinion surveys ask questions that people had not consid- ered before being interviewed. Few people are willing to admit that they know nothing about the topic or that they really have “no opinion.” They believe they should provide an answer even if they have little interest in the topic itself. The result is that polls often seem to “create” public opinion. Another prob- lem is the halo effect—the tendency of respondents to give “good-citizen” responses, whether the responses are truthful or not. For example, people do not like to admit that they do not vote or that they do not care about politics. Surveys regularly report higher percentages of people saying they voted in an election than the actual number of ballots cast would indicate. Many people give socially respectable answers, even to an anonymous interviewer, rather than answers that suggest prejudice, hatred, or ignorance. Field Research Fieldwork is the cornerstone of modern anthropology. Many sociologists and political scientists also obtain their information through field work. These social scientists study by direct, personal observation of people, events, and societies.

32 Part 1 / The Nature and Study of Power FOCUS The Death Penalty: A Hot-Button Issue The death penalty is a salient issue—nearly everyone has an opinion on this topic, and many people feel very intensely about it. Survey research indicates that opinion on the death penalty has been stable over the years. Con- sider the following poll conducted by the Gallup organization between 1936 and 2008. 90 79 80 80 70 68 75 68 70 60 59 66 50 40 38 54 57 66 65 30 31 20 47 2009 10 42 28 28 38 32 26 25 26 17 16 16 0 1937 1942 1947 1952 1957 1962 1967 1972 1977 1982 1987 1992 1997 2002 % For % Against SOURCE: Information from www.gallup.com. For nearly the past 40 years, majorities of those surveyed have reported that they are in favor of the death penalty, ranging from a low in 1972 of 57 percent to a high in 1994 of 80 percent. Yet in response to the question, “If you could choose between the following two approaches, which do you think is the better penalty for murder: the death penalty or life imprisonment, with absolutely no possibility of parole?”, smaller majorities chose the death penalty. From 1985 until 2006, at most 61 percent would select the death penalty over life in prison. The death penalty Life imprisonment % % 2006 47 48 2001 54 42 2000 49 47 1999 56 38 1997 61 29 1993 59 29 1992 50 37 1991 53 35 1986 55 35 1985 56 34 SOURCE: Information from www.gallup.com.

Chapter 2 / Social Sciences and the Scientific Method 33 But survey research cannot resolve a public policy issue. Indeed, interpretation of poll results on a hot-button issue often becomes a topic of conflict itself. Public opinion on aspects of the death penalty in the United States—particularly whether young offenders, or people with limited mental capacity because of disease or mental retardation, should be eligible to be executed—has played a role in shaping public policy. Indeed, by and large, the public opposes executing those with mental retardation or those who are mentally ill. The public also by and large opposes the execution of juveniles. For example, when asked whether the death penalty should be given to various populations, there are great variations among responses. Do you favor or oppose the death penalty for: The mentally ill 2002 May 6–9 Favor Oppose 19% 75% The mentally retarded Favor Oppose 13% 82% 2002 May 6–9 Juveniles 2002 May 6–9 Favor Oppose 26% 69% In some situations, public opinion has been an important factor in shaping public policy. For example, while the U.S. Supreme Court held in a 1989 decision (Penry v. Lynaugh [492 U.S. 584 (1989)]), that executing persons with mental retardation was not a violation of the Eighth Amendment, in 2002 in Atkins v. Virginia (536 U.S. 304), the Court ruled that a “national consensus” had evolved against the execution of the mentally retarded. The Court then ruled that such executions do violate the Eighth Amendment’s prohibition on cruel and unusual punishment.

34 Part 1 / The Nature and Study of Power Participant- © Jack Fields/CORBIS observation research involves the integra- tion of researcher into the field. The researcher observes and actually partici- pates in the behavior being studied. Here, a French anthropolo- gist studies Aru Island villagers, in Indonesia. What are the advan- tages of this type of research? field research Field research is essentially going where the action is, watching closely, and directly observing social taking notes.3 behavior Fieldwork is usually less structured than either experimental or survey participant research. Oftentimes, field research enables a more in-depth examination of the observation causes of behaviors than quantitative research allows. However, in field research, researchers both observe the scientist cannot control many variables, as in experimental research. Nor can and participate in the the scientist know whether the peoples or societies being studied are truly repre- behavior being studied sentative of all other peoples or societies, as in survey research. However, careful field reports can provide qualitative information that is often missing from exper- imental and survey research. Researchers can report on emotions, feelings, and beliefs that underlie people’s behavioral responses. Researchers can also report on attitudes, myths, symbols, and interpersonal relationships that could not be de- tected by other research methods. Most important, they can observe individuals, groups, and societies as they live in their subjects’ environment. Field research often involves participant observation, where the researcher both observes and participates in the society being studied. Direct participation (moving to Appalachia and getting a job as a coal miner, for example) can provide insights that would otherwise escape a researcher. However, personal participa- tion can also interfere with the detachment required for scientific inquiry. There is also the question of whether the scientist should identify himself or herself as a researcher, which could change the behavior of the people being studied, or con- ceal his or her identity, which could encourage people to act naturally but raises ethical questions. Some behavior simply cannot be observed if social scientists are identified as researchers. Consider the dilemma of the sociologist who wanted to study homosexual behavior in public toilets. It was not really feasible for him to go on field trips to public toilets, identifying himself as a sociologist and asking people if they

Chapter 2 / Social Sciences and the Scientific Method 35 were homosexuals seeking contacts, and, if so, if he could observe their be- ethnography havior. But he understood that, in some areas, public toilets had provided an systematic description important meeting ground for gay men who lacked access to other means of of a society’s customary meeting gay men. So instead he began visiting public toilets where he sus- behaviors, beliefs, and pected homosexual activity was taking place and volunteered to act as a “look- attitudes out” for those engaging in the action. He discovered that a lookout was an acceptable, even important, position, and he took advantage of it to study ho- case study mosexual behavior. Later, after the publication of his study, he came under an in-depth investigation attack by homosexuals and others for deceiving his subjects.4 of a particular event in order to understand it as In other field research, a researcher will attempt to be as unobtrusive as fully as possible possible in order not to influence or change the behavior of the subjects. For example, a sociologist researching the sex roles of children at play might push her own child on a swing while listening and observing another group of chil- dren playing “house” at a park.* Anthropology relies heavily on field research. To describe cultures accu- rately, many anthropologists choose to live among the people they are studying, directly observing and participating in their lives. Many early anthropologi- cal studies were intuitive: They produced in-depth, first-hand observations of societies, but these observations were not very systematic. Some would focus on child rearing, religion, art, language, or a particular practice specific to the area being studied and foreign to researchers. Later, anthropological field work became more disciplined, and anthropologists began systematic comparisons of cultures. Ethnography is the systematic description of a society’s customary behav- iors, beliefs, and attitudes. Ethnographic studies are usually produced by anthro- pologists who have spent some time living with, interviewing, and observing the people. Anthropologists in the field can test hypotheses by directly asking and observing the people and learning about the context of their behavior and be- liefs. For example, an anthropologist in the field may think that the society he or she is studying practices polygamy (one man marries more than one woman simultaneously) because it has more women than men. But as ethnographic stud- ies are gradually acquired for a larger number of different cultures, anthropol- ogists can begin to test hypotheses by cross-cultural comparisons. They may find reports of some societies that practice polygamy even though the number of men and women is equal. This finding would cast doubt on the hypothesis that polygamy is caused by gender-ratio imbalances. A case study is an in-depth investigation of a particular event in order to understand it as fully as possible. A case study may involve an examination of a single government decision, a single business firm, a single town, or a single society. In anthropology, a case study often will examine a specific topic in a given culture—mating rituals among the Ibakharu tribe in Algeria, for example. In political science, one can find case studies of the passage of pieces of legislation. For example, political scientist Janet M. Martin wrote about the legislative process concerning an education program in her book, Lessons from the Hill: The Legislative Journey of an Education Program. A hypothesis * Researchers typically must be granted consent by the human subjects of their research (or their parents or guardians).

36 Part 1 / The Nature and Study of Power TA B L E 2 - 3 USING THE STATISTICAL ABSTRACT The Statistical Abstract of the United States is published annually by the U.S. Census Bureau. Statistics in each edition are for the most recent year or period available by October of the preceding year. Each new edition contains nearly nine hundred tables. The original source of the data is provided in footnotes to each table. For example, Table 2-3 is a reproduction of Table 295 in the Statistical Abstract of the United States 2009, which summarizes U.S. crime rates from 1980 to 2006. The headings along the top of the table are column headings, and the headings at the left are row headings. In this table, the column headings indicate the type of crime, and the row headings indicate the number of crimes and the rate (number of crimes per 100,000 inhabitants) for the years 1980 through 2006. So, for example, we can see there were 23,000 murders in the United States in 1991 (note that the number of offenses is given in thousands), and we can see that there were 92,000 forcible rapes in 2006. We can also observe that the murder rate per 100,000 inhabitants fell from 10 in 1993 to 6 in 2006. In Chapter 12, “Power, Crime, and Violence,” we discuss social science explanations for changes in crime rates. CRIMES AND CRIME RATES BY TYPE OF OFFENSE: 1980 TO 2006 Violent Crime Property Crime Aggra- Motor Forcible vated Larceny/ Vehicle Year Total Murder* Rape Robbery Assault Total Burglary Theft Theft Number of offenses (1,000): 1,345 23 83 566 673 12,064 3,795 7,137 1,132 1980 . . . . . . . . . . . . . . . 1,328 19 88 498 1985 . . . . . . . . . . . . . . . 1,820 103 723 11,103 3,073 6,926 1,103 1990 . . . . . . . . . . . . . . . 1,932 23 109 1992 . . . . . . . . . . . . . . . 1,926 24 106 639 1,055 12,655 3,074 7,946 1,636 1993 . . . . . . . . . . . . . . . 1,858 25 102 1994 . . . . . . . . . . . . . . . 1,799 23 672 1,127 12,506 2,980 7,915 1,611 1995 . . . . . . . . . . . . . . . 1,689 22 97 1996 . . . . . . . . . . . . . . . 1,636 20 96 660 1,136 12,219 2,835 7,821 1,563 1997 . . . . . . . . . . . . . . . 1,534 18 96 1998 . . . . . . . . . . . . . . . 1,426 17 93 619 1,113 12,132 2,713 7,880 1,539 1999 . . . . . . . . . . . . . . . 1,425 16 89 2000 . . . . . . . . . . . . . . . 1,439 16 90 581 1,099 12,064 2,594 7,998 1,472 2001 **. . . . . . . . . . . . . . 1,424 16 91 2002 . . . . . . . . . . . . . . . 1,384 16 95 536 1,037 11,805 2,506 7,905 1,394 2003 . . . . . . . . . . . . . . . 1,360 17 94 2004 . . . . . . . . . . . . . . . 1,391 16 95 499 1,023 11,558 2,461 7,744 1,354 2005 . . . . . . . . . . . . . . . 1,418 17 94 2006 . . . . . . . . . . . . . . . 17 92 447 977 10,952 2,333 7,376 1,243 Rate per 100,000 population: 409 912 10,208 2,101 6,956 1,152 1980 . . . . . . . . . . . . . . . 1985 . . . . . . . . . . . . . . . 408 912 10,183 2,051 6,972 1,160 1990 . . . . . . . . . . . . . . . 1992 . . . . . . . . . . . . . . . 424 909 10,437 2,117 7,092 1,228 1993 . . . . . . . . . . . . . . . 1994 . . . . . . . . . . . . . . . 421 891 10,455 2,151 7,057 1,247 1995 . . . . . . . . . . . . . . . 1996 . . . . . . . . . . . . . . . 414 859 10,443 2,155 7,027 1,261 1997 . . . . . . . . . . . . . . . 1998 . . . . . . . . . . . . . . . 401 847 10,319 2,144 6,937 1,238 1999 . . . . . . . . . . . . . . . 2000 . . . . . . . . . . . . . . . 417 862 10,175 2,155 6,783 1,236 2001 **. . . . . . . . . . . . . . 2002 . . . . . . . . . . . . . . . 447 861 9,984 2,184 6,607 1,193 2003 . . . . . . . . . . . . . . . 2004 . . . . . . . . . . . . . . . 597 10 37 251 299 5,353 1,684 3,167 502 2005 . . . . . . . . . . . . . . . 558 8 37 2006 . . . . . . . . . . . . . . . 730 9 41 209 304 4,666 1,292 2,911 464 758 9 43 747 10 41 256 423 5,073 1,232 3,185 656 714 9 39 685 8 37 264 442 4,904 1,168 3,104 632 637 7 36 611 7 36 256 441 4,740 1,100 3,034 606 568 6 35 523 6 33 238 428 4,660 1,042 3,027 591 507 6 32 505 6 32 221 418 4,591 987 3,043 560 494 6 33 476 6 32 202 391 4,451 945 2,980 526 463 6 32 469 6 32 186 382 4,316 919 2,892 506 474 6 31 166 361 4,053 863 2,730 460 150 334 3,744 770 2,551 423 145 324 3,618 729 2,477 412 149 319 3,658 742 2,486 431 146 310 3,631 747 2,451 433 143 295 3,591 741 2,417 434 137 289 3,514 730 2,362 422 141 291 3,432 727 2,288 417 149 288 3,335 729 2,207 398 * Includes nonnegligent manslaughter. ** The murder and nonnegligent homicides that occurred as a result of the events of September 11, 2001, were not included in this table. SOURCE: U.S. Department of Justice, Federal Bureau of Investigation. Crime in the United States, annual. See also <http://www.fbi. gov/ucr/cius2006/index.html>.

Chapter 2 / Social Sciences and the Scientific Method 37 may be tested in a case study, but researchers know that a single case is not sufficient to make generalizations about other cases. A single case study is more useful for generating hypotheses to be explored later in comparative studies involving larger numbers of cases. However, some case studies in- volve limited comparisons, as when two, three, or four cases are studied simultaneously. Secondary-Source Data secondary-source data Social scientists do not always collect their own data—that is, primary-source data used by social data. Often, social scientists rely on data collected by government agencies, scientists that have been other organizations, or other researchers; these data are known as secondary- collected by other orga- source data. One of the most important sources of data for social scientists nizations, governments, is the U.S. Census Bureau, which not only provides the decennial census data or researchers on the population of the United States but also regularly collects and pub- lishes data on governments, housing, manufacturing, and so on. Each year, the Census Bureau also publishes the Statistical Abstract of the United States, which AP Photo/Bill Ross One way that social scientists assess public opinion is through the use of polls, including election day exit polls. These surveys, which rely on random sampling, predict which candidate will win an election before the polls close. One such poll was taken by in Denver, Colorado, where this surveyer asked voters for whom they voter in the 2008 presidential election. Exit polls also shed light on why individuals prefer one candidate over another. In the 2008 election, many of those who voted for Barack Obama said they believed in his message of change, particularly with regard to the nation’s economic situation.

38 Part 1 / The Nature and Study of Power summarizes facts about birth rates and death rates, education, income, health, welfare, housing, election outcomes, government taxing and spending, crime, national defense, employment, prices, business, transportation, agriculture, trade, and manufacturing. Footnotes to the data summarized in the Statistical Abstract tell where additional data can be found on each topic (Table 2-3). ABOUT THIS CHAPTER These questions were the focus of this chapter. Now that you have read it, you should be able to What is a science, and what is the scientific method? How can the subject matter of the social sciences • Define science and describe the scientific method. be studied scientifically? What are the obstacles • Illustrate how social scientists develop and test to the scientific study of human behavior and so- cial relations? How can theories and hypotheses be hypotheses. tested in social science research? How can data be • Describe the classic research design and collected? Social scientists are often accused of not being truly scientific. Are they guilty as charged, discuss some of the problems that social sci- and if so, why? What are the problems, the prom- entists have in applying this design and the ises, and the sometimes paradoxical effects of social scientific method to their research. science research? • Describe the methods that social scientists use in conducting research. ON THE WEB development, homeland security, interior, crime and justice, labor, state and international affairs, EXPLORING THE SOCIAL SCIENCES transportation, treasury, and veterans’ affairs to find an abundance of information collected by The website for this textbook (www.cengage. the U.S. government. com/politicalscience/harrison/powerandsoci- ety12e) offers resources for exploring the social • U.S. Census Bureau As depicted in the text, sciences on the Internet, as well as study tools perhaps the most extensive U.S. government including a glossary, practice quizzes, discussion database is that maintained by the U.S. Cen- questions, and Internet exercises. sus Bureau (www.census.gov). This site directs visitors to information, from A to Z, on the The U.S. government is the most reliable United States. It even keeps a population clock, single source of information on American soci- providing daily estimates of the U.S. and world ety. Virtually every U.S. government depart- populations. The Census Bureau also main- ment and agency maintains a website with access tains the online version of the most recently to extensive information relevant to a particular published Statistical Abstract of the United States sector of American society. Try searching the (http://www.census.gov/compendia/statab/). Internet using the various departments in the executive branch, including agriculture, com- merce, defense, education, energy, environment, health and human services, housing and urban

Chapter 2 / Social Sciences and the Scientific Method 39 • White House Another good place to start ABC News (http://abcnews.go.com/?lid=AB browsing is the White House website (www. CCOMGlobalMenu&lpos=News) (note the whitehouse.gov). It not only provides infor- absence of www.), CBS News (http://www. mation on the presidency but also allows cbsnews.com/), NBC News (http://www.nbc. visitors to click through to all other federal com/News_and_Sports/), CNN (www.cnn. departments and agencies. Of particular in- com), USA Today (www.usatoday.com), the terest might be the websites maintained by New York Times (www.nytimes.com), and the the U.S. State Department (www.state.gov) Washington Post (www.washingtonpost.com). and the U.S. Department of Defense (www. defense.gov). • Polling Organizations Several of the na- tion’s largest polling organizations, including • News Organizations The nation’s major Gallup (www.gallup.com), the Roper Center news organizations, both print and broadcast, for Public Opinion Research (www.ropercent- all maintain websites that post up-to-date sto- er.uconn.edu), and the National Organization ries on most of the topics of current interest for Research at the University of Chicago in the social sciences. Among the most com- (www.norc.org) host websites that oftentimes monly accessed news sites are the following: provide intriguing, up-to-date poll data. REVIEW QUIZ 5. A statement that applies to some proportion of circumstances is called MULTIPLE CHOICE a. a correlation. b. a theory. 1. A method of explanation that develops and c. a universal statement. tests theories about how observable facts or d. a probabilistic statement. events are related is called a. social science. 6. The group that will participate in the program b. the scientific method. or undergo the treatment under study is called c. field work. a. the halo group. d. a case study. b. the control group. c. the experimental group. 2. A tentative statement about a relationship be- d. the correlation group. tween observable facts or events is called a. a hypothesis. 7. In survey research, the group of people chosen b. significant. to represent the opinions of a larger group is c. a universal statement. called d. a scientific attitude. a. the population. b. the control group. 3. A significant statistical relationship is called c. the experimental group. a. correlation. d. the sample. b. a sample. c. a universal statement. 8. The aggregate of opinions of individuals on d. a scientific attitude. topics in survey research is called a. public opinion. 4. Referring to observable facts and events is b. the universe. known as c. a sample. a. a scientific attitude. d. the halo effect. b. empirical. c. a universal statement. d. normative.

40 Part 1 / The Nature and Study of Power 9. The systematic description of a society’s FILL IN THE BLANK customary behaviors, beliefs, and attitudes is called 11. Broadly defined, any organized body of a. a case study. knowledge is called ____________. b. field work. c. ethnography. 12. Doubt or skepticism about theories until they d. participant observation. have been scientifically tested is known as a _______________. 10. An in-depth investigation of a particular event in order to understand it as fully as pos- 13. A scientific test controlled by the researcher sible is called to observe effects of a specific program or a. a case study. treatment is called ________________. b. field work. c. ethnography. 14. Each person in the universe having an equal d. participant observation. chance of being selected in the sample for in- terviewing occurs in a _________________. 15. ___________reflects the range of responses in which a 95 percent chance exists that the sample reflects the universe. ANSWER KEY: 1. b; 2. a; 3. a; 4. b; 5. d; 6. c; 7. d; 8. a; 9. c; 10.a; 11. science; 12. scientific attitude; 13. an experiment; 14. random sample; 15. Sampling error.