-Earl_R._Babbie-_The_Practice_of_Social_Research_((BookFi)

374 ■ Chapter 12: Evaluation Research large high school, for example, you may be able Multiple Time-Series Designs to find another foreign language class in the same school that has a very similar student population: Sometimes the evaluation of processes occurring one that has about the same composition in terms outside of “pure” experimental controls can be of grade in school, gender, ethnicity, IQ, and so made easier by the use of more than one time- forth. The second class, then, could provide a point series analysis. Multiple time-series designs are of comparison even though it is not formally part an improved version of the nonequivalent control of the study. At the end of the semester, you could group design just described. Carol Weiss (1972: 69) give both classes the same foreign language test presents a useful example: and then compare performances. An interesting example of multiple time Here’s how two junior high schools were se- series was the evaluation of the Connecticut lected for purposes of evaluating a program aimed crackdown on highway speeding. Evaluators at discouraging tobacco, alcohol, and drug use: collected reports of traffic fatalities for several periods before and after the new program went The pairing of the two schools and their as- into effect. They found that fatalities went signment to “experimental” and “control” down after the crackdown, but since the series conditions was not random. The local Lung had had an unstable up-and-down pattern for Association had identified the school where many years, it was not certain that the drop we delivered the program as one in which was due to the program. They then compared administrators were seeking a solution to the statistics with time-series data from four admitted problems of smoking, alcohol, and neighboring states where there had been no drug abuse. The “control” school was chosen as changes in traffic enforcement. Those states a convenient and nearby demographic match registered no equivalent drop in fatalities. The where administrators were willing to allow our comparison lent credence to the conclusion surveying and breath-testing procedures. The that the crackdown had had some effect. principal of that school considered the exist- ing program of health education to be effective Although this study design is not as good as and believed that the onset of smoking was one in which subjects are assigned randomly, it’s relatively uncommon among his students. nonetheless an improvement over assessing the The communities served by the two schools experimental group’s performance without any were very similar. The rate of parental smok- comparison. That’s what makes these designs quasi ing reported by the students was just above 40 experiments instead of just fooling around. The percent in both schools. key in assessing this aspect of evaluation studies is comparability, as the following example (McAlister et al. 1980: 720) illustrates. In the initial set of observations, the experi- Rural development, a growing concern in the mental and control groups reported virtually the poor countries of the world, has captured the atten- same (low) frequency of smoking. Over the 21 tion and support of many rich countries. Through months of the study, smoking increased in both national foreign assistance programs and through groups, but it increased less in the experimental international agencies such as the World Bank, the group than in the control group, suggesting that developed countries are in the process of sharing the program affected students’ behavior. their technological knowledge and skills with the developing countries. Such programs have had multiple time-series designs The use of more mixed results, however. Often, modern techniques than one set of data that were collected over time, as do not produce the intended results when applied in accident rates over time in several states or cities, in traditional societies. so that comparisons can be made. Rajesh Tandon and L. Dave Brown (1981) undertook an experiment in which technological

Types of Evaluation Research Designs ■ 375 training would be accompanied by instruction in Figure 12-3 compares the number of active village organization. They felt it was important for initiatives by members of the two experimental poor farmers to learn how to organize and exert groups with those coming from the control groups. collective influence within their villages—getting Similar results were found for the other outcome needed action from government officials, for ex- measures. ample. Only then would their new technological skills bear fruit. Notice two things about the graph. First, there is a dramatic difference in the number of initiatives Both intervention and evaluation were at- by the two experimental groups as compared with tached to an ongoing program in which 25 villages the eleven controls. This would seem to confirm had been selected for technological training. Two the effectiveness of the special training program. poor farmers from each village had been trained in Second, notice that the number of initiatives also new agricultural technologies. Then they had been increased among the control groups. The research- sent home to share their new knowledge with ers explain this latter pattern as a result of conta- their village and to organize other farmers into gion. Because all the villages were near each other, “peer groups” who would assist in spreading that the lessons learned by peer-group members in the knowledge. Two years later, the authors randomly experimental groups were communicated in part to selected two of the 25 villages (subsequently called members of the control villages. Group A and Group B) for special training and 11 other untrained groups as controls. A careful com- This example illustrates the strengths of mul- parison of demographic characteristics showed the tiple time-series designs in situations where true experimental and control groups to be strikingly experiments are inappropriate to the program be- similar, suggesting they were sufficiently compa- ing evaluated. rable for the study. Qualitative Evaluations The peer groups from the two experimental vil- lages were brought together for special training in Although I’ve laid out the steps involved in tightly organization building. The participants were given structured, mostly quantitative evaluation research, some information about organizing and making evaluations can also be less structured and more demands on the government, and they were also qualitative. For example, Pauline Bart and Patricia given opportunities to act out dramas similar to the O’Brien (1985) wanted to evaluate different ways situations they faced at home. The training took to stop rape, so they undertook in-depth inter- three days. views with rape victims and with women who had successfully fended off rape attempts. As a general The outcome variables considered by the rule, they found that resistance (e.g., yelling, kick- evaluation all had to do with the extent to which ing, running away) was more likely to succeed members of the peer groups initiated group activi- than to make the situation worse, as women some- ties designed to improve their situation. Six types times fear it will. were studied. “Active initiative,” for example, was defined as “active effort to influence persons or Sometimes even structured quantitative evalu- events affecting group members versus passive ations can yield unexpected qualitative results. Paul response or withdrawal” (Tandon and Brown 1981: Steel is a social researcher specializing in the evalu- 180). The data for evaluation came from the jour- ation of programs aimed at pregnant drug users. nals that the peer-group leaders had been keep- One program he evaluated involved counseling by ing since their initial technological training. The public-health nurses, who warned pregnant drug researchers read through the journals and counted users that continued drug use would likely result the number of initiatives taken by members of the in underweight babies whose skulls would be an peer groups. Two researchers coded the journals average of 10 percent smaller than normal. In his independently and compared their work to test the in-depth interviews with program participants, reliability of the coding process. however, he discovered that the program omitted

376 ■ Chapter 12: Evaluation Research FIGURE 12-3 Active Initiatives over Time Source:Rajesh Tandon and L.Dave Brown,“Organization-Building for Rural Development:An Experiment in India,”Journal of Applied Behavioral Science, April–June 1981,p.182. one important piece of information: that under- that the program did not always operate on the sized babies were a bad thing. Many of the young ground the way it did in principle. Specifically, women Steel interviewed thought that smaller Steel discovered that whenever a young woman babies would mean easier deliveries. was arrested for drug use, her fellow inmates would advise her to get pregnant as soon as she In another program, a local district attorney was released on bail. That way, she would be able had instituted what would generally be regarded as to avoid prosecution (personal communication, a progressive, enlightened program. If a pregnant November 22, 1993). drug user were arrested, she could avoid prosecu- tion if she would (1) agree to stop using drugs and The most effective evaluation research is one (2) successfully complete a drug-rehabilitation that combines qualitative and quantitative compo- program. Again, in-depth interviews suggested nents. Making statistical comparisons is useful, and

The Social Context ■ 377 so is gaining an in-depth understanding of the pro- The Social Context cesses producing the observed results— or prevent- ing the expected results from appearing. This section looks at some of the logistical problems in evaluation research and presents some observa- The evaluation of the Tanzanian soap opera, tions about using evaluation research results. The presented earlier in this chapter, employed several social context also raises special ethical issues; we’ll research techniques. I’ve already mentioned the discuss these at the end of the chapter. listener surveys and data obtained from clinics. In addition, the researchers conducted numerous Logistical Problems focus groups to probe more deeply into the impact the shows had on listeners. Also, content analyses In a military context, logistics refers to moving sup- were done on the soap opera episodes themselves plies around—making sure people have food, guns, and on the many letters received from listeners. and tent pegs when they need them. Here, I use it to Both quantitative and qualitative analyses were refer to getting subjects to do what they’re supposed undertaken (Swalehe et al. 1995). to do, getting research instruments distributed and returned, and other seemingly simple tasks. These The soap opera research also offers an op- tasks are more challenging than you might guess! portunity to see the impact of different cultures on the conduct of research. I had an opportunity Motivating Sailors to experience this firsthand when I consulted on the evaluation of soap operas being planned in When Kent Crawford, Edmund Thomas, and Jef- Ethiopia. In contrast to the Western concern for frey Fink (1980) set out to find a way to motivate confidentiality in social research, respondents “low performers” in the U.S. Navy, they found out selected for interviews in rural Ethiopian vil- just how many problems can occur. The purpose of lages often took a special pride at being selected the research was to test a three-pronged program and wanted their answers broadly known in the for motivating sailors who were chronically poor community. performers and often in trouble aboard ship. First, a workshop was to be held for supervisory person- Or, sometimes, local researchers’ desires to nel, training them in the effective leadership of low please the client got in the way of the evaluation. performers. Second, a few supervisors would be For example, some pilot episodes were tested in selected and trained as special counselors and role focus groups to determine whether listeners would models—people the low performers could turn recognize any of the social messages being com- to for advice or just as sounding boards. Finally, municated. The results were more encouraging the low performers themselves would participate than could have been expected. When I asked how in workshops aimed at training them to be more the focus group subjects had been selected, the motivated and effective in their work and in their researcher described his introductory conversation: lives. The project was to be conducted aboard a “We would like you to listen to some radio pro- particular ship, with a control group selected from grams designed to encourage people to have small sailors on four other ships. families, and we’d like you to tell us whether we’ve been successful.” Not surprisingly, the small-family To begin, the researchers reported that the su- theme came through clearly to the focus group. pervisory personnel were not exactly thrilled with the program. These experiences, along with earlier com- ments in previous sections, hint at the possibility Not surprisingly, there was considerable of problems in the actual execution of evaluation resistance on the part of some supervisors research projects. Of course, all forms of research toward dealing with these issues. In fact, can run into problems, but evaluation research has their reluctance to assume ownership of the a special propensity for it, as we shall now explore further.

378 ■ Chapter 12: Evaluation Research problem was reflected by “blaming” any of within the context of real life. Although evaluation several factors that can contribute to their research is modeled after the experiment—which personnel problem. The recruiting system, suggests that the researchers have control over recruit training, parents, and society at large what happens—it takes place within frequently were named as influencing low performance— uncontrollable daily life. Of course, the participant- factors that were well beyond the control of the observer in field research doesn’t have control over supervisors. what is observed either, but that method doesn’t strive for control. Given the objectives of evalua- (Crawford et al. 1980: 488) tion research, lack of control can create real dilem- mas for the researcher. Eventually, the reluctant supervisors came around and “this initial reluctance gave way to guarded Administrative Control optimism and later to enthusiasm” (1980: 489). As suggested in the previous example, the lo- The low performers themselves were even gistical details of an evaluation project often fall more of a problem, however. The research design to program administrators. Let’s suppose you’re called for pre- and posttesting of attitudes and evaluating the effects of a “conjugal visit” program personalities, so that changes brought about by the on the morale of married prisoners. The program program could be measured and evaluated. allows inmates periodic visits from their spouses during which they can have sexual relations. On Unfortunately, all of the LPs (Low Performers) the fourth day of the program, a male prisoner were strongly opposed to taking these so-called dresses up in his wife’s clothes and escapes. Al- personality tests and it was therefore concluded though you might be tempted to assume that his that the data collected under these circum- morale was greatly improved by escaping, that turn stances would be of questionable validity. of events would complicate your study design in Ethical concerns also dictated that we not force many ways. Perhaps the warden will terminate the “testing” on the LPs. program altogether, and where’s your evaluation then? Or, if the warden is brave, he or she may (Crawford et al. 1980: 490) review the files of all those prisoners you selected randomly for the experimental group and veto the As a consequence, the researchers had to rely “bad risks.” There goes the comparability of your on interviews with the low performers and on experimental and control groups. As an alternative, the judgments of supervisors for their measures of stricter security measures may be introduced to attitude change. The subjects continued to present prevent further escapes, but the security measures problems, however. may have a dampening effect on morale. So the experimental stimulus has changed in the middle Initially, the ship’s command ordered 15 low of your research project. Some of the data will performers to participate in the experiment. Of the reflect the original stimulus; other data will reflect 15, however, one went into the hospital, another the modification. Although you’ll probably be able was assigned duties that prevented participation, to sort it all out, your carefully designed study has and a third went “over the hill” (absent without become a logistical snake pit. leave). Thus, the experiment began with 12 sub- jects. But before it was completed, three more Or suppose you’ve been engaged to evaluate subjects completed their tour of duty and left the the effect of race-relations lectures on prejudice Navy, and another was thrown out for disciplin- in the army. You’ve carefully studied the soldiers ary reasons. The experiment concluded, then, with available to you for study, and you’ve randomly eight subjects. Although the evaluation pointed to assigned some to attend the lectures and others to positive results, the very small number of subjects stay away. The rosters have been circulated weeks warranted caution in any generalizations from the experiment. The special, logistical problems of evalua- tion research grow out of the fact that it occurs

The Social Context ■ 379 in advance, and at the appointed day and hour, legislation to follow from the research. Instead, the the lectures begin. Everything seems to be going president said the commission was wrong. smoothly until you begin processing the files: The names don’t match. Checking around, you discover Less-dramatic examples of the failure to follow that military field exercises, KP duty, and a variety the implications of evaluation research could be of emergencies required some of the experimental listed endlessly. Undoubtedly every evaluation re- subjects to be elsewhere at the time of the lectures. searcher can point to studies he or she conducted— That’s bad enough, but then you learn that help- studies providing clear research results and obvious ful commanding officers sent others to fill in for policy implications—that were ignored, as “The the missing soldiers. And whom do you suppose Impact of ‘Three Strikes’ Laws” illustrates. they picked to fill in? Soldiers who didn’t have anything else to do or who couldn’t be trusted to There are three important reasons why the im- do anything important. You might learn this bit of plications of the evaluation research results are not information a week or so before the deadline for always put into practice. First, the implications may submitting your final report on the impact of the not always be presented in a way that the non- race-relations lectures. researchers can understand. Second, evaluation results sometimes contradict deeply held beliefs. These are some of the logistical problems That was certainly true in the case of the pornog- confronting evaluation researchers. You need to raphy commission. If “everybody knows” that be familiar with the problems to understand why pornography causes all manner of sexual deviance, some research procedures may not measure up to then research results to the contrary will probably the design of the classical experiment. As you read have little immediate impact. By the same token, reports of evaluation research, however, you’ll find people thought Copernicus was crazy when he said that—my earlier comments notwithstanding—it is the earth revolved around the sun. Anybody could possible to carry out controlled social research in tell the earth was standing still. The third barrier conjunction with real-life experiments. to the use of evaluation results is vested interests. If I’ve devised a new rehabilitation program that Use of Research Results I’m convinced will keep ex-convicts from return- ing to prison, and if people have taken to calling it One more facts-of-life aspect of evaluation research “The Babbie Plan,” how do you think I’m going to concerns how evaluations are used. Because the feel when your evaluation suggests the program purpose of evaluation research is to determine the doesn’t work? I might apologize for misleading success or failure of social interventions, you might people, fold up my tent, and go into another line of think it reasonable that a program would auto- work. More likely, I’d call your research worthless matically be continued or terminated based on the and begin intense lobbying with the appropriate results of the research. authorities to have my program continue. Reality isn’t that simple and reasonable, how- In the earlier example of the evaluation of ever. Other factors intrude on the assessment of drivers’ education, Philip Hilts reported some of the evaluation research results, sometimes blatantly reactions to the researchers’ preliminary results: and sometimes subtly. As president, Richard Nixon appointed a blue-ribbon national commission to Ray Burneson, traffic safety specialist with study the consequences of pornography. After a the National Safety Council, criticized the diligent, multifaceted evaluation, the commission study, saying that it was a product of a group reported that pornography didn’t appear to have (NHTSA) run by people who believe “that you any of the negative social consequences often can’t do anything to train drivers. You can only attributed to it. Exposure to pornographic materi- improve medical facilities and build stronger als, for example, didn’t increase the likelihood of cars for when the accidents happen. . . . This sex crimes. You might have expected liberalized knocks the whole philosophy of education.” (1981: 4)

380 ■ Chapter 12: Evaluation Research The Impact of “Three Strikes” Laws SACRAMENTO (AP)—The author of California’s five-year-old criminal justice system approximately $5.5 billion more per year,espe- “three strikes”law says it’s prevented more than a million crimes cially in prison costs as“career criminals”were sentenced to longer terms. and has saved $21.7 billion. Although the Rand group did not deny that the“three strikes”legislation would have some impact on crime—those serving long terms in prison Secretary of State Bill Jones offered his interpretation of the can’t commit crimes on the streets—a follow-up study (Greenwood, “three strikes”results to a Doris Tate Crimes Victim Bureau confer- Rydell,and Model 1996) suggested it was an inefficient way of attacking ence on Friday in Sacramento. crime.They estimated that a million dollars spent on“three strikes” would prevent 60 crimes,whereas the same amount spent on programs (BayInsider, March 1,1999) encouraging high school students to stay in school and graduate would prevent 258 crimes. The 1990s saw the passage of“three strikes“ laws at the federal level and in numerous states.The intention was to reduce crime rates by locking Criminologists have long recognized that most crimes are commit- up“career criminals.”Under the 1994 California law,for example,having ted by young men.Focusing attention on older“career criminals”has little a past felony conviction would double your punishment when you were or no effect on the youthful offenders.In fact,“three strikes”sentences convicted of your second felony,and the third felony conviction would disproportionately fall on those approaching the end of their criminal bring a mandatory sentence of 25 years to life.Over the years,only careers by virtue of growing older. California has enforced such laws with any vigor. In a more general critique,John Irwin and James Austin (1997) Those who supported the passage of“three strikes”legislation,such suggest that people in the United States tend to overuse prisons as as Bill Jones,quoted earlier,were quick to link the dramatic drop in crime a solution to crime,ignoring other,more effective,solutions.Often, rates during the 1990s to the new policy of getting tough with career imprisonment causes problems more serious than those it was intended criminals.While acknowledging that“three strikes”may not be the only to remedy. cause of the drop in crime,Jones added,“If you can have a 51 percent reduction in the homicide rate in five years,I would guarantee you three As with many other social interventions,however,much of the strikes is a big part of the reason.” support for“three strikes”laws in California and elsewhere stems mostly from public emotions about crime and the political implications of such In spite of the politicians’guarantees,other observers have looked emotions.Thus,evaluation research on these laws may eventually bring for additional evidence to support the impact of“three strikes”laws. about changes,but its impact is likely to be much slower than you might Some critics of these laws,for example,have noted that crime rates have logically expect. been dropping dramatically across the country,not only in California but in states that have no“three strikes”laws and in those where the courts Sources: Peter W.Greenwood,C.Peter Rydell,and Karyn Model,Diverting Children from have not enforced the“three strikes”laws that exist.In fact,crime rates a Life of Crime: Measuring Costs and Benefits (Santa Monica,CA:Rand Corporation, have dropped in those California counties that have tended to ignore that 1996);Peter W.Greenwood et al.,Three Strikes and You’re Out: Estimated Benefits and state’s law.Moreover,the drop in California crime rates began before the Costs of California’s New Mandatory-Sentencing Law (Santa Monica,CA:Rand Corpora- “three strikes”law went into effect. tion,1994);John Irwin and James Austin,It’s About Time: America’s Imprisonment Binge (Belmont,CA:Wadsworth 1997);“State Saved $21.7 Billion with Five-Year-Old In 1994,Peter Greenwood and his colleagues at the Rand Corpora- ‘Three Strikes’Law,”BayInsider, March 1,1999. tion estimated that implementation of the law would cost California’s By its nature, evaluation research takes place in the victim ends up suffering almost as much on the midst of real life, affecting it and being affected the witness stand as in the rape itself. Frequently by it. Here’s another example, well known to social the defense lawyers portray her as having encour- researchers. aged the sex act and being of shady moral charac- ter; other personal attacks are intended to deflect Rape Reform Legislation responsibility from the accused rapist. For years, many social researchers and other Criticisms such as these have resulted in a observers have noted certain problems with the variety of state-level legislation aimed at remedy- prosecution of rape cases. All too often, it is felt, ing the problems. Cassie Spohn and Julie Horney

The Social Context ■ 381 (1990) were interested in tracking the impact of TABLE 12-1 such legislation. The researchers summarize the Analysis of Rape Cases Before and After Legislation ways in which new laws were intended to make a difference: Rape The most changes are: (1) redefining rape and Before After replacing the single crime of rape with a series (N ϭ 2,252) (N ϭ 2,369) of graded offenses defined by the presence or absence of aggravating conditions; (2) chang- Outcome of case ing the consent standard by eliminating the requirement that the victim physically resist Convicted of original charge 45.8% 45.4% her attacker; (3) eliminating the requirement that the victim’s testimony be corroborated; Convicted of another charge 20.6 19.4 and (4) placing restrictions on the introduction of evidence of the victim’s prior sexual conduct. Not convicted 33.6 35.1 (1990: 2) Median prison sentence in months It was generally expected that such legislation For those convicted of original would encourage women to report being raped and would increase convictions when the cases charge 96.0 144.0 were brought to court. To examine the latter expectation, the researchers focused on the period For those convicted of another from 1970 to 1985 in Cook County, Illinois: “Our data file includes 4,628 rape cases, 405 deviate charge 36.0 36.0 sexual assault cases, 745 aggravated criminal sexual assault cases, and 37 criminal sexual assault cases” note, the one change that is evident—an increase (1990: 4). Table 12-1 shows some of what they in the length of sentences—cannot be attributed to discovered. the reform legislation itself. Spohn and Horney summarized these findings In addition to the analysis of existing statistics, as follows: Spohn and Horney interviewed judges and lawyers to determine what they felt about the impact of The only significant effects revealed by our the laws. Their responses were somewhat more analyses were increases in the average maxi- encouraging. mum prison sentences; there was an increase of almost 48 months for rape and of almost 36 Judges, prosecutors and defense attorneys months for sex offenses. Because plots of the in Chicago stressed that rape cases are taken data indicated an increase in the average sen- more seriously and rape victims treated more tence before the reform took effect, we mod- humanely as a result of the legal changes. eled the series with the intervention moved These educative effects clearly are important back one year earlier than the actual reform and should please advocates of rape reform date. The size of the effect was even larger and legislation. still significant, indicating that the effect should not be attributed to the legal reform. (1990: 17) (1990: 10) Thus, the study found other effects besides the qualitative results the researchers looked for. This Notice in the table that there was virtually no study demonstrates the importance of following up change in the percentages of cases ending in con- on social interventions to determine whether, in viction for rape or some other charge (e.g., assault). what ways, and to what degree they accomplished Hence the change in laws didn’t have any effect their intended results. on the likelihood of conviction. As the researchers Preventing Domestic Violence In a somewhat similar study, researchers in India- napolis focused their attention on the problem of wife battering, with a special concern for whether prosecuting the batterers can lead to subsequent

382 ■ Chapter 12: Evaluation Research violence. David Ford and Mary Jean Regoli (1992) Which of these possibilities most effectively set about studying the consequences of various prevents subsequent wife battering? That’s the options for prosecution allowed within the “India- question Ford and Regoli addressed. Here are some napolis Prosecution Experiment” (IPE). of their findings. Wife-battering cases can follow a variety of pat- First, their research shows that men who are terns, as Ford and Regoli summarize: brought to court for a hearing are less likely to continue beating their wives, no matter what the After a violent attack on a woman, someone outcome of the hearing. Simply being brought into may or may not call the police to the scene. If the criminal justice system has an impact. the police are at the scene, they are expected to investigate for evidence to support probable Second, women who have the right to drop cause for a warrantless arrest. If it exists, they charges later on are less likely to be abused subse- may arrest at their discretion. Upon making quently than those who do not have that right. In such an on-scene arrest, officers fill out a prob- particular, the combined policies of arresting de- able cause affidavit and slate the suspect into fendants by warrant and allowing victims to drop court for an initial hearing. When the police charges provides victims with greater security from are not called, or if they are called but do not subsequent violence than any of the other prosecu- arrest, a victim may initiate charges on her own tion policies do. by going to the prosecutor’s office and swear- ing out a probable cause affidavit with her However, giving victims the right to drop allegation against the man. Following a judge’s charges has a somewhat strange impact. Women approval, the alleged batterer may either be who exercise that right are more likely to be summoned to court or be arrested on a warrant abused later than those who insist on the prosecu- and taken to court for his initial hearing. tion proceeding to completion. The researchers interpret this as showing that future violence can (1992: 184) be decreased when victims have a sense of control supported by a clear and consistent alliance with What if a wife brings charges against her criminal justice agencies. husband and then reconsiders later on? Many courts have a policy of prohibiting such actions, in A decisive system response to any violation the belief that they are serving the interests of the of conditions for pretrial release, including of victim by forcing the case to be pursued to comple- course new violence, should serve notice that tion. In the IPE, however, some victims are offered the victim-system alliance is strong. It tells the possibility of dropping the charges if they so the defendant that the victim is serious in her choose later in the process. In addition, the court resolve to end the violence and that the system offers several other options. Because wife batter- is unwavering in its support of her interest in ing is largely a function of sexism, stress, and an securing protection. inability to deal with anger, some of the innovative possibilities in the IPE involve educational classes (Ford and Regoli 1992: 204) with anger-control counseling. The effectiveness of anger-control counsel- If the defendant admits his guilt and is will- ing cannot be assessed simply. Policies aimed at ing to participate in an anger-control counseling getting defendants into anger-control counseling program, the judge may postpone the trial for that seem to be relatively ineffective in preventing new purpose and can later dismiss the charges if the violence. The researchers noted, however, that the defendant successfully completes the program. policy effects should not be confused with actual Alternatively, the defendant may be tried and, if counseling outcomes. Some defendants scheduled found guilty, be granted probation provided he par- for treatment never received it. Considerably more ticipates in the anger-control program. Finally, the information on implementing counseling is needed defendant can be tried and, if found guilty, be given for a proper evaluation. a conventional punishment such as imprisonment. Moreover, the researchers cautioned that their results point to general patterns, and that battered

The Social Context ■ 383 wives must choose courses of action appropriate promote social programs. The example of Twende to their particular situations and should not act na Wakati at the outset of this chapter illustrated blindly on the basis of the overall patterns. The re- the methods initially developed by Sabido in the search is probably more useful in what it says about 1970s when he was Vice President for Research in ways of structuring the criminal justice system (giv- Mexico’s national broadcasting company Televisa. ing victims the right to drop charges, for example) Sabido’s first projects used television novelas to than in guiding the actions of individual victims. promote literacy and family planning. They were so successful that those methods have been used to Finally, the IPE offers an example of a com- promote a variety of social issues in the subsequent mon problem in evaluation research. Often, actual decades. practices differ from what might be expected in principle. For example, the researchers considered In part, the Sabido methodology concerns the the impact of different alternatives for bringing nature of the radio or television dramas: particu- suspects into court: Specifically, the court can issue larly the kinds of characters portrayed. Some char- either a summons ordering the husband to appear acters represent traditional points of view, some in court or a warrant to have the husband arrested. represent the modern views that the program- The researchers were concerned that having the ming is designed to promote, and some represent husband arrested might actually add to his anger a “transitional” point of view—they begin with over the situation. They were somewhat puzzled, traditional views but eventually shift to the modern therefore, to find no difference in the anger of views. Typically, when a transitional character signs husbands summoned or arrested. up for literacy classes, thousands of audience mem- bers do the same shortly thereafter. When the tran- The solution of the puzzle lay in the discrep- sitional character begins using condoms for family ancy between principle and practice: planning or safe sex, family planning clinics are mobbed the next day by men wanting condoms. Although a warrant arrest should in principle be at least as punishing as on-scene arrest, in The Sabido methodology extends beyond practice it may differ little from a summons. character definitions and plot structures. An E-E A man usually knows about a warrant for his project begins with thorough research into the arrest and often elects to turn himself in at his society where the change is being planned. A convenience, or he is contacted by the war- project in Ethiopia by the Population Media Center, rant service agency and invited to turn himself for example, aimed to lower the birthrate, encour- in. Thus, he may not experience the obvious age safe-sex practices, and enhance the status of punishment of, say, being arrested, handcuffed, women. The production of radio serial dramas was and taken away from a workplace. preceded by extensive research into the existing situations regarding the project’s aims. What was (Ford 1989: 9 –10) the birthrate? How did it differ in different regions of the country and among different ethnic groups? In summary, many factors besides the scientific What were the attitudes toward family planning? quality of evaluation research affect how its results In part these questions were answered through are used. And, as we saw earlier, factors outside national surveys. At the same time, qualitative the evaluator’s control can affect the quality of the researchers went into the countryside to observe study itself. But this “messiness” is balanced by the rural villages, talking with residents and sometimes potential contributions that evaluation research can recording the sounds of village life. make toward the betterment of human life. This formative research provided the writers The Sabido Methodology with ideas about issues to be raised and how to raise them. For example, the research indicated One of the clearest illustrations of the uses of evalu- that in some regions, abduction was still a common ation research results can be found in the omnibus method of mate selection: A man would kidnap a methodology developed by Miguel Sabido for the young woman, sexually assaulting her and holding use of “Entertainment-Education” (E-E) projects to

384 ■ Chapter 12: Evaluation Research her prisoner until she would consent to be his wife. the way that economists use indexes such as gross The formative research also revealed a widespread national product (GNP) per capita as an indicator of belief that condoms were infected with HIV, thus a nation’s economic development. meaning that condom use increased the risk of AIDS rather than reducing it. Suppose we wanted to compare the relative health conditions in different societies. One strategy The initial research also provided a baseline would be to compare their death rates (number of for subsequent evaluations. By knowing public deaths per 1,000 population). Or, more specifically, opinion toward family planning prior to the radio we could look at infant mortality: the number of programs, researchers could determine how much infants who die during their first year of life among these opinions had changed afterward. Prepro- every 1,000 births. Depending on the particular gramming measures of the use of family plan- aspect of health conditions we were interested in, ning centers could be compared with use levels we could devise any number of other measures: afterward. Many of these evaluation efforts ran physicians per capita, hospital beds per capita, days concurrently with the radio programming. For of hospitalization per capita, and so forth. Notice example, regular focus groups were used to moni- that intersocietal comparisons are facilitated by tor public reactions to each of the serial install- calculating per capita rates (dividing by the size of ments, examining whether people were reacting as the population). intended. Before we go further, recall from Chapter 11 The Sabido methodology provides an excellent the problems involved in using existing statistics. illustration of how research methods can be used In a word, they’re often unreliable, reflecting their to construct and evaluate social action programs modes of collection, storage, and calculation. With aimed at resolving social problems. To learn more this in mind, we’ll look at some of the ways we can about the Sabido methodology, see Barker and use social indicators for evaluation research on a Sabido (2005), which can be downloaded from the large scale. link on this book’s website: http://www.cengage .com/sociology/babbie. The Death Penalty and Deterrence Social Indicators Research Does the death penalty deter capital crimes such as Let’s now look at a type of research that combines murder? This question is hotly debated every time what you’ve learned about evaluation research a state considers eliminating or reinstating capital and about the analysis of existing data. A rapidly punishment and every time someone is executed. growing field in social research involves the devel- Those supporting capital punishment often argue opment and monitoring of social indicators, ag- that the threat of execution will deter potential gregated statistics that reflect the social condition of murderers from killing people. Opponents of capi- a society or social subgroup. Researchers use social tal punishment often argue that it has no effect in indicators to monitor aspects of social life in much that regard. Social indicators can help shed some light on the question. social indicators Measurements that reflect the quality or nature of social life, such as crime rates, If capital punishment actually deters people infant mortality rates, number of physicians per from committing murder, then we should expect 100,000 population, and so forth. Social indicators to find murder rates lower in those states that have are often monitored to determine the nature of so- the death penalty than in those that do not. The cial change in a society. relevant comparisons in this instance are not only possible, they’ve been compiled and published. Table 12-2 presents data compiled by William Bailey (1975) that directly contradict the view that

Social Indicators Research ■ 385 TABLE 12-2 Notice from the preceding discussion that it’s possible to use social indicators data for comparison Average Rate per 100,000 Population of First- and Second- across groups either at one time or across some Degree Murders for Capital-Punishment and Non-Capital- period of time. Often, doing both sheds the most Punishment States,1967 and 1968 light on the subject. Non-Capital- Capital- Though overall murder rates have increased Punishment Punishment substantially, by the way, the pattern observed by Bailey in 1967 and 1968 has persisted over time. States States In 1999, for example, the 38 death-penalty states had a combined murder rate of 5.86 per 100,000, 1967 1968 1967 1968 compared with a combined murder rate of 3.84 among the 12 states that lack the death penalty First-degree murder 0.18 0.21 1.47 1.58 (U.S. Bureau of the Census 2001: 22, 183). Second-degree murder 0.30 0.43 1.92 1.03 At present, work on the use of social indica- tors is proceeding on two fronts. On the one hand, Total murders 0.48 0.64 1.38 1.59 researchers are developing ever more-refined indicators—finding which indicators of a general Source:Adapted from William C.Bailey,“Murder and Capital Punishment,”in variable are the most useful in monitoring social William J.Chambliss,ed.,Criminal Law in Action.Copyright © 1975 by John Wiley life. At the same time, research is being devoted & Sons,Inc.Used by permission. to discovering the relationships among variables within whole societies. the death penalty deters murderers. In both 1967 and 1968, those states with capital punishment As with many aspects of social research, the had dramatically higher murder rates than those World Wide Web has become a valuable resource. without capital punishment did. Some people To pursue the possibilities of social indicators, you criticized the interpretation of Bailey’s data, saying might check out Sociometrics Corporation, for that most states had not used the death penalty in example (see the link on this book’s website). Or recent years, even when they had it on the books. simply search for “social indicators” using one of That could explain why it didn’t seem to work as the web search engines. a deterrent. Further analysis, however, contradicts this explanation. When Bailey compared those Computer Simulation states that hadn’t used the death penalty with those that had, he found no real difference in One of the most exciting prospects for social indica- murder rates. tors research lies in the area of computer simula- tion. As researchers begin compiling mathematical Another counterexplanation is possible, equations describing the relationships that link however. It could be the case that the interpreta- social variables to one another (for example, the tion given Bailey’s data was backward. Maybe the relationship between growth in population and existence of the death penalty as an option was the number of automobiles), those equations can a consequence of high murder rates: Those states be stored and linked to one another in a computer. with high rates instituted it; those with low rates With a sufficient number of adequately accurate didn’t institute it or repealed it if they had it on the equations on tap, researchers one day will be able books. It could be the case, then, that instituting to test the implications of specific social changes by the death penalty would bring murder rates down, computer rather than in real life. and repealing it would increase murders and still produce—in a broad aggregate—the data presented Suppose a state contemplated doubling the in Table 12-2. Not so, however. Analyses over time size of its tourism industry, for example. We could do not show an increase in murder rates when a enter that proposal into a computer-simulation state repeals the death penalty nor a decrease in model and receive in seconds or minutes a murders when one is instituted.

386 ■ Chapter 12: Evaluation Research description of all the direct and indirect conse- Ethics and Evaluation quences of the increase in tourism. We could know Research what new public facilities would be required, which public agencies such as police and fire de- As we have seen, evaluation research is by nature partments would have to be increased and by how interwoven with real-world issues. Sometimes the much, what the labor force would look like, what social interventions being evaluated raise ethi- kind of training would be required to provide it, cal issues. Evaluating the impact of busing school how much new income and tax revenue would be children to achieve educational integration will produced, and so forth, through all the intended throw the researchers directly into the political, and unintended consequences of the action. De- ideological, and ethical issues of busing itself. It’s pending on the results, the public planners might not possible to evaluate a sex-education program in say, “Suppose we increased the industry only by elementary schools without becoming involved in half,” and have a new printout of consequences the heated issues surrounding sex education itself, immediately. and the researcher will find remaining impartial difficult. The evaluation study design will require An excellent illustration of computer simula- that some children receive sex education—in tion linking social and physical variables is to be fact, you may very well be the one who decides found in the research of Donella and Dennis Mead- which children do. (From a scientific standpoint, ows and their colleagues at Dartmouth and the you should be in charge of selection.) This means Massachusetts Institute of Technology (Meadows that when parents become outraged that their et al. 1972, 1992). They’ve taken as input data child is being taught about sex, you’ll be directly known and estimated reserves of various nonre- responsible. placeable natural resources (for example, oil, coal, iron), past patterns of population and economic Now let’s look on the “bright” side. Maybe the growth, and the relationships between growth experimental program is of great value to those and use of resources. Using a complex computer- participating in it. Let’s say that the new industrial simulation model, they’ve been able to project, safety program being evaluated reduces inju- among other things, the probable number of years ries dramatically. What about the control-group various resources will last in the face of alternative members who were deprived of the program by usage patterns in the future. Going beyond the the research design? The evaluators’ actions could initially gloomy projections, such models also make be an important part of the reason that a control- it possible to chart out less gloomy futures, specify- group subject suffered an injury. ing the actions required to achieve them. Clearly, the value of computer simulation is not limited to Sometimes the name of evaluation research evaluation research, though it can serve an impor- has actually served as a mask for unethical be- tant function in that regard. havior. In Chapter 9 I discussed push polls, which pretend to evaluate the impact of various political This potentiality points to the special value of campaign accusations but intend to spread mali- evaluation research in general. Throughout hu- cious misinformation. That’s not the worst ex- man history, we’ve been tinkering with our social ample, however, as you’ll recall from the discussion arrangements, seeking better results. Evaluation of the Tuskegee experiments, in Chapter 3. research provides a means for us to learn right away whether a particular tinkering really makes Even in the most legitimate evaluation re- things better. Social indicators allow us to make search, the researcher almost always faces pres- that determination on a broad scale; coupling them sure from the people affected by the evaluation. with computer simulation opens up the possibility Often, as in the case of pharmaceutical testing, for of knowing how much we would like a particular example, those paying for the research may want a intervention, without having to experience its particular result. Evaluation researchers, therefore, risks. often find themselves under pressure to produce a particular finding.

Key Terms ■ 387 I’m sure there’s no need to point out that the intervention being studied and the population researchers must not be swayed by personal desires targeted by the intervention, and decide whether or sponsors’ demands in the design, execution, to use existing measures or devise new ones. and analysis of results; manipulating research to produce a desired result is never acceptable. This Types of Evaluation Research Designs is particularly important in the case of evalua- tion research, in that the real-world setting can • Evaluation researchers typically use experimen- create serious and far-reaching consequences for the people involved. Imagine a medical researcher tal or quasi-experimental designs. Examples of slanting drug testing results to suggest a new quasi-experimental designs include time-series drug is more effective than it is or covering up the studies and the use of nonequivalent control negative side effects of the drug, so that the drug groups. is given to patients who will not benefit from it or will actually be harmed by the “unknown” side • Evaluators can also use qualitative methods of effects. Or imagine that an evaluation of a prison rehabilitation program has been slanted to make data collection. Both quantitative and qualitative the program seem more effective than it is. Limited data analyses can be appropriate in evaluation resources might be diverted to support the ineffec- research, sometimes in the same study. tive program and possibly even harm the prisoners subjected to it. The Social Context My purpose in these comments has not been • Evaluation research entails special logistical to cast a shadow on evaluation research. Rather, I want to bring home the real-life consequences of problems because it’s embedded in the day-to-day the evaluation researcher’s actions. Ultimately, all events of real life. social research has ethical components. • The implications of evaluation research won’t MAIN POINTS necessarily be put into practice, especially if they Introduction conflict with official points of view. • Evaluation research is a form of applied research Social Indicators Research that studies the effects of social interventions. • Social indicators can provide an understanding of Topics Appropriate for Evaluation Research broad social processes. • Topics appropriate for evaluation research include • Computer-simulation models hold the promise of needs assessment studies, cost-benefit studies, allowing researchers to study the possible results monitoring studies, and program evaluations/out- of social interventions without having to incur come assessments. those results in real life. • Evaluation research is sometimes coupled with the Ethics and Evaluation Research intentions of participatory action research. • Sometimes the social interventions being assessed Formulating the Problem: Issues of in evaluation research themselves raise ethical Measurement issues. • A careful formulation of the problem, including • Evaluation research may entail added pressure to relevant measurements and criteria of success produce specific results, as desired by interested or failure, is essential in evaluation research. parties. In particular, evaluators must carefully specify outcomes, measure experimental contexts, specify • Fraudulent research results in an evaluation study can have severer consequences than conse- quences produced by other types of research. KEY TERMS The following terms are defined in context in the chapter and at the bottom of the page where the term is introduced, as well as in the comprehensive glossary at the back of the book. cost-benefit studies monitoring studies evaluation research multiple time-series designs

388 ■ Chapter 12: Evaluation Research needs assessment studies quasi experiments 5. The U.S. Bureau of Prisons engages in evalua- nonequivalent control group social indicators tion research regarding various aspects of prison program evaluation/ time-series design operations. Locate one of their studies on the web outcome assessment and write a short summary of the study design and the findings. (See the link on this book’s PROPOSING SOCIAL RESEARCH: website.) EVALUATION RESEARCH SPSS EXERCISES Evaluation research represents a research purpose rather than a particular method. In the proposal, See the booklet that accompanies your text for ex- you need to spell out the type of evaluation you’re ercises using SPSS (Statistical Package for the Social conducting and perhaps the implications of various Sciences). There are exercises offered for each chapter, possible outcomes. and you’ll also find a detailed primer on using SPSS. In earlier assignments, you’ll have spelled out Online Study Resources the data-collection and measurement methods to be used in your study. If your study is designed to de- If your book came with an access code card, visit termine the success or failure of a program, you may www.cengage.com/login to register. To purchase also want to specify the research results that will be access, please visit www.ichapters.com. deemed a positive or negative assessment in that re- 1. Before you do your final review of the chapter, gard. This may not always be appropriate or possible, but it adds integrity to the evaluation process when it take the CengageNOW pretest to help identify the can be done. areas on which you should concentrate. You’ll find information on this online tool, as well as REVIEW QUESTIONS AND EXERCISES instructions on how to access all of its great re- sources, in the front of the book. 1. Suppose a community establishes an alcohol- and 2. As you review, take advantage of the CengageNOW drug-free teen center as a way of reducing the use personalized study plan, based on your quiz of alcohol and drugs by teenagers. Describe how results. Use this study plan with its interactive ex- you might go about evaluating the effectiveness of ercises and other resources to master the material. the center. Indicate whether your design would be 3. When you’re finished with your review, take the experimental, quasi-experimental, or qualitative posttest to confirm that you’re ready to move on (or some combination of these). to the next chapter. 2. Review the evaluation of the Navy low-performer WEBSITE FOR THE PRACTICE program discussed in the chapter. Redesign the OF SOCIAL RESEARCH 12TH EDITION program and the evaluation to handle the prob- lems that appeared in the actual study. Go to your book’s website at www.cengage.com/ sociology/babbie for tools to aid you in studying for 3. Discuss some of the potential political and ethical your exams. You’ll find Tutorial Quizzes with feedback, issues that might be involved in the study you Internet Exercises, Flash Cards, Glossaries, and Essay Quiz- described in Exercise 1. zes, as well as InfoTrac College Edition search terms, sug- gestions for additional reading, Web Links, and primers 4. Take a minute to think of the many ways your for using data-analysis software such as SPSS. society has changed during your own lifetime. Specify three or four social indicators that could be used in monitoring the effects of at least one of those changes on the quality of life in your society.

This page intentionally left blank

13 Qualitative In this part of the book, we’ll discuss the analysis of Data Analysis social research data, and we’ll examine the steps that separate observation from the final reporting of 14 Quantitative findings. Data Analysis In Chapter 1, I made a fundamental distinction between qualitative and quantitative data. In the 15 The Elaboration subsequent discussions, we’ve seen that many of the Model fundamental concerns in social research apply equally to both types of data. The analysis of qualitative and 16 Statistical Analyses quantitative data, however, are quite different and will be discussed separately. 17 Reading and Writing Social Research Before outlining the specifics of Part 4, I want to offer an observation about the ease or difficulty of producing high-quality data analyses, as represented in the following table, where “1” is the easiest to do and “4” is the hardest. Qualitative Simplistic Sophisticated Quantitative 1 4 2 3

PART 4 Analysis of Data: Quantitative and Qualitative (1) In my work, I’ve seen that it’s relatively easy many established techniques for quantitative data to make some observations of social life and speculate analysis are powerful tools to use in that pursuit, the about the meaning of what has been observed. Unfortu- really powerful discoveries are never produced by the nately, such speculation is unlikely to make much of a rote administration of techniques. contribution to our understanding of social life. (4) The most difficult task for social scientists lies (2) Doing even a simplistic quantitative data in producing powerful analyses of qualitative data. This analysis is more difficult, because it requires at least requires the same dedication and ability discussed in some low-level statistical skills. All too often, however, (3); however, qualitative analysis depends more on the we’re confronted with statistical data analyses that don’t individual insights of the researcher than on the tools really mean much. Terms such as quantiphrenia and available to support the analysis. Qualitative analysis scientism have sometimes been used in reference to remains today as much an art as a science. attempts to mimic the physical sciences without any true meaning. I hope the chapters that make up this part of the book will give you some of the tools and sharpen the (3) Doing sophisticated, meaningful quantitative insights needed to produce sophisticated data analyses, data analyses requires much thought and imagination. whether qualitative or quantitative. It does not necessarily require high-powered statistics, however, as much of the work of Paul Lazarsfeld and Chapter 13 examines qualitative data analysis. Sam Stouffer shows. What’s needed instead is the We’ll begin by examining some of the theoretical willingness to search for, and the ability to recognize, groundings for this approach. Then we’ll look at some meaningful patterns among variables. Although the conceptual procedures you should find useful in the search for meaning among qualitative data. I’ll also 391

demonstrate some of the computer programs that have with quantitative data, but I think you’ll see how appro- been created specifically for qualitative data analysis. priate it is for reasoning with qualitative data as well. Near the conclusion of this chapter, we’ll examine some attempts to create criteria for assessing the quality of Chapter 16 provides an introduction to some of qualitative research. the more commonly used statistical methods in social science research, including an overview of some of the The first of several discussions on the logic of quan- more advanced methods of multivariate analysis. Rather titative data analysis is presented in Chapter 14. We’ll than merely showing how to compute statistics by begin with an examination of methods of analyzing and these methods (computers can do that), I’ve attempted presenting data related to a single variable. Then we’ll to place them in the context of earlier theoretical and turn to the relationship between two variables and learn logical discussions. Thus, you should come away from how to construct and read simple percentage tables. The this chapter knowing when to use various statistical chapter ends with a preview of multivariate analysis measures as well as how to compute them. and a discussion of sociological diagnostics, along with a look at ethical concerns. Finally, Chapter 17 addresses social research as literature: how to read it and how to write it. The Chapter 15 describes the elaboration model of data materials of this chapter are essentially bookends for analysis developed by Paul Lazarsfeld at Columbia Uni- the research process: a review of the literature early in versity. Here we’ll look further into multivariate analysis. the project involves the skills of reading social research, Chapter 15 also presents the logic of causal analysis and writing it comes into play in the communication through the use of percentage tables. We’ll apply this of your results to others in the form of your research same logic when we use other statistical techniques in report. Again, we also look at the ethics involved in this Chapter 16. This logical model was developed for use endeavor. 392

CHAPTER THIRTEEN Qualitative Data Analysis CHAPTER OVERVIEW Introduction Leviticus as Seen through NUD*IST Qualitative data analysis is the Linking Theory and Analysis Using NVivo to Understand nonnumerical assessment of Discovering Patterns Women Film Directors, by observations made through Grounded Theory Method Sandrine Zerbib participant observation,content Semiotics analysis,in-depth interviews,and Conversation Analysis The Qualitative Analysis other qualitative research of Quantitative Data techniques.Although qualitative Qualitative Data Processing analysis is as much an art as a Coding Evaluating the Quality science,it has its own logic and Memoing of Qualitative Research techniques,some of which are Concept Mapping enhanced by special computer Ethics and Qualitative programs. Computer Programs Data Analysis for Qualitative Data QDA Programs CengageNOW for Sociology Use this online tool to help you make the grade on your next exam. After reading this chapter, go to “Online Study Resources” at the end of the chapter for instructions on how to benefit from CengageNOW.

394 ■ Chapter 13: Qualitative Data Analysis Introduction and theory. As a result, I’ve already talked about qualitative data analysis in earlier discussions of Later chapters in Part 4 of the book will deal with field research and content analysis. In quantitative the quantitative analysis of social research data, research, it’s sometimes easy to get caught up in sometimes called statistical analysis. Recent decades the logistics of data collection and in the statistical of social science research have tended to focus on analysis of data, thereby losing sight of theory for quantitative data analysis techniques. This focus, a time. This is less likely in qualitative research, however, sometimes conceals another approach to where data collection, analysis, and theory are making sense of social observations: qualitative more intimately intertwined. analysis—methods for examining social research data without converting them to a numerical for- In the discussions to follow, we’ll use the mat. This approach predates quantitative analysis. image of theory offered by Anselm Strauss and It remains a useful approach to data analysis and Juliet Corbin (1994: 278) as consisting of “plausible is enjoying a resurgence of interest among social relationships proposed among concepts and sets of scientists. concepts.” They stress “plausible” to indicate that theories represent our best understanding of how Learning qualitative analysis techniques re- life operates. The more our research confirms a quires a different approach than learning quanti- particular set of relationships among particular tative ones. Although statistical analyses may concepts, however, the more confident we become intimidate some students, the steps involved can that our understanding corresponds to social sometimes be learned in a rote manner. That is, with reality. practice, the rote exercise of quantitative skills can produce an ever more sophisticated understanding Whereas qualitative research is sometimes of the logic that lies behind those techniques. undertaken for purely descriptive purposes—such as the anthropologist’s ethnography detailing ways It’s much more difficult to teach qualita- of life in a previously unknown tribe—the rest of tive analysis as a series of rote procedures. In this this chapter focuses primarily on the search for case, understanding must precede practice. In this explanatory patterns. As we’ll see, sometimes the chapter, we begin with the links between research patterns occur over time, and sometimes they take and theory in qualitative analysis. Then we exam- the form of causal relations among variables. Let’s ine some procedures that have proved useful in look at some of the ways qualitative researchers pursuing the theoretical aims. After considering uncover such patterns. some simple manual techniques, we’ll take some computer programs out for a spin. Discovering Patterns Linking Theory and Analysis John Lofland and his colleagues (2006:149–65) suggest six different ways of looking for patterns As suggested in Chapter 10 and elsewhere in in a particular research topic. Let’s suppose you’re this book, qualitative research methods involve interested in analyzing child abuse in a certain a continuing interplay between data collection neighborhood. Here are some questions you might ask yourself to make sense out of your data: qualitative analysis The nonnumerical exami- nation and interpretation of observations, for the 1. Frequencies: How often does child abuse occur purpose of discovering underlying meanings and among families in the neighborhood under patterns of relationships. This is most typical of field study? (Realize that there may be a difference research and historical research. between the frequency and what people are willing to tell you.)

Linking Theory and Analysis ■ 395 2. Magnitudes: What are the levels of abuse? How though, it’s useful to have even a partial explana- brutal are they? tion of overall orientations and actions. 3. Structures: What are the different types of abuse: You may also recall Chapter 1’s introduction physical, mental, sexual? Are they related in of idiographic explanation, wherein we attempt any particular manner? to understand a particular case fully. In the voting example, we would attempt to learn everything we 4. Processes: Is there any order among the elements could about all the factors that came into play in of structure? Do abusers begin with mental determining one person’s decision on how to vote. abuse and move on to physical and sexual This orientation lies at the base of what Huberman abuse, or does the order of elements vary? and Miles call a case-oriented analysis. 5. Causes: What are the causes of child abuse? Is In a case-oriented analysis, we would look it more common in particular social classes more closely into a particular case, say, Case or among different religious or ethnic groups? 005, who is female, middle-class, has parents Does it occur more often during good times with high expectations, and so on. These are, or bad? however, “thin” measures. To do a genuine case analysis, we need to look at a full history 6. Consequences: How does child abuse affect the of Case 005; Nynke van der Molen, whose victims, in both the short and the long term? mother trained as a social worker but is bitter What changes does it cause in the abusers? over the fact that she never worked outside the home, and whose father wants Nynke to For the most part, in examining your data you’ll work in the family florist shop. Chronology is look for patterns appearing across several observa- also important: two years ago, Nynke’s clos- tions that typically represent different cases under est friend decided to go to college, just before study, an approach called cross-case analysis. Nynke began work in a stable and just before A. Michael Huberman and Matthew Miles (1994: Nynke’s mother showed her a scrapbook from 435f) offer two strategies for cross-case analysis: the social work school. Nynke then decided to variable-oriented and the case-oriented analysis. enroll in veterinary studies. Variable-oriented analysis is similar to a model we’ve already discussed from time to time in this (1994: 436) book. If we were trying to predict the decision to attend college, Huberman and Miles suggest, we This abbreviated commentary should give you might consider variables such as “gender, socio- some idea of the detail involved in this type of economic status, parental expectations, school analysis. Of course, an entire analysis would be performance, peer support, and decision to attend more extensive and pursue issues in greater depth. college” (1994: 435). Thus, we would determine This full, idiographic examination, however, tells us whether men or women were more likely to attend nothing about people in general. It offers nothing college. The focus of our analysis would be on inter- in the way of a theory about why people choose to relations among variables, and the people observed attend college. would be primarily the carriers of those variables. cross-case analysis An analysis that involves an Variable-oriented analysis may remind you of examination of more than one case; this can be the discussion in Chapter 1 that introduced the either a variable-oriented or case-oriented analysis. idea of nomothetic explanation. The aim here is to achieve a partial, overall explanation using variable-oriented analysis An analysis that de- relatively few variables. The political pollster who scribes and/or explains a particular variable. attempts to explain voting intentions on the basis of two or three key variables is using this approach. case-oriented analysis An analysis that aims to There is no pretense that the researcher can pre- understand a particular case or several cases by look- dict every individual’s behavior nor even explain ing closely at the details of each. any one person’s motivations in full. Sometimes,

396 ■ Chapter 13: Qualitative Data Analysis Even so, in addition to understanding one per- from the ground up, with no preconceptions, son in great depth, the researcher sees the critical though some research may build and elaborate on elements of the subject’s experiences as instances earlier grounded theories. of more-general social concepts or variables. For example, Nynke’s mother’s social work training can Grounded theory was first developed by the also be seen as “mother’s education.” Her friend’s sociologists Barney Glaser and Anselm Strauss decision can be seen as “peer influence.” More (1967) in an attempt to come to grips with their specifically, these could be seen as independent clinical research in medical sociology. Since then, variables having an impact on the dependent vari- it has evolved as a method, with the cofound- able of attending college. ers taking it in slightly different directions. The following discussion will deal with the basic con- Of course, one case does not a theory make— cepts and procedures of the Grounded Theory hence Huberman and Miles refer to cross-case Method (GTM). analysis, in which the researcher turns to other subjects, looking into the full details of their lives In addition to the fundamental, inductive tenet as well but paying special attention to the variables of building theory from data, GTM employs the that seemed important in the first case. How much constant comparative method. As Glaser and and what kind of education did other subjects’ Strauss originally described this method, it involved mothers have? Is there any evidence of close four stages (1967: 105–13): friends attending college? 1. “Comparing incidents applicable to each Some subsequent cases will closely parallel category.” As Glaser and Strauss researched the first one in the apparent impact of particular the reactions of nurses to the possible death variables. Other cases will bear no resemblance of patients in their care, the researchers found to the first. These latter cases may require the that the nurses were assessing the “social loss” identification of other important variables, which attendant upon a patient’s death. Once this may invite the researcher to explore why some concept arose in the analysis of one case, they cases seem to reflect one pattern while others looked for evidence of the same phenomenon reflect another. in other cases. When they found the concept arising in the cases of several nurses, they Grounded Theory Method compared the different incidents. This process is similar to conceptualization as described in The cross-case method just described should sound Chapter 5—specifying the nature and dimen- somewhat familiar. In the discussion of grounded sions of the many concepts arising from the theory in Chapter 10, we saw how qualitative data. researchers sometimes attempt to establish theories on a purely inductive basis. This approach begins 2. “Integrating categories and their properties.” with observations rather than hypotheses and Here the researcher begins to note relationships seeks to discover patterns and develop theories among concepts. In the assessment of social loss, for example, Glaser and Strauss found Grounded Theory Method (GTM) An inductive that nurses took special notice of a patient’s approach to research, introduced by Barney Glaser age, education, and family responsibilities. For and Anselm Strauss, in which theories are generated these relationships to emerge, however, it was solely from an examination of data rather than be- necessary for the researchers to have noticed all ing derived deductively. these concepts. constant comparative method A component of 3. “Delimiting the theory.” Eventually, as the pat- the Grounded Theory Method in which observa- terns of relationships among concepts become tions are compared with one another and with the clearer, the researcher can ignore some of evolving inductive theory. the concepts that were initially noted but are evidently irrelevant to the inquiry. In addition

Linking Theory and Analysis ■ 397 to the number of categories being reduced, the FIGURE 13-1 theory itself may become simpler. In the exam- Matching Signs and their Meanings ination of social loss, for example, Glaser and Strauss found that the assessment processes There is no meaning inherent in any sign, could be generalized beyond nurses and dying however. Meanings reside in minds. So, a particu- patients: They seemed to apply to the ways all lar sign means something to a particular person. staff dealt with all patients (dying or not). However, the agreements we have about the meanings associated with particular signs make 4. “Writing theory.” Finally, the researcher must semiotics a social science. As Manning and Cullum- put his or her findings into words to be shared Swan point out: with others. As you may have already experi- enced for yourself, the act of communicating For example, a lily is an expression linked con- your understanding of something actually ventionally to death, Easter, and resurrection modifies and even improves your own grasp of as a content. Smoke is linked to cigarettes and the topic. In GTM, the writing stage is regarded to cancer, and Marilyn Monroe to sex. Each of as a part of the research process. A later section these connections is social and arbitrary, so that of this chapter (on memoing) elaborates on this many kinds of links exist between expression point. and content. This brief overview should give you an idea (1994: 466) of how grounded theory proceeds. The many techniques associated with GTM can be found To explore this contention, see if you can link both in print and on the web. One key publica- the signs with their meanings in Figure 13-1. I’m tion is Anselm Strauss and Juliet Corbin’s Basics of confident enough that you know all the “correct” Qualitative Research (1998), which elaborates on associations that there’s no need for me to give the and extends many of the concepts and techniques answers. (OK, you should have said 1c, 2a, 3b, 4e, found in the original Glaser/Strauss volume. On 5d.) The point is this: What do any of these signs the web, you can search for “grounded theory” to have to do with their “meanings”? Draft an e-mail see a wealth of articles. message to a Martian social scientist explaining the logic at work here. (You might want to include GTM is only one analytic approach to qualita- some “emoticons” like : ) —another example of tive data. In the remainder of this section, we’ll semiotics.) take a look at some other specialized techniques. There is no doubt a story behind each of the Semiotics linkages in Figure 13-1, and the meanings you and I “know” today have been socially con- Semiotics is commonly defined as the “science of structed. Semiotic analysis involves a search for the signs” and has to do with symbols and meanings. It’s commonly associated with content analysis, semiotics The study of signs and the meanings which was discussed in Chapter 11, though it can associated with them. This is commonly associated be applied in a variety of research contexts. with content analysis. Peter Manning and Betsy Cullum-Swan (1994: 466) offer some sense of the applicability of semiotics, as follows: “Although semiotics is based on language, language is but one of the many sign systems of varying degrees of unity, applicability, and complexity. Morse code, etiquette, mathemat- ics, music, and even highway signs are examples of semiotic systems.”

398 ■ Chapter 13: Qualitative Data Analysis FIGURE 13-2 Mixed Signals? meanings intentionally or unintentionally attached cated, Goffman asked. What in particular did the to signs. ads say about men and women? Consider the sign shown in Figure 13-2, from a Analyzing pictures containing both men and hotel lobby in Portland, Oregon. What’s being com- women, Goffman was struck by the fact that men municated by the rather ambiguous sign? The first were almost always bigger and taller than the sentence seems to be saying that the hotel is up-to- women accompanying them. (In many cases, in date with the current move away from tobacco in fact, the picture managed to convey the distinct the United States. Guests who want a smoke-free impression that the women were merely accom- environment need look no farther: This is a healthy panying the men.) Although the most obvious place to stay. At the same time, says the second explanation is that men are, on average, heavier sentence, the hotel would not like to be seen as and taller than women, Goffman suggested the inhospitable to smokers. There’s room for everyone pattern had a different meaning: that size and place- under this roof. No one need feel excluded. This ment implied status. Those larger and taller presum- sign is more easily understood within a marketing ably had higher social standing—more power and paradigm than one of logic. authority (1979: 28). Goffman suggested that the ads communicated that men were more important The “signs” examined in semiotics, of course, than women. are not limited to this kind of sign. Most are quite different, in fact. Signs are any things that In the spirit of Freud’s comment that “some- are assigned special meanings. They can include times a cigar is just a cigar” (he was a smoker), how logos, animals, people, and consumer products. would you decide whether the ads simply reflected Sometimes the symbolism is a bit subtle. A classic the biological differences in the average sizes of analysis can be found in Erving Goffman’s Gender men and women or whether they sent a message Advertisements (1979). Goffman focused on advertis- about social status? In part, Goffman’s conclusion ing pictures found in magazines and newspapers. was based on an analysis of the exceptional cases: The overt purpose of the ads, of course, was to sell those in which the women appeared taller than specific products. But what else was communi- the men. In these cases, the men were typically of

Linking Theory and Analysis ■ 399 a lower social status—the chef beside the society tiny of the way we converse with one another. In matron, for example. This confirmed Goffman’s the examination of ethnomethodology in Chapter main point that size and height indicated social 10, you saw some examples of conversation analy- status. sis. Here we’ll look a little more deeply into that technique. The same conclusion was to be drawn from pictures with men of different heights. Those of David Silverman (1999), reviewing the work higher status were taller, whether it was the gentle- of other CA theorists and researchers, speaks of man speaking to a waiter or the boss guiding the three fundamental assumptions. First, conversa- work of his younger assistants. Where actual height tion is a socially structured activity. Like other social was unclear, Goffman noted the placement of structures, it has established rules of behavior. heads in the picture. The assistants were crouch- For example, we’re expected to take turns, with ing down while the boss leaned over them. The only one person speaking at a time. In telephone servant’s head was bowed so it was lower than that conversations, the person answering the call is ex- of the master. pected to speak first (e.g., “Hello”). You can verify the existence of this rule, incidentally, by picking The latent message conveyed by the ads, then, up the phone without speaking. You may recall was that the higher a person’s head appeared in that this is the sort of thing ethnomethodologists the ad, the more important that person was. And tend to do. in the great majority of ads containing men and women, the former were clearly portrayed as more Second, Silverman points out that conversa- important. The subliminal message in the ads, tions must be understood contextually. The same whether intended or not, was that men are more utterance will have different meanings in different powerful than women and enjoy a higher status. contexts. For example, notice how the meaning of “Same to you!” varies if preceded by “I don’t like Goffman examined several differences besides your looks” or by “Have a nice day.” physical size in the portrayal of men and women. As another example, men were typically portrayed Third, CA aims to understand the structure and in active roles, women in passive ones. The (male) meaning of conversation through excruciatingly doctor examined the child while the (female) accurate transcripts of conversations. Not only are nurse or mother looked on, often admiringly. A the exact words recorded, but all the uhs, ers, bad man guided a woman’s tennis stroke (all the while grammar, and pauses are also noted. Pauses, in fact, keeping his head higher than hers). A man gripped are recorded to the nearest tenth of a second. the reins of his galloping horse, while a woman rode behind him with her arms wrapped around The practical uses of this type of analysis are his waist. A woman held the football, while a man many. Ann Marie Kinnell and Douglas Maynard kicked it. A man took a photo, which contained (1996), for example, analyzed conversations only women. between staff and clients at an HIV-testing clinic to examine how information about safe sex was com- Goffman suggested that such pictorial patterns municated. Among other things, they found that subtly perpetuated a host of gender stereotypes. the staff tended to provide standard information Even as people spoke publicly about gender equal- rather than try to speak directly to a client’s specific ity, these advertising photos established a quiet circumstances. Moreover, they seemed reluctant to backdrop of men and women in the “proper roles.” give direct advise about safe sex, settling for infor- mation alone. Conversation Analysis conversation analysis (CA) A meticulous analysis Ethnomethodology, as you’ll recall, aims to un- of the details of conversation, based on a complete cover the implicit assumptions and structures in transcript that includes pauses, hems, and also haws. social life. Conversation analysis (CA) seeks to pursue that aim through an extremely close scru-

400 ■ Chapter 13: Qualitative Data Analysis These discussions should give you some sense beginnings. Now you need that information. If all of the variety of qualitative analysis methods avail- your notes have been catalogued by topic, retriev- able to researchers. Now let’s look at some of the ing those you need should be straightforward. As a data-processing and data-analysis techniques com- simple format for coding and retrieval, you might monly used in qualitative research. have created a set of file folders labeled with vari- ous topics, such as “History.” Data retrieval in this Qualitative Data Processing case means pulling out the “History” folder and rifling through the notes it contains until you find Let me begin this section with a warning. The what you need. activity we’re about to examine is as much art as science. At the very least, there are no cut-and- As you’ll see later in this chapter, there are now dried steps that guarantee success. sophisticated computer programs that allow for a faster, more certain, and more precise retrieval It’s a lot like learning how to paint with process. Rather than looking through a “History” watercolors or compose a symphony. Education file, you can go directly to notes dealing with in such activities is certainly possible, and univer- the “Earliest History” or the “Founding” of the sity courses are offered in both. Each has its own movement. conventions and techniques as well as tips you may find useful as you set out to create art or music. Coding has another, even more important pur- However, instruction can carry you only so far. The pose. As discussed earlier, the aim of data analysis is final product must come from you. Much the same the discovery of patterns among the data, patterns can be said of qualitative data processing. that point to theoretical understandings of social life. The coding and relating of concepts is key to This section presents some ideas relating to this process and requires a more refined system the coding of qualitative data, writing memos, and than a set of manila folders. In this section, we’ll mapping concepts graphically. Although far from a assume that you’ll be doing your coding manually. “how-to” manual, these ideas give a useful starting A later section of the chapter will illustrate the point for finding order in qualitative data. use of computer programs for qualitative data analysis. Coding Coding Units Whether you’ve engaged in participant observa- tion, in-depth interviewing, collecting biographical As you may recall from the earlier discussion of narratives, doing content analysis, or some other content analysis, for statistical analysis it’s impor- form of qualitative research, you’ll now be in the tant to identify a standardized unit of analysis prior possession of a growing mass of data—most typi- to coding. If you were comparing American and cally in the form of textual materials. Now what do French novels, for example, you might evaluate you do? and code sentences, paragraphs, chapters, or whole books. It would be important, however, to code the The key process in the analysis of qualitative same units for each novel analyzed. This unifor- social research data is coding—classifying or catego- mity is necessary in a quantitative analysis, as it al- rizing individual pieces of data—coupled with some lows us to report something like “23 percent of the kind of retrieval system (see Chapter 11). Together, paragraphs contained metaphors.” This is only pos- these procedures allow you to retrieve materials sible if we’ve coded the same unit—paragraphs—in you may later be interested in. each of the novels. Let’s say you’re chronicling the growth of Coding data for a qualitative analysis, how- a social movement. You recall writing up some ever, is quite different. The concept is the organizing notes about the details of the movement’s earliest principle for qualitative coding. Here the units of text appropriate for coding will vary within a given

Qualitative Data Processing ■ 401 document. Thus, in a study of organizations, “Size” Creating Codes might require only a few words per coding unit, whereas “Mission” might take a few pages. Or, a So, what should your code categories be? Glaser lengthy description of a heated stockholders meet- and Strauss (1967: 101f) allow for the possibility of ing might be coded as “Internal Dissent.” coding data for the purpose of testing hypotheses that have been generated by prior theory. In that Realize also that a given code category may case, then, the codes would be suggested by the be applied to textual materials of quite differ- theory, in the form of variables. ent lengths. For example, some references to the organization’s mission may be brief, others lengthy. In this section, however, we’re going to focus Whereas standardization is a key principle in quan- on the more common processes of open cod- titative analysis, this is not the case in qualitative ing, axial coding, and selective coding. Strauss analysis. and Corbin (1998: 102) describe open coding as follows: Coding as a Physical Act To uncover, name, and develop concepts, Before continuing with the logic of coding, let’s we must open up the text and expose the take a moment to see what it actually looks like. thoughts, ideas, and meanings contained Lofland and his colleagues offer this description of therein. Without this first analytic step, the rest manual filing: of the analysis and the communication that fol- lows could not occur. Broadly speaking, during Prior to the widespread availability of personal open coding, data are broken down into dis- computers beginning in the late 1980s, coding crete parts, closely examined, and compared for frequently took the specific physical form of similarities and differences. Events, happenings, filing. The researcher established an expanding objects, and actions/interactions that are found set of file folders with code names on the tabs to be conceptually similar in nature or related and physically placed either the item of data in meaning are grouped under more abstract itself or a note that referenced its location in concepts termed “categories.” another file folder. Before photocopying was easily available and cheap, some fieldwork- Although the analysis of data will quickly ers typed their fieldnotes with carbon paper, advance to an iterative interplay of the three types wrote codes in the margins of the copies of the of coding, open coding is the logical starting point. notes, and cut them up with scissors. They then Beginning with some body of text (part of an placed the resulting slips of paper in corre- interview, for example), you read and reread a pas- sponding file folders. sage, seeking to identify the key concepts contained within it. Any particular piece of data may be given (2006: 203) several codes, reflecting as many concepts. For example, notice all the concepts contained in this As Lofland and his colleagues point out, per- comment by a student interviewee: sonal computers have greatly simplified this task. However, the image of slips of paper that contain I thought the professor should have given me at text and are put in folders representing code cat- least partial credit for the homework I turned in. egories is useful for understanding the process of coding. In the next section, when I suggest that we Some obvious codes are “Professor,” “Homework,” code a textual passage with a certain code, imagine and “Grading.” The result of open coding is the that we have the passage typed on a slip of paper and that we place it in a file folder bearing the open coding The initial classification and label- name of the code. Whenever we assign two codes ing of concepts in qualitative data analysis. In open to a passage, imagine placing duplicate copies of the coding, the codes are suggested by the researchers’ passage in two different folders representing the examination and questioning of the data. two codes.

402 ■ Chapter 13: Qualitative Data Analysis identification of numerous concepts relevant to the to these passages in the Book of Leviticus (Revised subject under study. The open coding of more and Standard Version): more text will lengthen the list of codes. 18:22 You shall not lie with a male as with a Axial coding aims to identify the core concepts 20:13 woman; it is an abomination. in the study. Although axial coding uses the results of open coding, more concepts can be identified If a man lies with a male as with a through continued open coding after the axial cod- woman, both of them have committed an ing has begun. Axial coding involves a regrouping abomination; they shall be put to death, of the data, in which the researcher uses the open- their blood is upon them. code categories and looks for more-analytic con- cepts. For example, the passage just given also car- Although the point of view expressed here ries the concept of “perceptions of fairness,” which seems unambiguous, you might decide to examine might appear frequently in the student interviews, it in more depth. Perhaps a qualitative analysis of thereby suggesting that it’s an important element Leviticus can yield a fuller understanding of where in understanding students’ concerns. Another axial these injunctions against homosexuality fit into the code reflected in the student comment might be larger context of Judeo-Christian morality. “power relationships,” because the professor is seen to exercise power over the student. Let’s start our analysis by examining the two passages just quoted. We might begin by coding Selective coding seeks to identify the central each passage with the label “Homosexuality.” This code in the study: the one that the other codes all is clearly a key concept in our analysis. Whenever related to. Both of the axial codes just mentioned we focus on the issue of homosexuality in our might be restructured as aspects of a more gen- analysis of Leviticus, we want to consider these two eral concept: “professor–student relationships.” Of passages. course, in a real data analysis, decisions such as the ones we’ve been discussing would arise from Because homosexuality is such a key concept, masses of textual data, not from a single quotation. let’s look more closely into what it means within The basic notion of the Grounded Theory Method the data under study. We first notice the way is that patterns of relationships can be teased out of homosexuality is identified: a man lying with a man an extensive, in-depth examination of a large body “as with a woman.” Although we can imagine a of observations. lawyer seeking admission to heaven saying, “But here’s my point; if we didn’t actually lie down . . .” Here’s a concrete example to illustrate how it seems safe to assume the passage refers to having you might engage in this form of analysis. Sup- sex, though what specific acts might or might not pose you’re interested in the religious bases for be included isn’t clear. homophobia. You’ve interviewed some people opposed to homosexuality who cite a religious Notice, however, that the injunctions appear basis for their feelings. Specifically, they refer you to concern male homosexuality only; lesbianism is not mentioned. In our analysis, then, each of these axial coding A reanalysis of the results of open passages might also be coded “Male Homosexual- coding in the Grounded Theory Method, aimed at ity.” This illustrates two more aspects of coding: identifying the important, general concepts. (1) Each unit can have more than one code and (2) hierarchical codes (one included within an- selective coding In Grounded Method Theory, other) can be used. Now each passage has two this analysis builds on the results of open coding and codes assigned to it. axial coding to identify the central concept that or- ganizes the other concepts that have been identified An even more general code might be intro- in a body of textual materials. duced at this point: “Prohibited Behavior.” This is important for two reasons. First, homosexuality is not inherently wrong, from an analytic standpoint. The purpose of the study is to examine the way

Qualitative Data Processing ■ 403 it’s made wrong by the religious texts in question. 11:12 Everything in the waters that has not Second, our study of Leviticus may turn up other 11:13 fins and scales is an abomination to you. behaviors that are prohibited. 11:14 And these you shall have in abomination There are at least two more critical concepts in 11:15 among the birds, they shall not be eaten, the passages: “Abomination” and “Put to Death.” 11:16 they are an abomination: the eagle, the Notice that although these are clearly related to 11:17 vulture, the osprey, “Prohibited Behavior,” they are hardly the same. 11:18 Parking without putting money in the meter is 11:19 the kite, the falcon according to its kind, prohibited, but few would call it an abomination 11:20 and fewer still would demand the death penalty for 11:41 every raven according to its kind, that transgression. Let’s assign these two new codes to our first two passages. 11:42 the ostrich, the nighthawk, the sea gull, the hawk according to its kind, At this point, we want to branch out from the 11:43 two key passages and examine the rest of Leviti- the owl, the cormorant, the ibis, cus. We therefore examine and code each of the 18:22 remaining chapters and verses. In our subsequent 19:6 the water hen, the pelican, the carrion analyses, we’ll use the codes we have already and vulture, add new ones as appropriate. When we do add 19:7 new codes, it will be important to review the pas- 19:8 the stork, the heron according to its kind, sages already coded to see whether the new codes the hoopoe, and the bat. apply to any of them. 20:13 All winged insects that go upon all Here are the passages we decide to code fours are an abomination to you. “Abomination.” (I’ve boldfaced the abominations.) Every swarming thing that swarms upon 7:18 If any of the flesh of the sacrifice of his the earth is an abomination; it shall not be 7:21 peace offering is eaten on the third eaten. 11:10 day, he who offers it shall not be ac- 11:11 cepted, neither shall it be credited to him; Whatever goes on its belly, and whatever it shall be an abomination, and he who goes on all fours, or whatever has many eats of it shall bear his iniquity. feet, all the swarming things that swarm upon the earth, you shall not eat; for they And if any one touches an unclean are an abomination. thing, whether the uncleanness of man or an unclean beast or any unclean You shall not make yourselves abominable abomination, and then eats of the flesh with any swarming thing that swarms; and of the sacrifice of the LORD’s peace of- you shall not defile yourselves with them, ferings, that person shall be cut off from lest you become unclean. his people. You shall not lie with a male as with a But anything in the seas or the rivers woman; it is an abomination. that has not fins and scales, of the swarming creatures in the waters and of It shall be eaten the same day you offer it, the living creatures that are in the waters, or on the morrow; and anything left over is an abomination to you. until the third day shall be burned with fire. They shall remain an abomination to you; of their flesh you shall not eat, If it is eaten at all on the third day, it is and their carcasses you shall have in an abomination; it will not be accepted, abomination. and every one who eats it shall bear his iniquity, because he has profaned a holy thing of the LORD; and that person shall be cut off from his people. If a man lies with a male as with a woman, both of them have committed an

404 ■ Chapter 13: Qualitative Data Analysis 20:25 abomination; they shall be put to death, As you can see, the death penalty is broadly their blood is upon them. applied in Levicitus: everything from swearing to murder, including male homosexuality somewhere You shall therefore make a distinction in between. between the clean beast and the unclean, and between the unclean bird and the An extended analysis of prohibited behavior, clean; you shall not make yourselves short of abomination and death, also turns up a abominable by beast or by bird or lengthy list. Among them are slander, vengeance, by anything with which the ground grudges, cursing the deaf, and putting stumbling teems, which I have set apart for you to blocks in front of blind people. In chapter 19, verse hold unclean. 19, Leviticus quotes God as ordering, “You shall not let your cattle breed with a different kind; you Male homosexuality, then, isn’t the only shall not sow your field with two kinds of seed; abomination identified in Leviticus. As you nor shall there come upon you a garment of cloth compare these passages, looking for similari- made of two kinds of stuff.” Shortly thereafter, he ties and differences, it will become apparent that adds, “You shall not eat any flesh with the blood in most of the abominations have to do with dietary it. You shall not practice augury or witchcraft. You rules—specifically those potential foods deemed shall not round off the hair on your temples or mar “unclean.” Other abominations flow from the the edges of your beard.” Tattoos were prohibited, mishandling of ritual sacrifices. “Dietary Rules” and though Leviticus is silent on body piercing. Refer- “Ritual Sacrifices” thus represent additional codes ences to all of these practices would be coded “Pro- to be used in our analysis. hibited Acts” and perhaps given additional codes as well (recall “Dietary Rules”). Earlier, I mentioned the death penalty as another concept to be explored in our analysis. I hope this brief glimpse into a possible analysis When we take this avenue, we discover that many will give you some idea of the process by which behaviors besides male homosexuality warrant the codes are generated and applied. You should also death penalty. Among them are these: have begun to see how such coding would allow you to better understand the messages being put 20:2 Giving your children to Molech (human forward in a text and to retrieve data appropriately sacrifice) as you need them. 20:9 Cursing your father or mother 20:10 Adultery with your neighbor’s wife Memoing 20:11 Adultery with your father’s wife 20:12 Adultery with your daughter-in-law In the Grounded Theory Method, the coding pro- 20:14 Taking a wife and her mother also cess involves more than simply categorizing chunks 20:15 Men having sex with animals (the animals of text. As you code data, you should also be using are to be killed, also) the technique of memoing—writing memos or 20:16 Women having sex with animals notes to yourself and others involved in the project. 20:27 Being a medium or wizard Some of what you write during analysis may end 24:16 Blaspheming the name of the Lord up in your final report; much of it will at least 24:17 Killing a man stimulate what you write. memoing Writing memos that become part of In GTM, these memos have a special the data for analysis in qualitative research such as significance. Strauss and Corbin (1998: 217) distin- grounded theory. Memos can describe and define guish three kinds of memos: code notes, theoretical concepts, deal with methodological issues, or offer notes, and operational notes. initial theoretical formulations. Code notes identify the code labels and their meanings. This is particularly important because,

Qualitative Data Processing ■ 405 as in all social science research, most of the terms FIGURE 13-3 we use with technical meanings also have mean- An Example of Concept Mapping ings in everyday language. It’s essential, therefore, to write down a clear account of what you mean comes from practice. Even if you don’t have a by the codes used in your analysis. In the Leviticus research project underway, you can practice now analysis, for example, you would want a code on class notes. Or start a journal and code it. note regarding the meaning of “Abomination” and how you’ve used that code in your analysis Concept Mapping of text. It should be clear by now that qualitative data Theoretical notes cover a variety of topics: analysts spend a lot of time committing thoughts to reflections of the dimensions and deeper mean- paper (or to a computer file), but this process is not ings of concepts, relationships among concepts, limited to text alone. Often, we can think out rela- theoretical propositions, and so on. All of us have tionships among concepts more clearly by putting ruminated over the nature of something, trying the concepts in a graphic format, a process called to think it out, to make sense out of it. In qualita- concept mapping. Some researchers put all their tive data analysis, it’s vital to write down these major concepts on a single sheet of paper, whereas thoughts, even those you’ll later discard as useless. others spread their thoughts across several sheets They will vary greatly in length, though you should of paper, blackboards, magnetic boards, computer limit them to a single main thought so that you pages, or other media. Figure 13-3 shows how we can sort and organize them later. In the Leviticus might think out some of the concepts of Goffman’s analysis, one theoretical note might discuss the examination of gender and advertising. (This im- way that most of the injunctions implicitly address age was created through the use of Inspiration, a the behavior of men, with women being mostly concept-mapping computer program.) incidental. concept mapping The graphic display of concepts Operational notes deal primarily with method- and their interrelations, useful in the formulation of ological issues. Some will draw attention to data- theory. collection circumstances that may be relevant to understanding the data later on. Others will consist of notes directing future data collection. Writing these memos occurs throughout the data-collection and analysis process. Thoughts demanding memos will come to you as you reread notes or transcripts, code chunks of text, or discuss the project with others. It’s a good idea to get in the habit of writing out your memos as soon as possible after the thoughts come to you. Notice that whereas we often think of writing as a linear process, starting at the beginning and moving through to the conclusion, memoing is very different. It might be characterized as a process of creating chaos and then finding order within it. To explore this process further, refer to the works cited in this discussion and at the end of the chapter. You’ll also find a good deal of informa- tion on the web. For example, see the link, on this book’s website, for Barney Glaser’s rules on memo- ing. Ultimately, the best education in this process

406 ■ Chapter 13: Qualitative Data Analysis Incidentally, many of the topics discussed in code words alongside passages in your notes so that this section have useful applications in quantitative you can search for those keywords later. as well as qualitative analyses. Certainly, concept mapping is appropriate in both types of analysis. Database and spreadsheet programs can also be The several types of memos would also be use- used for processing and analyzing qualitative data. ful in both. And the discussion of coding readily Figure 13-4 is a simple illustration of how some applies to the coding of open-ended questionnaire of the verses from Leviticus might be manipulated responses for the purpose of quantification and within a spreadsheet. The three columns to the left statistical analysis. (We’ll look at coding again in represent three of the concepts we’ve discussed. An the next chapter, on quantifying data.) “X” means that the passage to the right contains that concept. As shown, the passages are sorted Having noted the overlap of qualitative and in such a way as to gather all those dealing with quantitative techniques, it seems fitting now to punishment by death. Another simple “sort” com- address an instrument that is primarily associ- mand would gather all those dealing with sex, ated with quantitative research but that is proving with homosexuality, or any of the other concepts quite valuable for qualitative analysts as well—the coded. personal computer. QDA Programs Computer Programs for Qualitative Data While the simple spreadsheet illustration just given touched on how computers might be used for ana- The advent of computers—mainframe and lyzing qualitative social research data, there is now personal—has been a boon to quantitative re- a long list of sophisticated computer programs avail- search, allowing the rapid calculation of extremely able for this purpose. Where the analyst’s problem complex statistics. The importance of the computer used to be simply finding such a program, the prob- for qualitative research has been somewhat more lem now centers on choosing one. Here are a few slowly appreciated. Some qualitative researchers commonly used qualitative data analysis (QDA) were quick to adapt the basic capacities of comput- programs with online sites where you can learn ers to nonnumerical tasks, but it took a bit longer more about them and, often, download demo cop- for programmers to address the needs of qualitative ies (see all the links on this book’s website: http:// research per se. Today, however, several powerful www.cengage.com/sociology/babbie). programs are available. AnSWR Let’s start this section with a brief overview of Atlas.ti some of the ways you can use basic computer tools Ethno in qualitative research. Perhaps only those who Ethnograph can recall hours spent with carbon paper and HyperQual Wite-Out can fully appreciate the glory of com- HyperRESEARCH puters in this regard. “Easier editing” and “easier HyperTRANSCRIBE duplication” simply do not capture the scope of the MAXQDA advance. NVivo 8 (NUD*IST) QDA Miner Moving beyond the basic recording and storage Qualrus of data, simple word-processing programs can be SPAD used for some data analysis. The “find” or “search” TAMS command will take you to passages containing keywords. Or, going one step further, you can type

Computer Programs for Qualitative Data ■ 407 FIGURE 13-4 Using a Spreadsheet for Qualitative Analysis T-LAB on understanding the experiences of women film directors. Weft Leviticus as Seen through There are also some powerful online resources NUD*IST to assist you in choosing the program best suited to your needs. Sociologists at the University of Surrey, Let’s take a closer look at how qualitative data anal- England, have prepared an overview of these and ysis programs operate. We’ll consider one of the other programs with descriptions and contact in- programs just mentioned, NUD*IST (Nonnumeric formation. You can find this resource at the link on Unstructured Data, Index Searching, and Theoriz- this book’s website. ing). This popular program for teaching qualitative social research offers a fair representation of QDA Another excellent resource is “Choosing a programs. (Note that NVivo 8 has now superseded CAQDAS Software Package” by Ann Lewins and NUD*IST; however, some researchers still use the Christina Silver (2006), which can be found at the older variant of the software.) corresponding link on this book’s website. This will familiarize you with some of the key features Although it is possible to type directly into in QDA programs and will help you choose which NUD*IST the text materials to be coded, usually ma- one is best suited to your purposes. This website terials already in existence—such as field notes or, provides a brief description of each of the programs in this case, the verses of Leviticus—are imported into listed, along with contact links. the program. Menu-based commands do this easily, though the text must be in a plaintext format (that is, Let’s turn now to a couple of illustrations of without word-processing or other formatting). QDA programs at work. While all the available programs differ somewhat from one another, I Figure 13-5 shows how the text is dis- think these illustrations will give you a general played within NUD*IST. For the illustrations in sense of how to use computers to analyze qualita- this section, I’ve used the Macintosh version of tive data. We’ll begin with a brief examination of NUD*IST, but the Windows version is similar. The Leviticus, and then we’ll examine a project focused

408 ■ Chapter 13: Qualitative Data Analysis FIGURE 13-5 How Text Materials Are Displayed in NUD*IST film director illustration uses the Windows version Now let’s create a concept code: “homosex.” of NVivo, so you can note the difference and simi- This will stand for references to male homosexu- larities in the two platforms. ality. Figure 13-6 shows what the creation of a concept code looks like. To see the document, select its name in the “Document Explorer” window and click “Browse.” As we create codes for our concepts, we can The text window can be resized and moved around use them to code the text materials. Figure 13-7 the screen to suit your taste. illustrates how this is done. In the document browser, you can see that verse 20:13 has been Note the set of buttons in the upper right selected (indicated by the box outline around this corner of the illustration. These allow you to select verse). Having done that, we click the button la- portions of the text for purposes of editing, coding, beled “Add Coding” (not shown in this illustration). and other operations.

Computer Programs for Qualitative Data ■ 409 FIGURE 13-6 Creating the Code “homosex” This prompts the computer to ask us to identify passages coded “homosex.” This would allow us the appropriate code. The easiest way to respond is to see them all at once, looking for similarities and to click the “Browse” button, which presents you differences. with a list of the current codes. In this example, I selected “homosex” and entered the code ID (100). Figure 13-8 shows how NUD*IST would bring together the passages referring to male homo- As text materials are coded, the program can sexuality. To do this, all you do is select the code then be used for purposes of analysis. As a simple name in the “Node Explorer” window and click the example, we might want to pull together all the “Make Report” button.

410 ■ Chapter 13: Qualitative Data Analysis FIGURE 13-7 by women breaking into the male-dominated Coding a Text Passage world of film direction. To address this issue, she interviewed 30 women directors in depth. Having This simple example illustrates the possibilities compiled hours of recorded interviews, she turned opened up by a program designed specifically to NVivo as a vehicle for analysis. In the next sec- for qualitative data analysis. To get a little deeper tion, she directly describes her experiences with the into its use, let’s shift to a different research ex- ongoing process of qualitative data analysis. ample. Sandrine Zerbib is a French sociologist inter- ested in understanding the special difficulties faced

Computer Programs for Qualitative Data ■ 411 FIGURE 13-8 Reporting on “homosex” Using NVivo to Understand interview into NVivo. Because you will have tran- Women Film Directors, by scribed or at least read your interviews beforehand, Sandrine Zerbib you may be able to select the interview you think will be most fruitful. You should trust yourself, be- Most software for qualitative analysis allows cause you are becoming an expert in what you are researchers to simultaneously analyze several currently studying and also because comparing and interviews from different interviewers. However, I contrasting interviews should help you get a sense find it more efficient to start by importing only one of how accurate your analysis is.

412 ■ Chapter 13: Qualitative Data Analysis After having completed about 30 interviews FIGURE 13-9 with women filmmakers, I had a sense of what the Text of Interview with Joy main themes were, because they kept coming up in each interview. Nevertheless, I needed a tool for gender inequalities. According to her, having talent synthesizing those pages and pages of interviews. is not enough in Hollywood if there is a bias against I chose to start with my interview with Joy. I had women. I decided to add two nodes to this quote, made a note to myself to use her interview as a “DGA,” which I needed to create, and “discrimina- starting point. An older film director, she seemed to tion,” which I had already created. have strong points she wanted to get across. In Figure 13-12, I have attempted to transform In Figure 13-9, my interview with Joy has been some free nodes into index trees. The software is imported as a “text only” file. (Only part of the file flexible enough for me to move nodes, rename is visible in the window.) them, or see what quotes are under each node. You can attach a different node to a quote you At this point you are ready to enjoy the coding have wrongly coded. It is preferable to start with process. You can simply highlight words, sentences, free nodes before you build a hierarchy of codes or sections and add nodes (i.e., codes) to it. The (or tree), because it takes time and patience to un- first step is to create “free nodes,” that is, nodes derstand how categories are linked to one another. independent of one another. How much text you Coding other interviews should help you organize should highlight per code is a decision you will your coding system. have to make. However, keep in mind that you will have to use those quotes in the writing part Figure 13-13 illustrates my decision to import of your research. You will need to be convincing. two more interviews, Berta’s and Queena’s. I could You also want to deconstruct the whole interview. browse all three interviews on the same screen. Try to not leave anything out. It is easier to forgo using a quote because you have found a better one later than to have nothing to use because you were not consistent enough in your dissection of the interview. When you create a node, you first want to use wide categories that would be more inclusive of other potential quotes. But you also want to be specific enough for your coding system to have validity. In Figure 13-10, for instance, I have cre- ated the free node “past” because my interviewee referred to the past as being extremely challenging for women who wanted to be film directors. There were very few women directors back then, many fewer than today. I decided to add a definition of this node so that I could remember why I used “past” as a node. I also anticipated having another free node called “today.” Then I could move the “change” node to the index tree root and create “past” and “today” as subnodes under “change.” In Figure 13-11, I have highlighted a passage that deals with several things. Joy talks about the Directors Guild of America (DGA, or the directors’ union) and more specifically about the efforts of its president. She also expresses her feelings toward

The Qualitative Analysis of Quantitative Data ■ 413 FIGURE 13-10 Creating the Code “past” Because it was still early in the analysis process, the number assigned to each text unit, which I chose to analyze these two new interviews one allowed me to go back and see a quotation in the by one. It was now starting to make sense; I was context of the whole document. starting to see patterns. NVivo let me keep records of number of occurrences each node was attached This procedure is only one of the many capa- to a quote, not only in Joy’s interview but now also bilities of this program. You may want to spend in Berta’s and Queena’s. With several nodes often some time learning about this software before com- attached to a single quote, the qualitative analysis mitting to it. What seems to be an efficient tool for allowed me to find out which nodes were more me may not be for you. There is plenty qualitative likely to overlap with one another. research analysis software in the market; try to find out what works for you. One of my first observations was that the term sabotage was used fairly often by Joy and Queena. The Qualitative Analysis I decided to run a report that would synthesize all of Quantitative Data the quotes that I attached to the node “sabotage.” Figure 13-14 shows the first page of the report cre- Although it’s important and appropriate to dis- ated by NVivo. The program searched for all quotes tinguish between qualitative and quantitative under “sabotage,” which is a subnode of “discrimi- research, often to the point of discussing them nation,” for all online documents. It also provided

414 ■ Chapter 13: Qualitative Data Analysis FIGURE 13-11 Coding a Passage in the Interview separately, I don’t want you to get the idea that pattern verbally. Here’s a case where a picture is they’re incompatible or competing. Unless you can truly worth a thousand words. operate in both modes, you’ll limit your potential as a social researcher. Evaluating the Quality of Qualitative Research In Chapter 14, I’ll indicate some ways in which quantitative analyses can strengthen qualitative As you’ve seen in earlier chapters, there are studies. Conversely, in this chapter I’ll include an often clear guidelines for evaluating the qual- example of how quantitative data demand qualita- ity of quantitative research. In the case of survey tive assessment. research, for example, we can note the size of the sample, the manner in which it was selected, and Figure 13-15 presents FBI data on the hour the completion rate achieved. The questionnaire and day of crimes committed in the United States items are standardized and open to scrutiny. And (Maltz 1998: 401). These data are often presented as you’ll see in Chapters 14 and 16, researchers can in a tabular form, but notice how clearly the pat- use statistical tests to assess quantitative research terns of crime appear in this three-dimensional findings. graph. The picture itself conveys the meaning of the statistical data. Summarizing it in the form of equations—while possibly useful for certain pur- poses—adds nothing to the clarity of the picture. Indeed, there hardly seems a need to describe the

FIGURE 13-12 Creating an Index Tree FIGURE 13-13 Adding Two More Cases to the Analysis

416 ■ Chapter 13: Qualitative Data Analysis FIGURE 13-14 Judging the quality of qualitative research is Analyzing the Node “sabotage” more elusive, though no less important. Because there are many different forms of qualitative re- FIGURE 13-15 search, we’ll examine some fairly general guide- Number of One-on-One Homicides by Age of Victim and Age lines you can use to distinguish first-rate qualitative of Offender, Raw Data investigations from those not so well done. Source:Michael D.Maltz,“Visualizing Homocide:A Research Note,” Journal of In Chapter 5, we looked at two aspects of Quantitative Criminology 15,no.4 (1998):401. measurement quality: validity and reliability. That’s a reasonable way to start our look at assessing qualitative research. Validity, you’ll recall, involves the question of whether you’re measuring what you say you’re measuring. Remember, most of the things social scientists measure are products of human thought and agreement, not things that exist independently of human judgment. Prejudice, for example, isn’t real the way age or weight are. Nonetheless, we’ve all observed behaviors and orientations that we’ve gathered under the umbrella concept of “preju- dice.” To a degree, we mean the same general thing

Evaluating the Quality of Qualitative Research ■ 417 when we use the term, and we also have a lot of of assessing the consistency of both what was ob- differences in that regard. served and the process by which it was observed. When you design a survey questionnaire to Follow-up works by the same authors laid out measure prejudice, it’s important to assess the extent several ways in which qualitative research could be to which the questions asked and answers received assessed. Building on this foundation, several other actually reflect what we can agree to mean by the researchers offered somewhat modified schemes term. The same logic applies in qualitative research for both assessing qualitative research and increas- projects such as field observations or historical ing its quality. A more recent effort, by Britain’s studies. If field researchers characterize a subject of National Centre for Social Research, sought to observation as “prejudiced,” you should examine assist cabinet-level officials in assessing qualita- their basis for saying that. Qualitative researchers, tive research projects that evaluated government more than quantitative researchers, pay special programs. Although the study focused on the use attention to understanding life as the participants of qualitative methods for purposes of evaluation see it, so you may find the researchers in this case research, the 18 questions that organized such as- reporting that those who knew the subject in ques- sessments can be applied to most forms of qualita- tion also mentioned that he or she was prejudiced. tive research: Some qualitative researchers prefer to use the 1. How credible are the findings? term credibility in the place of validity in this context. This is done as a caution against the older, positivis- 2. How has knowledge or understanding been tic view that social concepts represent real phe- extended by the research? nomena that exist objectively and independently of human thought. Be warned, however, that some 3. How well does the evaluation address its origi- researchers use the term with other meanings that nal aims and purpose? fall quite far from that of validity. Also, in this text- book, my use of the term validity explicitly denies 4. How well is the scope for drawing wider infer- objective reality for the concepts we use and study. ence explained? Reliability is also a reasonable criterion of qual- 5. How clear is the basis of evaluative appraisal? ity with regard to qualitative research, though it needs to be applied appropriately. Recall this is a 6. How defensible is the research design? question of whether a measurement or observation technique would yield the same data if it were pos- 7. How well defended are the same design/target sible to measure or observe the same thing several selection of cases/documents? times independently. In the case of categorizing raw data, such as data that in-depth interviews or 8. How well is the eventual sample composition even the open-ended answers to survey questions and coverage described? might produce, we can ask more than one person to undertake the coding or categorizing process 9. How well was the data collection carried out? independently and see if they all produce the same results. In most aspects of social research, however, 10. How well has the approach to, and formulation the concept of reliability is more elusive, because of, analysis been conveyed? (1) what we are observing may be constantly changing and/or (2) the act of measuring (for 11. How well are the contexts of data sources example, asking a question) may affect the person retained and portrayed? being studied. Still, the basic concept of reliability, which some qualitative researchers prefer to call 12. How well has diversity of perspective and dependability, is meaningful for qualitative research. content been explored? Yvonna Lincoln and Egon Guba (1985), for ex- ample, proposed an “inquiry audit” for the purpose 13. How well has detail, depth and complexity (i.e., richness) of the data been conveyed? 14. How clear are the links between data, interpre- tation and conclusions—i.e., how well can the route to any conclusions be seen? 15. How clear and coherent is the reporting? 16. How clear are the assumptions/theoretical per- spectives/values that have shaped the form and output of the evaluation?

418 ■ Chapter 13: Qualitative Data Analysis 17. What evidence is there of attention to ethical Sometimes, you may even need to suppress issues? details that would let outsiders figure out who you are talking about. Thus, it may be appropriate to 18. How adequately has the research process been speak about interviewing “a church leader” rather documented? than “the head deacon.” And you may need to (Spencer et al. 2003: 22–28) suppress or alter age, race, or gender references if any would give away a subject’s identity. The key The attempt to settle on criteria for evaluat- principle is to respect the privacy of those we study. ing qualitative social research is far from over. For example, some researchers are wary of the British MAIN POINTS effort just delineated: They express concern about the implications of a government body specify- Introduction ing research criteria and suggest that the list grows out of philosophical and political orientations that • Qualitative analysis is the nonnumerical examina- have not been made clear (J. Smith and Hodkinson 2005). tion and interpretation of observations. Ethics and Qualitative Linking Theory and Analysis Data Analysis • Qualitative analysis involves a continual interplay At least two ethical issues cause special concern in the analysis and reporting of qualitative research. between theory and analysis. In analyzing qualita- First, because it calls so directly on subjective judg- tive data, we seek to discover patterns such as ments, researchers face an obvious risk of seeing changes over time or possible causal links among what they are looking for or want to find. The variables. risk is increased in the case of participatory action research or other projects involving an element of • Examples of approaches to the discovery and ex- social justice. Researcher bias is hardly inevitable, however. Experienced qualitative analysts avoid planation of such patterns are Grounded Theory this pitfall through a deliberate awareness of their Method (GTM), semiotics, and conversation own values and preferences as well as adherence analysis. to established techniques for data collection and analysis. And as an additional protection, the peer Qualitative Data Processing review inherent in the scientific research environ- ment encourages colleagues to point out any fail- • The processing of qualitative data is as much art ings in this regard. as science. Three key tools for preparing data Second, protecting subjects’ privacy becomes a for analysis are coding, memoing, and concept particularly important issue in qualitative research. mapping. The qualitative researcher will often analyze and report data collected from specific, identifiable • In contrast to the standardized units used in cod- individuals. Earlier, I indicated the importance of not revealing what we learn about subjects, though ing for statistical analyses, the units to be coded in I mostly discussed it in the context of data collec- qualitative analyses may vary within a document. tion. When writing up the results of your analyses, Although codes may be derived from the theory you will often have to actively conceal identities. being explored, more often researchers use open Individuals, organizations, and communities are coding, in which codes are suggested by the given pseudonyms to conceal their identities. researchers’ examination and questioning of the data. • Memoing is appropriate at several stages of data processing to capture code meanings, theoretical ideas, preliminary conclusions, and other thoughts that will be useful during analysis. • Concept mapping uses diagrams to explore rela- tionships in the data graphically. Computer Programs for Qualitative Data • Many computer programs, such as NUD*IST, are designed specifically to assist researchers in the analysis of qualitative data. In addition, research-

Review Questions and Exercises ■ 419 ers can take advantage of the capabilities of you write this portion of the proposal, of course, you common software tools such as word processors, can’t say anything about the conclusions you’ll draw. database programs, and spreadsheets. However, you can describe your initial plans for the analysis. I say “initial” plans because you may change The Qualitative Analysis directions somewhat as the data accumulate and pat- of Quantitative Data terns begin to emerge. In some cases, your analysis will begin as observations are being made and/or other • Although qualitative and quantitative methods of data being gathered, or you may plan to complete the data-collection phase before starting your data analysis may appear incompatible or in competi- analysis. tion, research often demands that both kinds be used in the same project. This the place to indicate whether you plan to employ a particular method of analysis, such as Evaluating the Quality grounded theory, semiotics, or conversational analysis. of Qualitative Research If you’re planning to use one of the computer pro- grams used for qualitative data analysis, mention that • Validity (credibility) and reliability (dependability) here as well. are reasonable criteria for evaluating qualitative REVIEW QUESTIONS AND EXERCISES research. 1. Review Goffman’s examination of gender in Ethics and Qualitative Data Analysis advertising, then collect and analyze a set of ad- vertising photos, from magazines or newspapers, • The subjective element in qualitative data analysis that allow you to explore the relationship between gender and status. provides an added challenge to avoiding bias in the interpretation of data. 2. Review the discussion of homosexuality in the Book of Leviticus and suggest ways that the • Since the qualitative data analyst will know the examination might be structured as a cross-case analysis. identity of subjects, protecting their privacy re- quires special care. 3. Imagine you were conducting a cross-case analysis of revolutionary documents such as the Declara- KEY TERMS tion of Independence and the Declaration of the Rights of Man and of the Citizen (from the French The following terms are defined in context in the Revolution). Identify the key concepts you might chapter and at the bottom of the page where the term code in the following sentence: is introduced, as well as in the comprehensive glossary at the back of the book. When in the Course of human events, it be- comes necessary for one people to dissolve axial coding memoing the political bands which have connected case-oriented analysis open coding them with another, and to assume among the concept mapping qualitative analysis Powers of the earth, the separate and equal constant comparative selective coding station to which the Laws of Nature and of method semiotics Nature’s God entitle them, a decent respect to conversation analysis (CA) variable-oriented the opinions of mankind requires that they cross-case analysis analysis should declare the causes which impel them to Grounded Theory the separation. Method (GTM) 4. Write one code note and one theoretical note for PROPOSING SOCIAL RESEARCH: Exercise 3. QUALITATIVE DATA ANALYSIS 5. Using the library, InfoTrac College Edition, or the In this chapter, we’ve seen some of the qualitative web, find a research report using conversation data analysis approaches that social researchers can analysis. Summarize the main conclusions in your use. Since you won’t have analyzed your data when own words.

420 ■ Chapter 13: Qualitative Data Analysis SPSS EXERCISES 2. As you review, take advantage of the Cengag- eNOW personalized study plan, based on your See the booklet that accompanies your text for ex- quiz results. Use this study plan with its interac- ercises using SPSS (Statistical Package for the Social tive exercises and other resources to master the Sciences). There are exercises offered for each chapter, material. and you’ll also find a detailed primer on using SPSS. 3. When you’re finished with your review, take the Online Study Resources posttest to confirm that you’re ready to move on to the next chapter. If your book came with an access code card, visit www.cengage.com/login to register. To purchase WEBSITE FOR THE PRACTICE access, please visit www.ichapters.com. OF SOCIAL RESEARCH 12TH EDITION 1. Before you do your final review of the chapter, Go to your book’s website at www.cengage.com/ take the CengageNOW pretest to help identify the sociology/babbie for tools to aid you in studying for areas on which you should concentrate. You’ll your exams. You’ll find Tutorial Quizzes with feedback, find information on this online tool, as well as Internet Exercises, Flash Cards, Glossaries, and Essay Quiz- instructions on how to access all of its great re- zes, as well as InfoTrac College Edition search terms, sug- sources, in the front of the book. gestions for additional reading, Web Links, and primers for using data-analysis software such as SPSS.

CHAPTER FOURTEEN Quantitative Data Analysis CHAPTER OVERVIEW Introduction Handling “Don’t Knows” Numerical Descriptions in Often,social data are converted Quantification of Data Qualitative Research to numerical form for statistical Developing Code analyses.In this chapter,we’ll begin Categories Bivariate Analysis with the process of quantifying Codebook Construction Percentaging a Table data,then turn to analysis. Data Entry Constructing and Reading Quantitative analysis may be Bivariate Tables descriptive or explanatory;it may Univariate Analysis involve,one,two,or several Distributions Introduction to Multivariate variables.We begin our examination Central Tendency Analysis of how quantitative analyses are Dispersion done with some simple but Continuous and Discrete Sociological Diagnostics powerful ways of manipulating Variables data in order to attain research Detail versus Manageability Ethics and Quantitative Data conclusions. Analysis Subgroup Comparisons “Collapsing” Response Categories CengageNOW for Sociology Use this online tool to help you make the grade on your next exam. After reading this chapter, go to “Online Study Resources” at the end of the chapter for instructions on how to benefit from CengageNOW.

422 ■ Chapter 14: Quantitative Data Analysis Introduction Whereas the writing and check marks on a ques- tionnaire are qualitative in nature, a scribbled age is In Chapter 13, we saw some of the logic and easily converted to quantitative data. techniques by which social researchers analyze the qualitative data they’ve collected. This chapter ex- Other data are also easily quantified: trans- amines quantitative analysis, or the techniques by forming male and female into “1” and “2” is hardly which researchers convert data to numerical forms rocket science. Researchers can also easily assign and subject them to statistical analyses. numerical representations to such variables as reli- gious affiliation, political party, and region of the country. To begin we’ll look at quantification—the process of converting data to a numerical format. This Some data are more challenging, however. If a involves converting social science data into a ma- survey respondent tells you that he or she thinks chine-readable form—a form that can be read and the biggest problem facing Woodbury, Vermont, manipulated by computers and similar machines today is “the disintegrating ozone layer,” the com- used in quantitative analysis. puter can’t process that response numerically. You must translate by coding the responses. We’ve al- The rest of the chapter will present the logic ready discussed coding in connection with content and some of the techniques of quantitative data analysis (Chapter 11) and again in connection with analysis—starting with the simplest case, univariate qualitative data analysis (Chapter 13). Now we analysis, which involves one variable, then discuss- look at coding specifically for quantitative analysis. ing bivariate analysis, which involves two variables. We’ll end with a brief introduction to multivariate To conduct a quantitative analysis, research- analysis, or the examination of several variables ers often must engage in a coding process after the simultaneously, such as age, education, and prejudice. data have been collected. For example, open-ended questionnaire items result in nonnumerical re- Before we can do any sort of analysis, we need sponses, which need to be coded before analysis. As to quantify our data. Let’s turn now to the basic with content analysis, the task is to reduce a wide steps involved in converting data into machine- variety of idiosyncratic items of information to a readable forms amenable to computer processing more limited set of attributes composing a variable. and analysis. Suppose, for example, that a survey researcher asks respondents, “What is your occupation?” The Quantification of Data responses to such a question will vary considerably. Although he or she can assign a separate numerical Today, quantitative analysis is almost always code to each reported occupation, this procedure handled by computer programs such as SPSS will not facilitate analysis, which typically depends and MicroCase. For those programs to work their on several subjects having the same attribute. magic, they must be able to read the data you’ve collected in your research. If you’ve conducted a The variable occupation has many preestablished survey, for example, some of your data are inher- coding schemes. One such scheme distinguishes ently numerical: age or income, for example. professional and managerial occupations, clerical occupations, semiskilled occupations, and so forth. quantitative analysis The numerical representa- Another scheme distinguishes different sectors of tion and manipulation of observations for the pur- the economy: manufacturing, health, education, pose of describing and explaining the phenomena commerce, and so forth. Still others combine both. that those observations reflect. Using an established coding scheme gives you the advantage of being able to compare your research results with those of other studies. (See, for in- stance, the Standard Occupational Classification [SOC] at the Bureau of Labor Statistics website,

Quantification of Data ■ 423 via the link on this book’s website: http://www TABLE 14-1 .cengage.com/sociology/babbie.) Student Responses That Can Be Coded“Financial Concerns” The occupational coding scheme you choose Financial Concerns should be appropriate to the theoretical concepts being examined in your study. For some studies, Tuition is too high X coding all occupations as either white-collar or Not enough parking spaces blue-collar might be sufficient. For others, self- Faculty don’t know what they are doing X employed and not self-employed might do. A peace Advisors are never available X researcher might wish to know only whether the Not enough classes offered occupation depended on the defense establishment Cockroaches in the dorms or not. Too many requirements Cafeteria food is infected Although you should tailor the coding scheme Books cost too much to meet particular requirements of the analysis, Not enough financial aid you should also keep one general guideline in mind. If the data are coded to maintain a great deal Not enough classes offered of detail, code categories can always be combined during an analysis that does not require such detail. Cockroaches in the dorms If the data are coded into relatively few, gross categories, however, there’s no way during analysis Too many requirements to recreate the original detail. To keep your options open, it’s a good idea to code your data in greater Cafeteria food is infected detail than you plan to use in the analysis. Books cost too much Developing Code Categories Not enough financial aid There are two basic approaches to the coding process. First, you can begin with a relatively well- Take a minute to review these responses and see developed coding scheme. You may choose to do whether you can identify some categories repre- this because it serves your research purpose. Thus, sented. Realize that there is no right answer; you as suggested previously, the peace researcher might could generate several coding schemes from these code occupations in terms of their relationship to answers. the defense establishment. Or, you may want to use an existing coding scheme because it allows Let’s start with the first response: “Tuition is too you to compare your findings with those of previ- high.” What general areas of concern does that re- ous research. sponse reflect? One obvious possibility is “Financial Concerns.” Are there other responses that would fit Second, you can generate codes from your into that category? Table 14-1 shows which of the data, as discussed in Chapter 13. Let’s say we’ve questionnaire responses could do just that. asked students in a self-administered campus survey to state what they believe is the biggest In more general terms, the first answer can problem facing their college today. Here are a few also be seen as reflecting nonacademic concerns. of the answers they might have written in. This categorization would be relevant if your re- search interest included the distinction between Tuition is too high academic and nonacademic concerns. If that were the case, the responses might be coded as shown in Not enough parking spaces Table 14-2. Faculty don’t know what they are doing Notice that I didn’t code the response “Books cost too much” in Table 14-2, because this con- Advisors are never available cern could be seen as representing both of the categories. Books are part of the academic program,