326 ■ Chapter 11: Paradigms, Methods, and Ethics of Qualitative Field Research One issue I’m particularly interested in (Babbie students said they would be pleased if someone 1985) is the nature of responsibility for public mat- cleaned up a public place. It seemed likely that ters: Who’s responsible for making the things that everyone else would be pleased, too, if we asked we share work? Who’s responsible for keeping them how they would react to someone’s cleaning public spaces—parks, malls, buildings, and so on— up litter in a public place or otherwise taking per- clean? Who’s responsible for seeing that broken sonal responsibility for fixing some social problem. street signs get fixed? Or, if a strong wind knocks over garbage cans and rolls them around the street, To settle the issue, I suggested that my students who’s responsible for getting them out of the road? start fixing the public problems they came across in the course of their everyday activities. As they On the surface, the answer to these questions did so, I asked them to note the answers to two is pretty clear. We have formal and informal agree- questions: ments in our society that assign responsibility for these activities. Government custodians are respon- 1. How did they feel while they were fixing a sible for keeping public places clean. T ransportation public problem they had not been assigned department employees are responsible for the responsibility for? street signs, and perhaps the police are responsible for the garbage cans rolling around on a windy day. 2. How did others around them react? And when these responsibilities are not fulfilled, we tend to look for someone to blame. My students picked up litter, fixed street signs, put knocked-over traffic cones back in What fascinates me is the extent to which the place, cleaned and decorated communal lounges assignment of responsibility for public things to in their dorms, trimmed trees that blocked visi- specific individuals not only relieves others of the bility at intersections, repaired public playground responsibility but actually prohibits them from equipment, cleaned public restrooms, and took taking responsibility. It’s my notion that it has be- care of a hundred other public problems that come unacceptable for someone like you or me to weren’t “their responsibility.” take personal responsibility for public matters that haven’t been assigned to us. Most reported feeling very uncomfortable doing whatever they did. They felt foolish, goody- Let me illustrate what I mean. If you were goody, conspicuous, and all the other feelings that walking through a public park and you threw keep us from performing these activities routinely. down a bunch of trash, you’d discover that your In almost every case, their personal feelings of dis- action was unacceptable to those around you. comfort were increased by the reactions of those People would glare at you, grumble to each other; around them. One student was removing a dam- perhaps someone would say something to you aged and long-unused newspaper box from the about it. Whatever the form, you’d be subjected to bus stop, where it had been a problem for months, definite, negative sanctions for littering. Now here’s when the police arrived, having been summoned the irony. If you were walking through that same by a neighbor. Another student decided to clean park, came across a bunch of trash that someone out a clogged storm drain on his street and found else had dropped, and cleaned it up, it’s likely that himself being yelled at by a neighbor who insisted your action would also be unacceptable to those that the mess should be left for the street clean- around you. You’d probably face negative sanctions ers. Everyone who picked up litter was sneered at, for cleaning it up. laughed at, and generally put down. One young man was picking up litter scattered around a trash When I first began discussing this pattern can when a passerby sneered, “Clumsy!” It became with students, most felt the notion was absurd. clear to us that there are only three acceptable ex- A lthough we would be negatively sanctioned for planations for picking up litter in a public place: littering, cleaning up a public place would obvi- ously bring positive sanctions: People would be 1. You did it and got caught—somebody forced pleased with us for doing it. Certainly, all my you to clean up your mess.
Special Considerations in Qualitative Field Research ■ 327 2. You did it and felt guilty. One of the challenging dimensions of the work was that it brought us face to face with a reality 3. You’re stealing litter. that demanded that we act with or on behalf of the teenagers. The work of relationship build- In the normal course of things, it’s simply not ac- ing is time consuming and energy sapping— ceptable for people to take responsibility for public many research approaches do not require the things. formation of “caring relationships” with the researched. Building relationships between old Clearly, we could not have discovered the and young, from different class backgrounds nature and strength of agreements about taking and diverse life experiences require a sus- personal responsibility for public things except tained and shared commitment from all. through field research. Social norms suggest that taking responsibility is a good thing, sometimes (2009: 75) referred to as good citizenship. Asking people what they thought about taking responsibility Two important aspects of qualitative research would have produced a solid consensus that it was need to be stressed. First, a wide range of studies good. Only going out into life, doing it, and watch- fall under the umbrella “qualitative field research.” ing what happened gave us an accurate picture. As we’ll see in this chapter, various epistemologies within different paradigms have quite different As an interesting footnote to this story, my approaches to basic questions such as “What are students and I found that whenever people could data?” “How should we collect data?” and “How get past their initial reactions and discover that should we analyze data?” Second, we should re- the students were simply taking responsibility for member that the questions we want to answer in fixing things for the sake of having them work, our research determine the types of methods we the passersby tended to assist. Although there are need to use. A question such as “How do women some very strong agreements making it “unsafe” construct their everyday lives in order to perform to take responsibility for public things, the willing- their roles as mothers, partners, and breadwin- ness of one person to rise above those agreements ners?” could be addressed by in-depth interviews seemed to make it safe for others to do so, and and direct observations. The assessment of advertis- they did. ing campaigns might profit from focus group dis- cussions. In most cases, we’ll find that researchers Field research is not to be confused with jour- have alternate methods to choose from. nalism. Social scientists and journalists may use similar techniques, but they have quite a different In summary, then, field research offers the ad- relationship to data. For instance, individual inter- vantage of probing social life in its natural habitat. viewing is a common technique in journalism and Although some things can be studied adequately sociology; nevertheless, sociologists are not simply through questionnaires or in the laboratory, oth- concerned with reporting about a subject’s attitude, ers cannot. And direct observation in the field lets belief, or experience. A sociologist’s goal is to treat researchers observe subtle communications and an interview as data that need to be analyzed to other events that might not be anticipated or mea- understand social life more generally. sured otherwise. Byrne, Canavan, and Millar (2009) suggest this Special Considerations distinction can go even deeper. The voice-centered in Qualitative Field Research relational (VCR) method focuses on who is speak- ing in communications and who is listening, taking There are specific things to take into account accounts of the difference between the two actors in every research method, and qualitative field and the impact of those differences. Often, the research is no exception. When you use field listener is the researcher. This approach shows up during interviews and during the analysis of tran- scripts. The authors say about their study that dealt with Irish teenagers:
328 ■ Chapter 11: Paradigms, Methods, and Ethics of Qualitative Field Research research methods, you’re confronted with deci- This type of research introduces an ethical sions about the role you’ll play as an observer and issue, one on which social researchers themselves your relations with the people you’re observing. are divided. Is it ethical to deceive the people Let’s examine some of the issues involved in these you’re studying in the hope that they will confide decisions. in you as they will not confide in an identified researcher? Do the potential benefits to be gained The Various Roles of the Observer from the research offset such considerations? Although many professional associations have ad- In field research, observers can play any of several dressed this issue, the norms to be followed remain roles, including participating in what they want somewhat ambiguous when applied to specific to observe (this was the situation of the students situations. who fixed public things). In this chapter, I’ve used the term field research rather than the frequently Related to this ethical consideration is a used term participant observation, because field scientific one. No researcher deceives his or her researchers need not always participate in what subjects solely for the purpose of deception. Rather, they’re studying, though they usually do study it it’s done in the belief that the data will be more directly at the scene of the action. As Catherine valid and reliable, that the subjects will be more Marshall and Gretchen Rossman point out: natural and honest if they do not know the re- searcher is doing a research project. If the people The researcher may plan a role that entails being studied know they’re being studied, they varying degrees of “participantness”—that is, might modify their behavior in a variety of ways. the degree of actual participation in daily life. This is known as the problem of reactivity. At one extreme is the full participant, who goes about ordinary life in a role or set of roles First, they might expel the researcher. Second, constructed in the setting. At the other extreme they might modify their speech and behavior to is the complete observer, who engages not at appear more “respectable” than would otherwise all in social interaction and may even shun in- be the case. Third, the social process itself might be volvement in the world being studied. And, of radically changed. Students making plans to burn course, all possible complementary mixes along down the university administration building, for the continuum are available to the researcher. example, might give up the plan altogether once they learn that one of their group is a social scien- (1995: 60) tist conducting a research project. The complete participant, in this sense, may be On the other side of the coin, if you’re a a genuine participant in what he or she is studying complete participant, you may affect what you’re (for example, a participant in a campus demonstra- studying. Suppose, for example, that you’re asked tion) or may pretend to be a genuine participant. for your ideas about what the group should do In any event, whenever you act as the complete next. No matter what you say, you will affect participant, you must let people see you only as the process in some fashion. If the group follows a participant, not as a researcher. For instance, if your suggestion, your influence on the process is you’re using this technique to study a group made obvious. If the group decides not to follow your up of uneducated and inarticulate people, it would suggestion, the process whereby the suggestion not be appropriate for you to talk and act like a is rejected may affect what happens next. Finally, university professor or student. if you indicate that you just don’t know what should be done next, you may be adding to a gen- reactivity The problem that the subjects of social eral feeling of uncertainty and indecisiveness in research may react to the fact of being studied, thus the group. altering their behavior from what it would have been normally. Ultimately, anything the participant-observer does or does not do will have some effect on what’s being observed; it’s simply inevitable. More seriously,
Special Considerations in Qualitative Field Research ■ 329 there is no complete protection against this ef- Marshall and Rossman (1995: 60–61) also note fect, though sensitivity to the issue may provide that the researcher can vary the amount of time a partial protection. (This influence, called the spent in the setting being observed: You can be a Hawthorne effect, was discussed more fully in full-time presence on the scene or just show up Chapter 9.) now and then. Moreover, you can focus your at- tention on a limited aspect of the social setting or Because of these several considerations, ethi- seek to observe all of it—framing an appropriate cal and scientific, the field researcher frequently role to match your aims. chooses a different role from that of complete par- ticipant. You could participate fully with the group When Jeffrey Kidder set out to study the cul- under study but make it clear that you were also ture of bike messengers in New York City, he found undertaking research. As a member of the volley- it appropriate to identify his research role to some ball team, for example, you might use your posi- of those he observed but not others (2005: 349): tion to launch a study in the sociology of sports, letting your teammates know what you’re doing. While I did have an academic motivation in There are dangers in this role also, however. The working as a messenger, it should be made people being studied may shift much of their atten- clear that my participation within the messen- tion to the research project rather than focusing on ger world was neither forced nor faked. To the the natural social process, making the process being contrary, my lifelong interest in bicycles and observed no longer typical. Or, conversely, you alternative transportation melded seamlessly yourself may come to identify too much with the with the messenger lifestyle. interests and viewpoints of the participants. You may begin to “go native” and lose much of your During the course of my fieldwork, most of scientific detachment. the messengers with whom I came in contact were unaware of my research; this was a mat- At the other extreme, the complete observer ter of necessity. In New York City, a messenger studies a social process without becoming a part crosses paths with hundreds of messengers a of it in any way. Quite possibly, because of the day. The numerous individuals that helped researcher’s unobtrusiveness, the subjects of study form my understandings of messenger style might not realize they’re being studied. Sitting at could not all be approached to sign consent a bus stop to observe jaywalking at a nearby in- forms. Messengers with whom I had reoccur- tersection is one example. Although the complete ring contact were informed of my sociological observer is less likely to affect what’s being studied interest. and less likely to “go native” than the complete participant, she or he is also less likely to develop a Different situations ultimately require different full appreciation of what’s being studied. Observa- roles for the researcher. Unfortunately, there are tions may be more sketchy and transitory. no clear guidelines for making this choice—you must rely on your understanding of the situation Fred Davis (1973) characterizes the extreme and your own good judgment. In making your roles that observers might play as “the Martian” decision, however, you must be guided by both and “the Convert.” The latter involves delving methodological and ethical considerations. Because more and more deeply into the phenomenon these often conflict, your decision will frequently under study, running the risk of “going native.” be difficult, and you may find sometimes that your We’ll examine this risk further in the next section. role limits your study. To appreciate the “Martian” approach, imagine Relations to Subjects that you were sent to observe some newfound life on Mars. Probably you would feel yourself inescap- Having introduced the different roles field re- ably separate from the Martians. Some social scien- searchers might play in connection with their tists adopt this degree of separation when observing observations, we now focus more specifically on cultures or social classes different from their own.
330 ■ Chapter 11: Paradigms, Methods, and Ethics of Qualitative Field Research how researchers may relate to the subjects of their than as objects of ridicule. The difficulty of adopt- study and to the subjects’ points of view. ing others’ views led William Shaffir and Robert S tebbins (1991: 1) to conclude that “fieldwork We’ve already noted the possibility of pretend- must certainly rank with the more disagreeable ing to occupy social statuses we don’t really occupy. a ctivities that humanity has fashioned for itself.” Consider now how you would think and feel in such a situation. There is, of course, a danger in adopting the points of view of the people you’re studying. Suppose you’ve decided to study a religious When you abandon your objectivity in favor of cult that has enrolled many people in your neigh- adopting such views, you lose the possibility of borhood. You might study the group by joining seeing and understanding the phenomenon within it or pretending to join it. Take a moment to ask frames of reference unavailable to your subjects. yourself what the difference is between “really” On the one hand, accepting the belief that the joining and “pretending” to join. The main differ- world will end Thursday night allows you to ap- ence is whether or not you actually take on the preciate aspects of that belief available only to beliefs, attitudes, and other points of view shared believers; stepping outside that view, however, by the “real” members. If the cult members be- makes it possible for you to consider some reasons lieve that Jesus will come next Thursday night to why people might adopt such a view. You may destroy the world and save the members of the discover that some did so as a consequence of per- cult, do you believe it or do you simply pretend to sonal trauma (such as unemployment or divorce), believe it? whereas others were brought into the fold through their participation in particular social networks Traditionally, social scientists have tended to (for example, all their Facebook friends joined emphasize the importance of “objectivity” in such the cult). Notice that the cult members might dis matters. In this example, that injunction would agree with those “objective” explanations, and you be to avoid getting swept up in the beliefs of the might not come up with them to the extent that group. Without denying the advantages associated you had operated legitimately within the group’s with such objectivity, social scientists today also views. recognize the benefits gained by immersing them- selves in the points of view they’re studying, what Anthropologists sometimes use the term emic John Lofland and his colleagues (2006: 70) refer to perspective in reference to taking on the point of as “selective competence” or “insider knowledge, view of those being studied. In contrast, the etic skill, or understanding.” Ultimately, you won’t be perspective maintains a distance from the native able to fully understand the thoughts and actions of point of view in the interest of achieving more the cult members unless you can adopt their points objectivity. of view as true—at least temporarily. To fully ap- preciate the phenomenon you’ve set out to study, The apparent dilemma here is that both of you need to believe that Jesus is coming Thursday these postures offer important advantages but also night. In some settings, this can also help you gain seem mutually exclusive. In fact, it’s possible to rapport with your subjects (see the discussion on assume both postures. Sometimes you can simply rapport later in this chapter). shift viewpoints at will. When appropriate, you can fully assume the beliefs of the cult; later, you Adopting an alien point of view is an uncom- can step outside those beliefs (more accurately, fortable prospect for most people. It can be hard you can step inside the viewpoints associated with enough merely to learn about views that seem social science). As you become more adept at this strange to you; you may sometimes find it hard kind of research, you may come to hold contra- just to tolerate certain views. But to take them on dictory viewpoints simultaneously, rather than as your own can be ten times worse. Robert Bellah switching back and forth. (1970, 1974) has offered the term symbolic realism to indicate the need for social researchers to treat During my study of trance channeling—in the beliefs they study as worthy of respect rather which people allow spirits to occupy their bodies
Special Considerations in Qualitative Field Research ■ 331 and speak through them—I found I could partici- Caroline Knowles (2006) raises a somewhat dif- pate fully in channeling sessions without becoming ferent issue with regard to the researcher’s relation- alienated from conventional social science. Rather ship to subjects in the field. In her interview study than “believing” in the reality of channeling, I found of British expatriates living in Hong Kong, she no- it possible to suspend beliefs in that realm: neither ticed that some were particularly difficult for her to believing it to be genuine (like most of the other deal with. When she found herself writing research participants) nor disbelieving it (like most scien- notes explaining why the project would not profit tists). Put differently, I was open to either p ossibility. from her interviewing them further, she forced her- Notice how this differs from our normal need to self to look more deeply into the interactional dy- “know” whether such things are legitimate or not. namics in question—with an emphasis on her side of the relationships. She examined why certain in- Social researchers often refer to the concerns formants made her uncomfortable and then pressed just discussed as a matter of reflexivity, in the sense through the discomfort to continue interviewing. of things acting on themselves. Thus, your own She found that factors such as the attitudes they ex- characteristics can affect what you see and how pressed, their rude interaction styles, and the nature you interpret it. The issue is broader than that, of the relationship she was establishing with them however, and applies to the subjects as well as to contributed to her reaction. In the end, she gained the researcher. Imagine yourself interviewing a a much deeper understanding of her subjects than homeless person (1) on the street, (2) in a home- would have been possible if she had limited herself less shelter, or (3) in a social welfare office. The to those who were cooperative and nice. research setting could affect the person’s responses. In other words, you might get different results de- Similarly, Broom, Hand, and Kelly (2009) ex- pending on where you conducted the interview. amined the impact of gender when conducting in- Moreover, you might act differently as a researcher depth interviews with cancer patients. Did it matter in those different settings. If you reflect on this whether patients were interviewed by someone issue, you’ll be able to identify other aspects of of the same or of the opposite sex? It did. Prostate the research encounter that complicate the task of cancer patients were more graphic in describing “simply observing what’s so.” their experiences to a male interviewer than to a woman. Similarly, a breast-cancer patient’s feelings The problem we’ve just been discussing could of disfigurement, for example, were expressed dif- be seen as psychological, occurring mostly inside ferently to male and female interviewers. Before the researchers’ or subjects’ heads. There is a cor- you decide that sex-matching is the best policy, responding problem at a social level, however. notice that a cancer patient’s overall experience When you become deeply involved in the lives includes same-sex and opposite-sex relations. As I of the people you’re studying, you’re likely to have said frequently in this book, the impact of the be moved by their personal problems and crises. observer, whether in experiments, surveys, or field Imagine, for example, that one of the cult mem- research often cannot be avoided, but we can be bers becomes ill and needs a ride to the hospital. conscious of it and take it into account in under- Should you provide transportation? Sure. Sup- standing what we have observed. pose someone wants to borrow money to buy a stereo. Should you loan it? Probably not. Sup- This discussion of the field researcher’s relation- pose they need the money for food? ships to subjects flies in the face of the usual view of “scientific objectivity.” Before concluding this section, There are no black-and-white rules for resolv- let’s take the issue one step further. ing situations such as these, but you should realize that you’ll need to deal with them regardless of In the conventional view of science, differ- whether or not you reveal that you’re a researcher. ences of power and status separate the researcher Such problems do not tend to arise in other types of from the subjects of research. When we discussed research—surveys and experiments, for example— experimental designs in Chapter 9, for example, it but they are part and parcel of field research. was obvious who was in charge: the experimenter,
332 ■ Chapter 11: Paradigms, Methods, and Ethics of Qualitative Field Research Research in Real Life Class Acts: Service and Inequality in Luxury Hotels by Rachel Sherman. © 2007 Rachel Sherman. Published by the University of California Press. Photograph © Walter B. McKenzie/Getty Images. Class Acts: Service and Inequality in Luxury Hotels What could seem like a clearer status relationship than between a guest in a luxury hotel and the room service and other staff who serve that guest’s needs? In fact, Rachel Sherman has found a far more com- plex process than you might imagine. She is particularly interested in how service workers balance their relationships with management and their relationships with guests. Unlike manufacturing workers, the hotel service staff must deal with both supervisors and consumers, even when the demands of the two conflict. In part, she discovered that service workers in hotels often receive more discretion regarding how to serve guests’needs than we might expect. This has a posi- tive impact on the worker’s sense of self as well as providing a good experience for guests. Sherman’s observations and conclusions came from months spent as a service worker in two luxury hotels. She made her research identity known to management and was able to move around through many of the different service jobs: making reservations, deliv- ering room-service meals, parking cars, carrying bags, housekeeping, and many other tasks that the guests in luxury hotels expect. Her im- mersion in the research allowed her access to data she would not have found out otherwise. who organized things and told the subjects what out to study what were originally called “primitive” to do. Often the experimenter was the only per- societies, there was no doubt that the anthropolo- son who even knew what the research was really gists knew best. Whereas the natives “believed” in about. Something similar might be said about witchcraft, for example, the anthropologists “knew” survey research. The person running the survey it wasn’t really true. Whereas the natives said some designs the questions, decides who will be selected of their rituals would appease the gods, the anthro- for questioning, and analyzes the data collected. pologists explained that the “real” functions of these rituals were the creation of social identity, the estab- Sociologists often look at these sorts of re- lishment of group solidarity, and so on. lationships as power or status relationships. In experimental and survey designs, the researcher The more social researchers have gone into clearly has more power and a higher status than the field to study their fellow humans face-to-face, the people being studied do. The researchers have however, the more they have become conscious of a special knowledge that the subjects don’t enjoy. these implicit assumptions about researcher supe- They’re not so crude as to say they’re superior to riority, and the more they have considered alterna- their subjects, but there is a sense in which that’s tives. As we turn now to the various paradigms of implicitly assumed. (Notice that there is a similar, field research, we’ll see some of the ways in which implicit assumption about the writers and readers that ongoing concern has worked itself out. See the of textbooks.) Research in Real Life feature “Class Acts: Service and Inequality in Luxury Hotels” above for an ex- In field research, such assumptions can be prob- ample of field research on status. lematic. When the early European anthropologists set
Some Qualitative Field Research Paradigms ■ 333 Some Qualitative Field social life on the streets, he needed to become Research Paradigms more of an insider. He made contact with “Doc,” his key informant, who appeared to be one of Although I’ve described field research as simply the street-gang leaders. Doc let Whyte enter his going where the action is and observing it, there world, and Whyte got to participate in the activi- are actually many different approaches to this ties of the people of Cornerville. His study of- research method. This section examines several fered something that surveys could not: a richly field research paradigms: naturalism, ethnometh- detailed picture of life among the Italian immi- odology, grounded theory, case studies and the grants of Cornerville. extended case method, institutional ethnography, and participatory action research. Although this An important feature of Whyte’s study is that survey won’t exhaust the variations on the method, he reported the reality of the people of Cornerville it should give you a broad appreciation of the on their terms. The naturalist approach is based on possibilities. telling “their” stories the way they “really are,” not the way the ethnographer understands “them.” It’s important to recognize that there are no The narratives collected by Whyte are taken at specific methods attached to these paradigms. You face value as the social “truth” of the Cornerville could do ethnomethodology or institutional eth- residents. nography by analyzing court hearings or conduct- ing group interviews, for example. The important Forty-five years later, David Snow and Leon distinctions of this section are epistemological, hav- Anderson (1987) conducted exploratory field re- ing to do with what data mean, regardless of how search into the lives of homeless people in Austin, they were collected. Texas. Their main task was to understand how the homeless construct and negotiate their identity Naturalism while knowing that the society they live in attaches a stigma to homelessness. Snow and Anderson Naturalism is an old tradition in qualitative believed that, to achieve this goal, the collection research. The earliest field researchers operated of data had to arise naturally. Like Whyte in Street on the positivist assumption that social reality Corner Society, they found some key informants was “out there,” ready to be naturally observed whom they followed in their everyday journeys, and reported by the researcher as it “really is” such as at their day-labor pickup sites or under ( Gubrium and Holstein 1997). This tradition bridges. Snow and Anderson chose to memorize started in the 1930s and 1940s at the University the conversations they participated in or the “talks” of Chicago’s sociology department, whose faculty that homeless people had with each other. At the and students fanned out across the city to observe end of the day, the two researchers debriefed and and understand local neighborhoods and com- wrote detailed field notes about all the “talks” they munities. The researchers of that era and their encountered. They also taped in-depth interviews research approach are now often referred to as with their key informants. the Chicago School. Snow and Anderson reported “hanging One of the earliest and best-known studies out” with homeless people over the course of that illustrates this research tradition is William 12 months for a total of 405 hours in 24 different Foote Whyte’s ethnography of Cornerville, an Italian American neighborhood, in his book Street naturalism An approach to field research based on Corner Society (1943). An ethnography is a study the assumption that an objective social reality exists that focuses on detailed and accurate description and can be observed and reported accurately. rather than explanation. Like other naturalists, Whyte believed that in order to learn fully about ethnography A report on social life that focuses on detailed and accurate description rather than explanation.
334 ■ Chapter 11: Paradigms, Methods, and Ethics of Qualitative Field Research settings. Out of these rich data, they identified Wall’s 2008 article, “Easier Said than Done: three related patterns in homeless people’s con- Writing an Autoethnography.” versations. First, the homeless showed an attempt to “distance” themselves from other homeless You can learn more about these variants on people, from the low-status job they currently ethnography by searching the web or your campus had, or from the Salvation Army they depended library. A later section of this chapter will examine on. Second, they “embraced” their street-life institutional ethnography, which links individuals and identity—their group membership or a certain be- organizations. lief about why they are homeless. Third, they told “fictive stories” that always contrasted with their In Chapter 8, we saw how the Internet is everyday life. For example, they would often say affecting survey research. Eric Anderson (2005) that they were making much more money than used the Internet to launch a qualitative, in-depth they really were, or even that they were “going to interviewing study of male cheerleaders. He began be rich.” by using a search engine to identify men whose online profiles contained an interest in cheerlead- While ethnographers seek to discover and un- ing. He contacted them via instant messaging and derstand the patterns of living among those they requested taped, telephone interviews. are studying, Mitchell Duneier (1999) has warned against what he calls the “ethnographic fallacy.” Anderson then used snowball sampling to in- This refers to an overgeneralization and oversim- crease the number of cheerleaders to study. This is plification of the patterns observed. Despite the just another example of the wide variety of venues existence of patterns within groups, there is also for ethnographic study. diversity, and you need to be wary of broad as- sertions suggesting that “the poor,” “the French,” Ethnomethodology or “cheerleaders” act or think in certain ways as though all members of the group do so. Ethnomethodology, which I introduced as a re- search paradigm in Chapter 3, is a unique approach Whereas this chapter aims at introducing to qualitative field research. It has its roots in the some of the different approaches available to philosophical tradition of phenomenology, which you in qualitative field research, please real- can explain why ethnomethodologists are skepti- ize that this discussion of ethnography merely cal about the way people report their experience of sketches some of the many avenues social re- reality (Gubrium and Holstein 1997). Alfred Schutz searchers have established. If you’re interested (1967, 1970), who introduced phenomenology, in this general approach, you might want to argued that reality was socially constructed rather explore the idea of virtual ethnography, which than being “out there” for us to observe. People uses ethnographic techniques for inquiry into describe their world not “as it is” but “as they make cyberspace. Or, in a different direction, auto- sense of it.” Thus, phenomenologists would argue ethnography intentionally assumes a personal that Whyte’s street-corner men were describing stance, breaking with the general proscription their gang life as it made sense to them. Their re- against the researcher getting involved at that ports, however, would not tell us how and why it level. Lest autoethnography seem a simple and/ made sense to them. For this reason, researchers or trivial undertaking, you might look at Sarah cannot rely on their subjects’ stories to depict social realities accurately. ethnomethodology An approach to the study of social life that focuses on the discovery of implicit, Whereas traditional ethnographers believe in usually unspoken assumptions and agreements; this immersing themselves in a particular culture and method often involves the intentional breaking of reporting their informants’ stories as if they rep- agreements as a way of revealing their existence. resented reality, phenomenologists see a need to “make sense” out of the informants’ perceptions of the world. Following in this tradition, some field researchers have felt the need to devise techniques
Some Qualitative Field Research Paradigms ■ 335 that reveal how people make sense of their every- have any trouble knowing what it means in casual day world. As we saw in Chapter 3, the sociologist interactions, as the unsuspecting subject revealed Harold Garfinkel suggested that researchers break in his final comment. the rules so that people’s taken-for-granted expecta- tions would become apparent. This is the technique Ethnomethodologists, then, are not simply that Garfinkel called ethnomethodology. interested in subjects’ perceptions of the world. In these cases, we could imagine that the subjects may Garfinkel became known for engaging his have thought that the experimenters were rude, students to perform a series of what he called stupid, or arrogant. The conversation itself, not the “breaching experiments” designed to break away informants, is the object of ethnomethodological from the ordinary (Heritage 1984). For instance, studies. In general, ethnomethodology focuses on Garfinkel (1967) asked his students to do a “con- the “underlying patterns” of interactions that regu- versation clarification experiment.” Students were late our everyday lives. told to engage in an ordinary conversation with an acquaintance or a friend and to ask for clarification Ethnomethodologists believe that researchers about any of this person’s statements. Through this who use a naturalistic analysis “[lose] the ability technique, they uncovered elements of conversa- to analyze the commonsense world and its cul- tion that are normally taken for granted. Here are ture if [they use] analytical tools and insights that two examples of what Garfinkel’s students reported are themselves part of the world or culture being (1967: 42): studied” (Gubrium and Holstein 1997: 43). D. L. W ieder provides an excellent example of how Case 1 different a naturalistic approach is from an ethno- The subject was telling the experimenter, a methodological approach (Gubrium and Holstein 1997). In his study Language and Social Reality: member of the subject’s car pool, about having The Case of Telling the Convict Code (1988), Wieder had a flat tire while going to work the previous started to approach convicts in a halfway house in day. a traditional ethnographic style: He was going to become an insider by befriending the inmates and I had a flat tire. by conducting participant observations. He took (E) What do you mean, you had a flat tire? careful notes and recorded interactions among She appeared momentarily stunned. Then inmates and between inmates and staff. His first she answered in a hostile way: “What do you concern was to describe the life of the convicts mean, ‘What do you mean?’ A flat tire is a of the halfway house the way it “really was” for flat tire. That is what I meant. Nothing special. them. Wieder’s observations allowed him to report What a crazy question.” on a “convict code” that he thought was the source of the deviant behavior expressed by the inmates Case 6 toward the staff. This code, which consisted of a se- The victim waved his hand cheerily. ries of rules such as “Don’t kiss ass,” “Don’t snitch,” (S) How are you? and “Don’t trust the staff,” was followed by the (E) How I am in regard of what? My inmates who interfered with the staff members’ at- tempts to help them make the transition between health, my finances, my school work, my peace prison and the community. of mind, my . . . ? It became obvious to Wieder that the code was (S) (Red in the face and suddenly out of more than an explanation for the convicts’ devi- control.) Look I was just trying to be polite. ant behavior; it was a “method of moral persua- Frankly, I don’t give a damn how you are. sion and justification” (Wieder 1988: 175). At this point he changed his naturalistic approach to an By setting aside or “bracketing” their expec- ethnomethodological one. Whereas naturalistic tations from these everyday conversations, the field researchers aim to understand social life as the experimenters made visible the subtleties of mun- dane interactions. For example, although “How are you?” has many possible meanings, none of us
336 ■ Chapter 11: Paradigms, Methods, and Ethics of Qualitative Field Research participants understand it, ethnomethodologists are © Lynda Koolish more intent on identifying the methods through which understanding occurs. In the case of the Anselm L. Strauss, a pioneer qualitative researcher, was a principal convict code, Wieder came to see that convicts used founder of the Grounded Theory Method. the code to make sense of their own interactions with other convicts and with the staff. The eth- way of avoiding the biases that can arise from nography of the halfway house thus shifted to an interpretations of initial observations. ethnography of the code. For instance, the convicts would say, “You know I won’t snitch,” referring to • Obtain multiple viewpoints: In part this refers the code as a way to justify their refusal to answer Wieder’s question (168). According to Wieder, the to the different points of view of participants code “operated as a device for stopping or chang- in the events under study, but Strauss and ing the topic of conversation” (175). Even the staff Corbin suggest that different observational would refer to the code to justify their reluctance techniques may also provide a variety of to help the convicts. Although the code was some- viewpoints. thing that constrained behavior, it also functioned as a tool for the control of interactions. • Periodically step back: As data accumulate, you’ll Grounded Theory begin to frame interpretations about what is going on, and it’s important to keep checking Grounded theory originated from the collaboration your data against those interpretations. As of Barney Glaser and Anselm Strauss, sociologists Strauss and Corbin (1998: 45) say, “The data who brought together two main traditions of re- themselves do not lie.” search: positivism and interactionism. Essentially, grounded theory is the attempt to derive theories • Maintain an attitude of skepticism: As you begin to from an analysis of the patterns, themes, and com- mon categories discovered in observational data. interpret the data, you should regard all those The first major presentation of this method can be interpretations as provisional, using new ob- found in Glaser and Strauss’s book, The Discovery of servations to test those interpretations, not just Grounded Theory (1967). Grounded theory can be confirm them. described as an approach that attempts to combine a naturalist approach with a positivist concern for a • Follow the research procedures: Grounded theory “systematic set of procedures” in doing qualitative research. allows for flexibility in data collection as theo- ries evolve, but Strauss and Corbin (1998: 46) Anselm Strauss and Juliet Corbin (1998: 43–46) stress that three techniques are essential: have suggested that grounded theory allows the re- “making comparisons, asking questions, and searcher to be scientific and creative at the same time, sampling.” as long as the researcher follows these guidelines: Grounded theory emphasizes research proce- • Think comparatively: The authors suggest that it dures. In particular, systematic coding is important for achieving validity and reliability in the data is essential to compare numerous incidents as a grounded theory An inductive approach to the study of social life that attempts to generate a theory from the constant comparing of unfolding observa- tions. This is very different from hypothesis testing, in which theory is used to generate hypotheses to be tested through observations.
Some Qualitative Field Research Paradigms ■ 337 analysis. Because of this somewhat positivistic view current academic program; these changes are then of data, grounded theorists are quite open to the made by the most powerful decision-making body. use of qualitative studies in conjunction with quan- titative ones. Here are two examples of the imple- Shopping Romania mentation of this approach. Much has been written about large-scale changes Studying Academic Change caused by the shift from socialism to capitalism in the former USSR and its Eastern European allies. Clifton Conrad’s (1978) study of academic Patrick Jobes and his colleagues (1997) wanted to change in universities is an early example of the learn about the transition on a smaller scale among grounded theory approach. Conrad hoped to un- average Romanians. They focused on the task of cover the major sources of changes in academic shopping. curricula and at the same time understand the process of change. Using the grounded theory idea Noting that shopping is normally thought of as of theoretical s ampling—whereby groups or insti- a routine, relatively rational activity, the research- tutions are selected on the basis of their theoreti- ers suggested that it could become a social problem cal relevance—Conrad chose four universities for in a radically changing economy. They used the the purpose of his study. In two, the main vehicle Grounded Theory Method to examine Romanian of change was the formal curriculum committee; shopping as a social problem, looking for the ways in the other two, the vehicle of change was an ad in which ordinary people solved the problem. hoc group. Their first task was to learn something about Conrad explained, step by step, the advantage how Romanians perceived and understood the of using the grounded theory approach in building task of shopping. The researchers—participants his theory of academic change. He described the in a social problems class—began by interviewing process of systematically coding data in order to 40 shoppers and asking whether they had experi- create categories that must “emerge” from the data enced problems in connection with shopping and and then assessing the fitness of these categories what actions they had taken to cope with those with each other. Going continuously from data to problems. theory and theory to data allowed him to reassess the validity of his initial conclusions about aca- Once the initial interviews were completed, demic change. the researchers reviewed their data, looking for categories of responses—the shoppers’ most com- For instance, it first seemed that academic mon problems and solutions. One of the most com- change was mainly caused by an administrator mon problems was a lack of money. This led to the who was pushing for it. By reexamining the data researchers’ first working hypothesis: The “socio- and looking for more-plausible explanations, Con- economic position of shoppers would be associated rad found the pressure of interest groups a more with how they perceived problems and sought convincing source of change. The emergence of solutions” (1997: 133). This and other hypotheses these interest groups actually allowed the adminis- helped the researchers focus their attention on trator to become an agent of change. more-specific variables in subsequent interviewing. Assessing how data from each of the two types As they continued, they also sought to inter- of universities fit with the other helped refine the view other types of shoppers. When they inter- theory building. Conrad concluded that changes viewed students, for example, they discovered that in university curricula are based on the following different types of shoppers were concerned with process: Conflict and interest groups emerge different kinds of goods, which in turn affected the because of internal and external social structural problems faced and the solutions tried. forces; they push for administrative intervention and recommendation to make changes in the As the researchers developed additional hy- potheses in response to the continued interview- ing, they also began to develop a more or less
338 ■ Chapter 11: Paradigms, Methods, and Ethics of Qualitative Field Research standardized set of questions to ask shoppers. Ini- theory—case studies can form the basis for the de- tially, all the questions were open-ended, but they velopment of more-general, nomothetic theories. eventually developed closed-ended items as well. Michael Burawoy and his colleagues (1991) This study illustrates the key, inductive prin- have suggested a somewhat different relation- ciples of grounded theory: Data are collected in the ship between case studies and theory. For them, absence of hypotheses. The initial data are used to the e xtended case method has the purpose of determine the key variables as perceived by those discovering flaws in, and then modifying, existing being studied, and hypotheses about relationships social theories. This approach differs importantly among the variables are similarly derived from the from some of the others already discussed. data collected. Continuing data collection yields refined understanding and, in turn, sharpens the Whereas the grounded theorists seek to enter focus of data collection itself. the field with no preconceptions about what they’ll find, Burawoy suggests just the opposite: to try “to Case Studies lay out as coherently as possible what we expect to and the Extended Case Method find in our site before entry” (Burawoy et al. 1991: 9). Burawoy sees the extended case method as a way Social researchers often speak of case studies. A to rebuild or improve theory instead of approving case study focuses attention on a single instance or rejecting it. Thus, he looks for all the ways in of some social phenomenon, such as a village, a which observations conflict with existing theories family, or a juvenile gang. As Charles Ragin and and what he calls “theoretical gaps and silences” Howard Becker (1992) point out, there is little con- (1991: 10). This orientation to field research im- sensus on what may constitute a “case,” and the plies that knowing the literature beforehand is term is used broadly. The case being studied, for ex- actually a must for Burawoy and his colleagues, ample, might be a period of time rather than a par- whereas grounded theorists would worry that ticular group of people. The limitation of attention knowing what others have concluded might bias to a particular instance of something is the essential their observations and theories. characteristic of the case study. To illustrate the extended case method, I’ll pres- The chief purpose of case studies may be de- ent two examples of studies by Burawoy’s students. scriptive, as when an anthropologist describes the culture of a preliterate tribe. Or the in-depth study Teacher–Student Negotiations of a particular case can yield explanatory insights, as when the community researchers Robert and Leslie Hurst (1991) set out to study the patterns Helen Lynd (1929, 1937) and W. Lloyd Warner of interaction between teachers and students of a (1949) sought to understand the structure and pro- junior high school. She went into the field armed cess of social stratification in small-town USA. with existing contradictory theories about the “official” functions of the school. Some theories Case study researchers may seek only an idio- suggested that the purpose of schools is to pro- graphic understanding of the particular case under mote social mobility, whereas others suggested examination, or—as we’ve seen with grounded that schools mainly reproduce the status quo in the form of a stratified division of labor. The official case study The in-depth examination of a single roles assigned to teachers and students could be instance of some social phenomenon, such as a interpreted in terms of either view. v illage, a family, or a juvenile gang. Hurst was struck, however, by the contrast extended case method A technique developed by between these theories and the types of interac- Michael Burawoy in which case study observations tions she observed in the classroom. In her own are used to discover flaws in and to improve existing experiences as a student, teachers had total rights social theories. over the minds, bodies, and souls of their pupils. She observed something quite different at a school in a lower-middle-class neighborhood in Berkeley,
Some Qualitative Field Research Paradigms ■ 339 California—Emerald Junior High School, where In summary, what emerges from Hurst’s study she volunteered as a tutor. She had access to the is an attempt to improve the traditional sociological classroom of Mr. Henry (an eighth-grade English understanding of education by adding the idea that teacher) as well as other teachers’ classrooms, the classroom, school, and family have separate func- lunchroom, and English Department meetings. tions, which in turn can explain the emergence of She wrote field notes based on the negotiations “negotiated order” in the classroom. between students and teachers. She explained the nature of the student–teacher negotiations she wit- The Fight against AIDS nessed by focusing on the separation of functions among the school, the teacher, and the family. Katherine Fox (1991) set out to study an agency whose goal was to fight the AIDS epidemic by In Hurst’s observation, the school fulfilled the bringing condoms and bleach for cleaning needles function of controlling its students’ “bodies”—for to intravenous drug users. It’s a good example example, by regulating their general movements of finding the limitations of well-used models of and activities within the school. The students’ theoretical explanation in the realm of understand- “minds” were to be shaped by the teacher, whereas ing deviance—specifically, the “treatment model” students’ families were held responsible for their that predicted that drug users would come to the “souls”; that is, families were expected to socialize clinic and ask for treatment. Fox’s interactions with students regarding personal values, attitudes, sense outreach workers—most of whom were part of the of property, and sense of decorum. When students community of drug addicts or former prostitutes— don’t come to school with these values in hand, the contradicted that model. teacher, according to Hurst, “must first negotiate with the students some compromise on how the To begin, it was necessary to understand the students will conduct themselves and on what will drug users’ subculture and use that knowledge to be considered classroom decorum” (1991: 185). devise more-realistic policies and programs. The target users had to be convinced, for example, that Hurst explained that the constant bargaining the program workers could be trusted, that they between teachers and students is an expression of were really interested only in providing bleach and the separation between “the body,” which is the condoms. The target users needed to be sure they school’s concern, and “the soul” as family domain. were not going to be arrested. The teachers, who had limited sanctioning power to control their students’ minds in the classroom, Fox’s field research didn’t stop with an ex- were using forms of negotiations with students so amination of the drug users. She also studied the that they could “control . . . the student’s body and agency workers, discovering that the outreach sense of property” (1991: 185), or as Hurst defines program meant different things to the research it, “babysit” the student’s body and soul. directors and the outreach workers. Some of the volunteers who were actually providing the Hurst says she differs from the traditional socio- bleach and condoms were frustrated about the logical perspectives as follows: minor changes they felt they could make. Many thought the program was just a bandage on the I do not approach schools with a futuristic eye. AIDS and drug-abuse problems. Some resented I do not see the school in terms of training, having to take field notes. Directors, on the other socializing, or slotting people into future hier- hand, needed reports and field notes so that they archies. To approach schools in this manner is could validate their research in the eyes of the to miss the negotiated, chaotic aspects of the federal and state agencies that financed the proj- classroom and educational experience. A futur- ect. Fox’s study showed how the AIDS research ist perspective tends to impose an order and project developed the bureaucratic inertia typical purpose on the school experience, missing its of established organizations: Its goal became that day-to-day reality. of sustaining itself. (1991: 186)
340 ■ Chapter 11: Paradigms, Methods, and Ethics of Qualitative Field Research Courtesy Dorothy E. Smith Both of these studies illustrate how the ex- tended case method can operate. The researcher enters the field with full knowledge of existing theories but aims to uncover contradictions that require the modification of those theories. One criticism of the case study method is the limited generalizability of what may be observed in a single instance of some phenomenon. This risk is reduced, however, when more than one case is studied in depth: the comparative case study method. You can find examples of this in the dis- cussion of comparative and historical methods in Chapter 10 of this book. Institutional Ethnography Dorothy Smith, a pioneering social researcher and founder of institutional ethnography. Institutional ethnography is an approach origi- nally developed by Dorothy Smith (1978) to bet- This approach links the “microlevel” of every- ter understand women’s everyday experiences by day personal experiences with the “macrolevel” of discovering the power relations that shape those institutions. As M. L. Campbell puts it, experiences. Today this methodology has been ex- tended to the ideologies that shape the experiences Institutional ethnography, like other forms of of any oppressed subjects. ethnography, relies on interviewing, observa- tions and documents as data. Institutional Smith and other sociologists believe that if ethnography departs from other ethnographic researchers ask women or other members of subor- approaches by treating those data not as the dinated groups about “how things work,” they can topic or object of interest, but as “entry” into discover the institutional practices that shape their the social relations of the setting. The idea is to realities (M. L. Campbell 1998; D. Smith 1978). tap into people’s expertise. The goal of such inquiry is to uncover forms of oppression that more-traditional types of research (1998: 57) often overlook. Here are two examples of this approach. Dorothy Smith’s methodology is similar to ethnomethodology in the sense that the subjects Mothering, Schooling, themselves are not the focus of the inquiry. The and Child Development institutional ethnographer starts with the personal experiences of individuals but proceeds to uncover Our first example of institutional ethnography is a the institutional power relations that structure study by Alison Griffith (1995), who collected data and govern those experiences. In this process, the with Dorothy Smith on the relationship among researcher can reveal aspects of society that would mothering, schooling, and children’s development. have been missed by an inquiry that began with Griffith started by interviewing mothers from three the official purposes of institutions. cities of southern Ontario about their everyday work of creating a relationship between their fami- institutional ethnography A research technique lies and the school. This was the starting point for in which the personal experiences of individuals are used to reveal power relationships and other characteristics of the institutions within which they operate.
Some Qualitative Field Research Paradigms ■ 341 other interviews with parents, teachers, school ad lesbian students. However, instead of emphasizing ministrators, social workers, school psychologists, the subjects’ viewpoints, both analyses focused on and central office administrators. the power relations that shaped these women’s e xperiences and reality. In her findings, Griffith explained how the dis- course about mothering had shifted its focus over Participatory Action Research time from a mother–child interaction to “child- centered” recommendations. She saw a distinct Our final field research paradigm takes us further similarity in the discourse used by schools, the along in our earlier discussion of the status and media (magazines and television programs), the power relationships linking researchers to the sub- state, and child-development professionals. jects of their research. Within the participatory action research (PAR) paradigm, the researcher’s Teachers and child-development profession- function is to serve as a resource to those being als saw the role of mothers in terms of a necessary studied—typically, disadvantaged groups—as an collaboration between mothers and schools for opportunity for them to act effectively in their own the child to succeed not only in school but also interest. The disadvantaged subjects define their in life. Because of unequal resources, all mothers problems, define the remedies desired, and take the do not participate in this discourse of “good” child lead in designing the research that will help them development the same way. Griffith found that realize their aims. working-class mothers were perceived as weaker than middle-class mothers in the “stimulation” This approach began in Third World research effort of schooling. Griffith argues that this child- development, but it spread quickly to Europe and development discourse, embedded in the school North America (Gaventa 1991). It comes from a institution, perpetuates the reproduction of class by vivid critique of classical social science research. making middle-class ideals for family–school rela- According to the PAR paradigm, traditional re- tions the norm for everyone. search is perceived as an “elitist model” (Whyte, Greenwood, and Lazes 1991) that reduces the Compulsory Heterosexuality “subjects” of research to “objects” of research. A ccording to many advocates of the PAR perspec- The second illustration of institutional ethnography tive, the distinction between the researcher and the is taken from Didi Khayatt’s (1995) study of the in- researched should disappear. They argue that the stitutionalization of compulsory heterosexuality in subjects who will be affected by research should schools and its effects on lesbian students. In 1990, also be responsible for its design. Khayatt began her research by interviewing 12 Toronto lesbians, 15 to 24 years of age. Beginning Implicit in this approach is the belief that re- with the young women’s viewpoint, she expanded search functions not only as a means of knowledge her inquiry to other students, teachers, guidance production but also as a “tool for the education and counselors, and administrators. development of consciousness as well as mobiliza- tion for action” (Gaventa 1991: 121–22). Advocates Khayatt found that the school’s administrative of participatory action research equate access to practices generated a compulsory heterosexuality, information with power and argue that this power which produced a sense of marginality and vul- has been kept in the hands of the dominant class, nerability among lesbian students. For example, the school didn’t punish harassment and name- participatory action research (PAR) An ap- calling directed at gay students. The issue of homo proach to social research in which the people being sexuality was excluded from the curriculum lest it studied are given control over the purpose and pro- appear to students as an alternative to hetero- cedures of the research; intended as a counter to the sexuality. implicit view that researchers are superior to those they study. In both of the studies I’ve described, the inquiry began with the women’s standpoint—mothers and
342 ■ Chapter 11: Paradigms, Methods, and Ethics of Qualitative Field Research sex, ethnicity, or nation. Once people see them- The team had access to all financial information selves as researchers, they automatically regain and was authorized to call on anyone within the power over knowledge. company. This strategy allowed workers to make suggestions outside the realm usually available Participatory action research poses a special to them. According to Whyte and his colleagues, challenge to researchers. On the one hand, par- “reshaping the box enabled the CST to call upon ticipants in the social situation ideally become em- management to explain and justify all staff ser- powered to frame research relevant to their needs, vices” (1991: 27). Because of the changes suggested as they define those needs. At the same time, the by the CST and implemented by management, the researcher brings special skills and insights that company saved the targeted $3.2 million. nonresearchers lack. So who should be in charge? Andrew Sense (2006: 1) suggests that this decision Management was so pleased by this result that may have to be made in the moment: “Do I take it expanded the wire harness CST project to three the ‘passenger’ position on the bus or do I take the other departments that were threatened by compe- ‘driver’ seat and be a little more provocative to en- tition. Once again, management was happy about ergise the session[?] My view at this moment is to the money saved by the teams of workers. judge it on the day.” The Xerox case study is an interesting example Examples of this approach include research of participatory action research because it shows on community power structures, corporate re- how the production of knowledge does not always search, and “right-to-know” movements (Whyte, have to be an elitist enterprise. The “experts” do Greenwood, and Lazes 1991). Here are two ex- not necessarily have to be the professionals. Ac- amples of corporate research that used a PAR cording to Whyte and his colleagues, “At Xerox, approach. participatory action research created and guided a powerful process of organizational learning—a The Xerox Corporation process whereby leaders of labor and management learned from each other and from the consultant/ A participatory action research project took place at facilitator, while he learned from them” (1991: 30). the Xerox corporation at the instigation of leaders of both management and the union. Management’s PAR and Welfare Policy goal was to lower costs so that the company could thrive in an increasingly competitive market. The Participatory action research often involves union suggested a somewhat broader scope: im- poor people, as they are typically less able than proving the quality of working life while lowering other groups to influence the policies and ac- manufacturing costs and increasing productivity. tions that affect their own lives. Bernita Quoss, Margaret Cooney, and Terri Longhurst (2000) Company managers began by focusing at- report a research project involving welfare policy tention on shop-level problems; they were less in Wyoming. University students, many of them concerned with labor contracts or problematic welfare recipients, undertook research and lob- managerial policies. At the time, management bying efforts aimed at getting Wyoming to accept had a plan to start an “outsourcing” program that postsecondary education as “work” under the would lay off 180 workers, and the union had state’s new welfare regulations. begun mobilizing to oppose the plan. Peter Lazes, a consultant hired by Xerox, spent the first month This project began against the backdrop convincing management and the union to create a of the 1996 Personal Responsibility and Work “cost study team” (CST) that included workers in Opportunity Reconcilliation Act (PRWORA), which the wire harness department. eliminated education waivers that had been Eight full-time workers were assigned to the available under the previous welfare law, the CST for six months. Their task was to study the 1988 Family Support Act (FSA). These waivers possibilities of making changes that would save had permitted eligible participants in the cash the company $3.2 million and keep the 180 jobs.
Conducting Qualitative Field Research ■ 343 assistance AFDC program to attend college as been unusually deliberate in framing a variety of an alternative to work training requirements. paradigms to enrich the observation of social life. Empirical studies of welfare participants who received these waivers have provided evidence The impact of researcher paradigms on the that education, in general, is the most effec- conduct of research is nowhere more explicitly rec- tive way to stay out of poverty and achieve ognized than in the case of kaupapa Maori research, self-sufficiency. a form of participatory action research devel- oped within the indigenous Maori community of (Quoss, Cooney, and Longhurst 2000: 47) New Zealand. As Shayne Walker, Anaru Eketone, and Anita Gibbs (2006) report, an adherence to The students began by establishing an organi- Maori culture shapes not only the purposes of such zation called Empower and by making presenta- research but also its processes and practices. In a tions on campus to enlist broad student and faculty study of foster care, for example, the purpose of the support. They compiled existing research relevant study was established by those most directly con- to the issue and established relationships with cerned. The method of collecting data conformed members of the state legislature. By the time the to Maori practices, including public gatherings. The 1997 legislative session opened, the students were implications derived from the analysis of data were actively engaged in the process of modifying state tailored to Maori ways of doing things. welfare laws to offset the shift in federal policy. Conducting Qualitative The students prepared and distributed fact Field Research sheets and other research reports that would be relevant to the legislators’ deliberations. They at- So far in this chapter we’ve examined the kinds of tended committee meetings and lobbied legislators topics appropriate to qualitative field research, spe- on a one-to-one basis. When erroneous or mislead- cial considerations in doing this kind of research, ing data were introduced into the discussions, the and a sampling of paradigms that direct different student-researchers were on hand to point out the types of research efforts. Along the way we’ve seen errors and offer corrections. some examples that illustrate field research in ac- tion. To round out the picture, we turn now to Ultimately, they succeeded. Welfare recipients specific ideas and techniques for conducting field in Wyoming were allowed to pursue postsecondary research, beginning with how researchers prepare education as an effective route out of poverty. for work in the field. Some researchers speak of emancipatory Preparing for the Field r esearch, which Ardha Danieli and Carol Wood- hams (2005: 284) define as “first and foremost a Suppose for the moment that you’ve decided to process of producing knowledge which will be of undertake field research on a campus political benefit to oppressed people; a political outcome.” organization. Let’s assume further that you’re not Both qualitative and quantitative methods can a member of that group, that you do not know a be used to pursue this goal, but it goes well be- great deal about it, and that you’ll identify your- yond simply learning what’s so, even as seen from self to the participants as a researcher. This section the subjects’ point of view. The authors focus on will use this example and others to discuss some the study of disability, and they note similarities of the ways you might prepare yourself before in the development of emancipatory research and undertaking direct observations. early feminist research. emancipatory research Research conducted for As you can see, the seemingly simple p rocess the purpose of benefiting disadvantaged groups. of observing social action as it occurs has subtle though important variations. As we saw in Chapter 3, all our thoughts occur within and are shaped by paradigms, whether we’re conscious of it or not. Qualitative field researchers have
344 ■ Chapter 11: Paradigms, Methods, and Ethics of Qualitative Field Research As is true of all research methods, you would to develop an identity with the people to be stud- be well advised to begin with a search of the re ied. If you wish to study dishwashers in a restau- levant literature, filling in your knowledge of rant, the most direct method would be to get a job the subject and learning what others have said as a dishwasher. In the case of the student political about it. (Library research is discussed at length group, you might simply join the group. in Appendix A.) Many of the social processes appropriate to In the next phase of your research, you might field research are open enough to make your wish to discuss the student political group with oth- contact with the people to be studied rather sim- ers who have already studied it or with anyone else ple and straightforward. If you wish to observe likely to be familiar with it. In particular, you might a mass demonstration, just be there. If you wish find it useful to discuss the group with one or more to observe patterns in jaywalking, hang around informants (discussed in Chapter 5). Perhaps you busy streets. have a friend who is a member, or you can meet someone who is. This aspect of your preparation Whenever you wish to make more-formal is likely to be more effective if your relationship contact with the people, identifying yourself as a with the informant extends beyond your research researcher, you must establish a rapport with them. role. In dealing with members of the group as in- You might contact a participant with whom you formants, you should take care that your initial feel comfortable and gain that person’s assistance. discussions do not compromise or limit later aspects In studying a formal group, you might approach of your research. Keep in mind that the impression the groups’ leaders, or you might find that one of you make on the informant, the role you establish your informants can introduce you. (See the Tips for yourself, may carry over into your later effort. and Tools feature “Establishing Rapport” for more For example, creating the initial impression that on this.) you may be an undercover FBI agent is unlikely to facilitate later observations of the group. Although you’ll probably have many options in making your initial contact with the group, You should also be wary about the informa- realize that your choice can influence your sub- tion you get from informants. Although they may sequent observations. Suppose, for example, have more direct, personal knowledge of the sub- that you’re studying a university and begin with ject under study than you do, what they “know” high-level administrators. This choice is likely to is probably a mixture of fact and point of view. have a couple of important consequences. First, M embers of the political group in our example your initial impressions of the university will be (as well as members of opposing political groups) shaped to some extent by the administrators’ would be unlikely to provide completely unbiased views, which will differ quite a bit from those information. Before making your first contact with of students or faculty. This initial impression the student group, then, you should already be may influence the way you observe and inter- quite familiar with it, and you should understand pret events subsequently—especially if you’re its general philosophical context. unaware of the influence. There are many ways to establish your initial Second, if the administrators approve of your contact with the people you plan to study. How research project and encourage students and fac- you do it will depend, in part, on the role you in- ulty to cooperate with you, the latter groups will tend to play. Especially if you decide to take on the probably look on you as somehow aligned with the role of complete participant, you must find a way administration, which can affect what they say to you. For example, faculty members might be reluc- rapport An open and trusting relationship; espe- tant to tell you about plans to organize through the cially important in qualitative research between Teamster’s Union. researchers and the people they’re observing. In making direct, formal contact with the people you want to study, you’ll be required to give them some explanation of the purpose of your
Conducting Qualitative Field Research ■ 345 Tips and Tools Establishing Rapport 2. B e an attentive listener rather than a talker. You should not remain mute, of course, but you should talk primarily (a) to elicit more In qualitative field research, it’s almost always vital that you be able information from the other person or (b) to answer questions they to establish rapport with those you’re observing, especially if your may have about you and your research. observations include in-depth interviews and interactions. Rapport might be defined as a open and trusting relationship. But how do you 3. Don’t argue with your subjects. While you don’t have to agree with do that? any points of view expressed by your subjects, you should never argue with them nor try to change their minds. Keep reminding Let’s assume that you’ll be identifying yourself as a researcher. You’ll yourself that your genuine purpose is to understand their world need to explain your research purpose in a nonthreatening way. Say that and how it makes sense to them—whether it works for you or you are there to learn about them and understand them, not to judge not. A little humility may help with this. You’ll be able to hear and them or cause them any problems. This will work best if you understand people better if you don’t start out feeling superior to them. 1. Actually have a genuine interest in understanding the people you’re observing and can communicate that interest to them. This gives 4. Be relaxed and appropriate to the setting. Some people are more them a sense of self-worth, which will increase their willingness formal or informal than others, and you’ll do well to take on their to open up to you. Pretending to be interested is not the same as general style or at least find a way to relax with whatever style is really being interested. In fact, if you aren’t interested in learning most comfortable for them. If you can get them to relax and enjoy what things look like from the point of view of those you’re observ- the interaction, you’ll have achieved the rapport you need. And ing, you might consider another activity and not waste their time you’ll probably enjoy the interaction yourself. and your own. study. Here again, you face an ethical dilemma. Qualitative Interviewing Telling them the complete purpose of your research might eliminate their cooperation altogether or In part, field research is a matter of going where significantly affect their behavior. On the other the action is and simply watching and listening. hand, giving only what you believe would be an As the baseball legend Yogi Berra said, “You can acceptable explanation may involve outright de- see a lot just by observing”—provided that you’re ception. Your decisions in this and other matters paying attention. At the same time, as I’ve already will probably be largely determined by the purpose indicated, field research can involve more-active of your study, the nature of what you’re studying, inquiry. Sometimes it’s appropriate to ask people the observations you wish to use, and similar fac- questions and record their answers. Your on-the- tors, but you must also take ethical considerations spot observations of a full-blown riot will lack into account. something if you don’t know why people are riot- ing. Ask somebody. Previous field research offers no fixed rule— methodological or ethical—to follow in this regard. When Cecilia Menjívar (2000) wanted to learn Your appearance as a researcher, regardless of your about the experiences of Salvadoran immigrants in stated purpose, may result in a warm welcome San Francisco, she felt in-depth interviews would from people who are flattered that a scientist finds be a useful technique, along with personal obser- them important enough to study. Or, it may result vations. Before she was done, she had discovered in your being totally ostracized or worse. It prob- a much more complex system of social processes ably wouldn’t be a good idea, for example, to burst and structures than we would have imagined. into a meeting of an organized crime syndicate and Although it was important for new immigrants to announce that you’re writing a term paper on or- have a support structure of family members already ganized crime. in the United States, Menjívar found that her inter viewees were often reluctant to call on relatives for
346 ■ Chapter 11: Paradigms, Methods, and Ethics of Qualitative Field Research help, for several reasons. On the one hand, they but not a set of questions that must be asked with might jeopardize those family members who were particular words and in a particular order. At the here illegally and living in poverty. At the same same time, the qualitative interviewer, like the time, asking for help would put them in debt to survey interviewer, must be fully familiar with the those helping them out. Menjívar also discovered questions to be asked. This allows the interview to that Salvadoran gender norms put women immi- proceed smoothly and naturally. grants in an especially difficult situation, because they were largely prohibited from seeking the help A qualitative interview is essentially a con- of men they weren’t related to, lest they seem to versation in which the interviewer establishes a obligate themselves sexually. These are the kinds of general direction for the conversation and pursues discoveries that can emerge from open-ended, in- specific topics raised by the respondent. Ideally, the depth interviewing. respondent does most of the talking. If you’re talk- ing more than 5 percent of the time, that’s prob- We’ve already discussed interviewing in ably too much. Chapter 8, and much of what was said there applies to qualitative field interviewing. The in- Steinar Kvale (1996: 3–5) offers two metaphors terviewing you’ll do in connection with field ob- for interviewing: the interviewer as a “miner” or servation, however, is different enough to demand as a “traveler.” The first model assumes that the a separate treatment. In surveys, questionnaires subject possesses specific information and that the are rigidly structured; however, less-structured interviewer’s job is to dig it out. By contrast, in the interviews are more appropriate to field research. second model, the interviewer Herbert and Riene Rubin (1995: 43) describe the distinction as follows: “Qualitative interviewing wanders through the landscape and enters into design is flexible, iterative, and continuous, rather conversations with the people encountered. than prepared in advance and locked in stone.” The traveler explores the many domains of the They elaborate in this way: country, as unknown territory or with maps, roaming freely around the territory. . . . The in- Design in qualitative interviewing is iterative. terviewer wanders along with the local inhabit- That means that each time you repeat the basic ants, asks questions that lead the subjects to tell process of gathering information, analyzing it, their own stories of their lived world. winnowing it, and testing it, you come closer to a clear and convincing model of the phenom- Asking questions and noting answers is a natu- enon you are studying. . . . ral human process, and it seems simple enough to add it to your bag of tricks as a field researcher. Be The continuous nature of qualitative in- a little cautious, however. Wording questions is a terviewing means that the questioning is re tricky business. All too often, the way we ask ques- designed throughout the project. tions subtly biases the answers we get. Sometimes we put our respondent under pressure to look (1995: 46–47) good. Sometimes we put the question in a particu- lar context that omits altogether the most relevant Unlike a survey, a qualitative interview answers. is an interaction between an interviewer and a respondent in which the interviewer has a general Suppose, for example, that you want to find plan of inquiry, including the topics to be covered, out why a group of students is rioting and pillaging on campus. You might be tempted to focus your qualitative interview Contrasted with survey questioning on how students feel about the dean’s interviewing, the qualitative interview is based on recent ruling that requires students always to carry a set of topics to be discussed in depth rather than The Practice of Social Research with them on campus. based on the use of standardized questions. (Makes sense to me.) Although you may collect a great deal of information about students’ attitudes toward the infamous ruling, they may be rioting
Conducting Qualitative Field Research ■ 349 As with all other aspects of field research, in- methods. This means that the participants do not terviewing improves with practice. Fortunately, it’s statistically represent any meaningful population. something you can practice any time you want. However, the purpose of the study is to explore Practice on your friends. rather than to describe or explain in any definitive sense. Nevertheless, typically more than one focus Focus Groups group is convened in a given study because of the serious danger that a single group of 7 to 12 people Although our discussions of field research so far will be too atypical to offer any generalizable have focused on studying people in the process of insights. living their lives, researchers sometimes bring peo- ple into the laboratory for qualitative interviewing William Gamson (1992) used focus groups to and observation. The focus group method, which examine how U.S. citizens frame their views of po- is also called group interviewing, is essentially a litical issues. Having picked four issues—affirmative qualitative method. It is based on structured, semi- action, nuclear power, troubled industries, and structured, or unstructured interviews. It allows the Arab–Israeli conflict—Gamson undertook a the researcher/interviewer to question several content analysis of press coverage to get an idea of individuals systematically and simultaneously. This the media context within which we think and talk data-collection technique is frequently used in about politics. Then the focus groups were con- political and market research but is used for other vened for a firsthand observation of the process of purposes as well. In Silent Racism, for example, Bar- people discussing issues with their friends. bara Trepagnier (2006) used focus groups to exam- ine the persistence of racism among “well-meaning Richard Krueger points to five advantages of white people.” focus groups: In a hypothetical market-research example, 1. The technique is a socially oriented re- imagine that you’re thinking about introducing a search method capturing real-life data in a new product. Let’s suppose that you’ve invented social environment. a new computer that not only does word process- ing, spreadsheets, data analysis, and the like but 2. It has flexibility. also contains a fax machine, CD and DVD player/ recorder, microwave oven, and coffeemaker. To 3. It has high face validity. highlight its computing and coffee-making features, you’re thinking of calling it “The Compulator.” 4. It has speedy results. You figure the new computer will sell for about $28,000, and you want to know whether people 5. It is low in cost. are likely to buy it. Your prospects might be well served by focus groups. (1988: 47) In a focus group, typically 5 to 15 people are In addition to these advantages, group dynam- brought together in a private, comfortable envi- ics frequently bring out aspects of the topic that ronment to engage in a guided discussion of some would not have been anticipated by the researcher topic—in this case, the acceptability and salability and would not have emerged from interviews with of The Compulator. The subjects are selected on the individuals. In a side conversation, for example, a basis of relevance to the topic under study. Given couple of the participants might start joking about the likely cost of The Compulator, your focus group the results of leaving out one letter from a prod- participants would probably be limited to upper- uct’s name. This realization might save the manu- income groups, for example. Other, similar consid- facturer great embarrassment later on. erations might figure into the selection. focus group A group of subjects interviewed to- Participants in focus groups are not likely to gether, prompting a discussion. The technique is be chosen through rigorous probability-sampling frequently used by market researchers, who ask a group of consumers to evaluate a product or discuss a type of commodity, for example.
348 ■ Chapter 11: Paradigms, Methods, and Ethics of Qualitative Field Research Notice how the interview begins by wander- than talking. (Do this in ordinary conversations, ing off into a story about the respondent’s uncle. and people will actually regard you as a great The first attempt to focus things back on the stu- conversationalist.) dent’s own choice of major (“Did you talk to your uncle . . . ?”) fails. The second attempt (“So is your John Lofland and his colleagues (2006: 69–70) main interest . . . ?”) succeeds. Now the student is suggest that researchers should adopt the role providing the kind of information you’re looking of the “socially acceptable incompetent” when for. It’s important for field researchers to develop interviewing. That is, offer yourself as someone the ability to “control” conversations in this fash- who does not understand the situation you find ion. At the same time, of course, you need to be on yourself in and must be helped to grasp even the the alert for “distractions” that point to unexpect- most basic and obvious aspects of that situation: “A edly important aspects of your research interest. naturalistic investigator, almost by definition, is one who does not understand. She or he is ‘ignorant’ Herbert and Riene Rubin offer several ways and needs to be ‘taught.’ This role of watcher and to control a “guided conversation,” including the asker of questions is the quintessential student role” following: (Lofland et al. 2006: 69). If you can limit the number of main topics, Interviewing needs to be an integral part of it is easier to maintain a conversational flow the entire field research process. Later, I’ll stress from one topic to another. Transitions should the need to review your observational notes every be smooth and logical. “We have been talking night—making sense out of what you’ve observed, about mothers, now let’s talk about fathers,” getting a clearer feel for the situation you’re study- sounds abrupt. A smoother transition might ing, and finding out what you should pay more be, “You mentioned your mother did not care attention to in further observations. In the same how you performed in school—was your father fashion, you’ll need to review your notes on in- more involved?” The more abrupt the transi- terviews, recording especially effective questions tion, the more it sounds like the interviewer and detecting all those questions you should have has an agenda that he or she wants to get asked but didn’t. Start asking such questions the through, rather than wanting to hear what the next time you interview. If you’ve recorded the interviewee has to say. interviews, replay them as a useful preparation for future interviews. (1995: 123) Steinar Kvale (1996: 88) details seven stages in Because field research interviewing is so much the complete interviewing process: like normal conversation, researchers must keep reminding themselves that they are not having 1. Thematizing: Clarifying the purpose of the inter- a normal conversation. In normal conversations, views and the concepts to be explored each of us wants to come across as an interest- ing, worthwhile person. If you watch yourself 2. Designing: Laying out the process through the next time you chat with someone you don’t which you’ll accomplish your purpose, includ- know too well, you’ll probably find that much of ing a consideration of the ethical dimension your attention is spent on thinking up interesting things to say—contributions to the conversation 3. Interviewing: Doing the actual interviews that will make a good impression. Often, we don’t really hear each other, because we’re not really 4. Transcribing: Creating a written text of the listening—we’re too busy thinking of what we’ll interviews say next. As an interviewer, the desire to appear interesting is counterproductive. The interviewer 5. Analyzing: Determining the meaning of gath- needs to make the other person seem interest- ered materials in relation to the purpose of the ing, by being interested—and by listening more study 6. Verifying: Checking the reliability and validity of the materials 7. Reporting: Telling others what you’ve learned
350 ■ Chapter 11: Paradigms, Methods, and Ethics of Qualitative Field Research Krueger also notes some disadvantages of the successfully for extended discussions of sexuality, focus group method, however: among Swedish teenagers in a youth detention home. 1. Focus groups afford the researcher less control than individual interviews. Like other social research techniques, focus groups are adapting to new communication modal- 2. Data are difficult to analyze. ities. George Silverman (2005), for example, offers a discussion of telephone and online focus groups. 3. Moderators require special skills. Recording Observations 4. Difference between groups can be troublesome. The greatest advantage of the field research method is the presence of an observing, thinking 5. Groups are difficult to assemble. researcher on the scene of the action. Even tape re- corders and cameras cannot capture all the relevant 6. The discussion must be conducted in a aspects of social processes, although both of those c onducive environment. devices can be quite useful to the field researcher. Consequently, in both direct observation and (1988: 48) interviewing, it’s vital to make full and accurate notes of what goes on. If possible, take notes on As we’ve seen, the group interview presents your observations as you observe. When that’s not several advantages, but it also has its challenges. feasible, write down your notes as soon as possible In a focus group interview, much more than in afterward. any other type of interview, the interviewer has to develop the skills of a moderator. Controlling the In your notes, include both your empirical dynamic within the group is a major challenge. observations and your interpretations of them. In Letting one interviewee dominate the focus group other words, record what you “know” has hap- interview reduces the likelihood that the other pened and what you “think” has happened. Be subjects will express themselves. This can generate sure to identify these different kinds of notes for the problem of group conformity or groupthink, what they are. For example, you might note that which is the tendency for people in a group to con- Person X spoke out in opposition to a proposal form with the opinions and decisions of the most made by a group leader (an observation), that you outspoken members of the group. This danger is think this represents an attempt by Person X to compounded by the possibility that only one or take over leadership of the group (an interpreta- two people sometimes dominate the conversation. tion), and that you think you heard the leader Interviewers need to be aware of this phenomenon comment to that effect in response to the opposi- and try to get everyone to participate fully on all the tion (a tentative observation). issues brought in the interview. Adding to the chal- lenge, of course, is that the interviewer must resist Of course, you can’t observe everything; nor overdirecting the interview and the interviewees, can you record everything you do observe. Just thus bringing her or his own views into play. as your observations will represent a sample of all possible observations, your notes will represent a Although focus group research differs from sample of your observations. The idea, of course, is other forms of qualitative field research, it further to record the most pertinent ones. illustrates the possibilities for doing social research face-to-face with those we wish to understand. In The Tips and Tools feature “Interview Transcript addition, David Morgan (1993) suggests that focus Annotated with Researcher Memos” provides an groups are an excellent device for generating ques- extract from an in-depth interview with a woman tionnaire items for a subsequent survey. film director, given by Sandrine Zerbib. Notice that the illustration contains a portion of an in-depth Because they center on a particular topic and interview along with some of Zerbib’s memos, take relatively little time, focus groups are typi- cally regarded as an “in-depth” research technique. However, Carolina Överlien, Karin Aronsson, and Margareta Hydén (2005) have used the technique
Conducting Qualitative Field Research ■ 351 Tips and Tools Interview Transcript Annotated J: Yeah, oh yeah, I mean . . . a lot of women have commiserated with Researcher Memos about, you know when you have to walk on the set for the first time, they’re all used to working like a well-oiled machine and Thursday August 26, 12:00–1:00 they say,“Oh, here is the woman, something different”and sometimes they can be horrible, they can resist your directing and R: What is challenging for women directors on a daily experience, on they can, they can sabotage you, by taking a long time to light, or a daily life? to move sets, or to do something . . . and during that time you’re wasting time, and that goes on a report, and the report goes to J: Surviving. the front [368] office, and, you know, and so on and so on and so on and so forth. And people upstairs don’t know what the circum- R: OK. Could you develop a little bit on that? [I need to work on my stances are, and they are not about to fire a cinematographer that interview schedule so that my interviewee answers with more is on their show for ever and ever . . . nor do they want to know elaboration without having to probe.] that this guy is a real bastard, and making your life a horror. They don’t want to know that, so therefore, they go off, because she’s a J: Yeah, I mean it’s all about trying to get, you know, in, trying to get woman let’s not hire any more women, since he has problems with the job, and try, you know, to do a great job so that you are invited women. You know, so, there is that aspect. back to the next thing. And particularly since they are so many, you know, difficulties in women directing. It makes it twice as hard to [I need to review the literature on institutional discrimination. It gain into this position where you do an incredible job, because . . . seems that the challenges that Joy is facing are not a matter of a par- you can’t just do an average job, you have to [347] do this job that ticular individual. She is in a double bind situation where whether she just knocks your socks off all the time, and sometimes you don’t complains or not, she will not be treated equal to men. Time seems to be get the opportunity to do that, because either you don’t have a one quantifiable measurement of how well she does her job and, as ob- good producer or you have so many pressures that you can’t see served in other professions, the fact that she is a woman is perceived as straight or your script is lousy, and you have to make a silk purse a handicap. Review literature on women in high management position. out of sow’s ear. You know, you have a lot of extra strikes against I need to keep asking about the dynamics between my interviewees and you than the average guy who has similar problems, because you the crewmembers on the set. The cinematographer has the highest sta- are a woman and they look at it, and women are more visible than tus on the set under the director. Explore other interviews about reasons men . . . in unique positions. for conflict between them.] [It seems that Joy is talking about the particularities of the film [Methods (note to myself for the next interviews):Try to avoid phone industry. There are not that many opportunities and in order to keep interviews unless specific request from the interviewee. It is difficult to working, she needs to build a certain reputation. It is only by continuing assess how the interviewee feels with the questions. Need body language to direct that she can maintain or improve her reputation. She thinks that because I become more nervous about the interview process.] it is even harder for women but does not explain it.] Note: R is the interviewer and J is the director-subject. A number in brackets represents R: Hum . . . what about on the set did you experience, did it feel . . . a word that was inaudible from the interview. It is the number that appeared on the did people make it clear that you were a woman, and you transcribing machine, with each interview starting at count 0. The numbers help the felt treated differently? [I am trying to get her to speak about researcher locate a passage quickly when he or she reviews the interview. more specific and more personal experiences without leading her answer] written during her review of the interview later on. by preparing standardized recording forms in ad- Chapter 13 will present extensive, computerized vance. In a study of jaywalking, for example, you analyses from this study on women film directors. might anticipate the characteristics of pedestrians that are most likely to be useful for analysis—age, Some of your most important observations can gender, social class, ethnicity, and so forth—and be anticipated before you begin the study; others prepare a form in which observations of these will become apparent as your observations prog- variables can be recorded easily. Alternatively, you ress. Sometimes you can make note taking easier
352 ■ Chapter 11: Paradigms, Methods, and Ethics of Qualitative Field Research might develop a symbolic shorthand in advance I obtained the vast majority of data for this article to speed up recording. For studying audience par- through informal interviews. I unobtrusively ticipation at a mass meeting, you might want to took notes throughout the day and at social construct a numbered grid representing the differ- events. Upon returning home, these data were ent sections of the meeting room; then you could compiled into my field notes. During the work- record the location of participants easily, quickly, day and during races, parties, and other social and accurately. gatherings, casual conversations provided the truest glimpses into messenger beliefs, ideologies, None of this advance preparation should limit and opinions. To this end, I avoided formal in- your recording of unanticipated events and aspects terviews and instead allowed my questions to be of the situation. Quite the contrary, speedy han- answered by normal talk within the social world. dling of anticipated observations can give you more freedom to observe the unanticipated. I know this method sounds logical, but it takes self-discipline to put it into practice. Careful obser- You’re already familiar with the process of tak- vation and note taking can be tiring, especially if ing notes, just as you already have at least informal it involves excitement or tension and if it extends experience with field research in general. Like over a long period. If you’ve just spent eight hours good field research, however, good note taking re- observing and making notes on how people have quires careful and deliberate attention and involves been coping with a disastrous flood, your first de- specific skills. Some guidelines follow. (You can sire afterward will likely be to get some sleep, dry learn even more from Lofland et al. 2006: 110–17.) clothes, or a drink. You may need to take some inspiration from newspaper reporters who undergo First, don’t trust your memory any more than the same sorts of hardships then write their stories you have to—it’s untrustworthy. To illustrate this to meet their deadlines. point, try this experiment. Recall the last three or four movies you saw that you really liked. Now, Third, you’ll inevitably wonder how much you name five of the actors or actresses. Who had the should record. Is it really worth the effort to write longest hair? Or can you remember what your out all the details you can recall right after the ob- boyfriend, girlfriend, or best friend was wearing servational session? The general guideline is yes. yesterday? (Remembering what you were wearing Generally, in field research you can’t be really sure yesterday may even be a challenge.) of what’s important and what’s unimportant until you’ve had a chance to review and analyze a great Even if you pride yourself on having a pho- volume of information, so you should record even tographic memory, it’s a good idea to take notes things that don’t seem important at the outset. either during the observation or as soon afterward They may turn out to be significant after all. Also, as possible. If you take notes during observation, the act of recording the details of something “un- do it unobtrusively, because people are likely to important” may jog your memory on something behave differently if they see you taking down that is important. everything they say or do. Realize that most of your field notes will not Second, it’s usually a good idea to take notes be reflected in your final report on the project. Put in stages. In the first stage, you may need to take more harshly, most of your notes will be “wasted.” sketchy notes (words and phrases) in order to But take heart: Even the richest gold ore yields keep abreast of what’s happening. Then go off by only about 30 grams of gold per metric ton, mean- yourself and rewrite your notes in more detail. If ing that 99.997 percent of the ore is wasted. Yet, you do this soon after the events you’ve observed, that 30 grams of gold can be hammered out to the sketchy notes should allow you to recall most cover an area 18 feet square—the equivalent of of the details. The longer you delay, the less likely about 685 book pages. So take a ton of notes, and you’ll be able to recall things accurately and fully. plan to select and use only the gold. In his study of bike messengers in New York City, mentioned earlier, Jeffrey Kidder reports on this process (2005: 349):
Conducting Qualitative Field Research ■ 347 for some other reason. Perhaps most are simply There’s something we can learn in this regard joining in for the excitement. Properly done, field from the martial arts. The aikido master never research interviewing enables you to find out. resists an opponent’s blow but instead accepts it, joins with it, and then subtly redirects it in a more In both qualitative and quantitative research, appropriate direction. Field interviewing requires we tend to think of using face-to-face or telephone an analogous skill. Instead of trying to halt your interviews. When Nicole Ison (2009) set out to respondent’s line of discussion, learn to take what conduct in-depth interviews with young people he or she has just said and branch that comment with cerebral palsy, their speech difficulties created back in the direction appropriate to your purposes. a special problem. Her solution was to conduct Most people love to talk to anyone who’s really e‑mail interviews. Even in those cases where typ- interested. Stopping their line of conversation tells ing may have been difficult, the subjects could them that you are not interested; asking them to work at their own pace, avoiding the frustration elaborate in a particular direction tells them that that would probably have attended spoken inter- you are. views. Subjects could create their responses and review them to be sure they had accurately ex- Consider this hypothetical example in which pressed their intended communications. you’re interested in why college students chose their majors. Although you may set out to conduct inter- views with a reasonably clear idea of what you You: What are you majoring in? want to ask, one of the special strengths of field research is its flexibility. In particular, the answers Resp: Engineering. evoked by your initial questions should shape your subsequent ones. It doesn’t work merely to You: I see. How did you come to choose ask preestablished questions and record the an- engineering? swers. Instead, you need to ask a question, listen carefully to the answer, interpret its meaning for Resp: I have an uncle who was voted the best your general inquiry, and then frame another engineer in Arizona in 2005. question either to dig into the earlier answer or to redirect the person’s attention to an area more You: Gee, that’s great. relevant to your inquiry. In short, you need to be able to listen, think, and talk almost at the same Resp: Yeah. He was the engineer in charge of time. d eveloping the new civic center in Tucson. It was written up in most of the engineering The discussion of probes in Chapter 8 provides journals. a useful guide to getting answers in more depth without biasing later answers. More generally, field You: I see. Did you talk to him about your be- interviewers need the skills involved in being a coming an engineer? good listener. Be more interested than interesting. Learn to say things like “How is that?” “In what Resp: Yeah. He said that he got into engineer- ways?” “How do you mean that?” “What would ing by accident. He needed a job when he be an example of that?” Learn to look and listen graduated from high school, so he went to expectantly, and let the person you’re interviewing work as a laborer on a construction job. He fill in the silence. spent eight years working his way up from the bottom, until he decided to go to college At the same time, you can’t afford to be a to- and come back nearer the top. tally passive receiver. You’ll go into your interviews with some general (or specific) questions you want You: So is your main interest civil engineering, answered and some topics you want addressed. At like your uncle, or are you more interested times you’ll need the skill of subtly directing the in some other branch of engineering? flow of conversation. Resp: Actually, I’m leaning more toward electri- cal engineering—computers, in particular. I started messing around with a Macintosh when I was in high school, and my long- term plan is . . .
Strengths and Weaknesses of Qualitative Field Research ■ 353 Like other aspects of field research (and all Or you may wish to undertake participant observa- research for that matter), proficiency comes with tion of interactions in pricey Paris nightclubs. practice. The nice thing about field research is you can begin practicing now and can continue Field research has several weaknesses as well. practicing in almost any situation. You don’t have First, being qualitative rather than quantitative, it’s to be engaged in an organized research project to not an appropriate means for arriving at statistical practice observation and recording. You might start descriptions of a large population. Observing casual by volunteering to take the minutes at committee political discussions in laundromats, for example, meetings, for example. Or just pick a sunny day on would not yield trustworthy estimates of the future campus, find a shady spot, and try observing and voting behavior of the total electorate. Neverthe- recording some specific characteristics of the people less, the study could provide important insights who pass by. You can do the same thing at a shop- into how political attitudes are formed. ping mall or on a busy street corner. Remember that observing and recording are professional skills To assess field research further, let’s focus on and, like all worthwhile skills, they improve with the issues of validity and reliability. Recall that va- practice. lidity and reliability are both qualities of measure- ments. Validity concerns whether measurements Strengths and Weaknesses actually measure what they’re supposed to rather of Qualitative Field Research than something else. Reliability, on the other hand, is a matter of dependability: If you made the same Like all research methods, qualitative field research measurement again and again, would you get the has distinctive strengths and weaknesses. As I’ve same result? Let’s see how field research stacks up already indicated, field research is especially effec- in these respects. tive for studying subtle nuances in attitudes and behaviors and for examining social processes over Validity time. As such, the chief strength of this method lies in the depth of understanding it permits. Field research seems to provide measures with Whereas other research methods may be chal- greater validity than do survey and experimen- lenged as “superficial,” this charge is seldom lodged tal measurements, which are often criticized as against field research. superficial and not really valid. Let’s review a cou- ple of field research examples to see why this is so. Flexibility is another advantage of field research. As discussed earlier, you can modify your “Being there” is a powerful technique for gain- field research design at any time. Moreover, you’re ing insights into the nature of human affairs in all always prepared to engage in field research, when- their rich complexity. Listen, for example, to what ever the occasion should arise, whereas you could this nurse reports about the impediments to pa- not as easily initiate a survey or an experiment. tients’ coping with cancer: Field research can be relatively inexpensive as Common fears that may impede the coping well. Other social science research methods may process for the person with cancer can include require costly equipment or an expensive research the following: staff, but field research typically can be undertaken by one researcher with a notebook and a pencil. —Fear of death—for the patient, and the This is not to say that field research is never expen- implications his or her death will have for sive. The nature of the research project may require significant others. a large number of trained observers, for example. Expensive recording equipment may be needed. —Fear of incapacitation—because cancer can be a chronic disease with acute episodes that may result in periodic stressful periods, the variability of the person’s ability to cope and constantly adjust may require a dependency upon others for activities of daily living and may consequently become a burden.
354 ■ Chapter 11: Paradigms, Methods, and Ethics of Qualitative Field Research —Fear of alienation—from significant set of statistics on the median number of fistfights o thers and health care givers, thereby creating occurring during a specified period. helplessness and hopelessness. These examples point to the superior validity —Fear of contagion—that cancer is of field research, as compared with surveys and ex- transmissible and/or inherited. periments. The kinds of comprehensive measure- ments available to the field researcher tap a depth —Fear of losing one’s dignity—losing con- of meaning in concepts such as common fears of trol of all bodily functions and being totally cancer patients and “toughness” (or concepts such vulnerable. as liberal and conservative) that are generally un- available to surveys and experiments. Instead of (Garant 1980: 2167) specifying concepts, field researchers commonly give detailed illustrations. Observations and conceptualizations such as these are valuable in their own right. In addition, Reliability they can provide the basis for further research— both qualitative and quantitative. Field research, however, can pose problems of reliability. Suppose you were to characterize your Now listen to what Joseph Howell has to say best friend’s political orientations according to about “toughness” as a fundamental ingredient of everything you know about him or her. Your as- life on Clay Street, a white, working-class neigh- sessment of your friend’s politics would appear to borhood in Washington, D.C.: have considerable validity; certainly it’s unlikely to be superficial. We couldn’t be sure, however, that Most of the people on Clay Street saw them- another observer would characterize your friend’s selves as fighters in both the figurative and politics the same way you did, even with the same literal sense. They considered themselves amount of observation. strong, independent people who would not let themselves be pushed around. For Bobbi, being Although they are in-depth, field research a fighter meant battling the welfare department measurements are also often very personal. How and cussing out social workers and doctors I judge your friend’s political orientation depends upon occasion. It meant spiking Barry’s beer greatly on my own, just as your judgment depends with sleeping pills and bashing him over the on your political orientation. Conceivably, then, head with a broom. For Barry it meant telling you could describe your friend as middle-of-the- off his boss and refusing to hang the door, an road, although I might feel that I’ve been observing act that led to his being fired. It meant going a fire-breathing radical. through the ritual of a duel with Al. It meant pushing Bubba around and at times getting As I suggested earlier, researchers who use rough with Bobbi. qualitative techniques are conscious of this issue and take pains to address it. Individual researchers June and Sam had less to fight about, often sort out their own biases and points of view, though if pressed they both hinted that they, and the communal nature of science means that too, would fight. Being a fighter led Ted into their colleagues will help them in that regard. Nev- near conflict with Peg’s brothers, Les into ertheless, it’s prudent to be wary of purely descrip- conflict with Lonnie, Arlene into conflict with tive measurements in field research—your own, Phyllis at the bowling alley, etc. or someone else’s. If a researcher reports that the members of a club are fairly conservative, such a (1973: 292) judgment is unavoidably linked to the researcher’s own politics. You can be more trusting of compara- Even without having heard the episodes How- tive evaluations: identifying who is more conserva- ell refers to in this passage, you have the distinct tive than who, for example. Even if you and I had impression that Clay Street is a tough place to live. That “toughness” shows far more powerfully through these field observations than it would in a
Ethics and Qualitative Field Research ■ 355 different political orientations, we would probably • Is it ethical to develop a calculated stance to- agree pretty much in ranking the relative conserva- tism of the members of a group. ward other humans, that is, to be strategic in your relations? As a means for both increasing and document- ing the trustworthiness of qualitative research, • Is it ethical to take sides or to avoid taking sides Glenn Bowen (2009) illustrates the use of an “audit trail,” which records the researcher’s deci- in a factionalized situation? sions throughout the conduct of the research and the analysis of data. Decisions on the coding of • Is it ethical to “pay” people with trade-offs for interview responses would be an example. Some computer programs for qualitative data analysis access to their lives and minds? provide for the recording of an audit trail. • Is it ethical to “use” people as allies or infor- While the audit trail is suggested to counter concerns that qualitative analysis might lack rigor, mants in order to gain entree to other people or a similar technique would be appropriate for quan- to elusive understandings? titative research. While the results of measurement decisions in designing a quantitative survey are Participation observation brings special ethical explicit in the actual wording of questionnaires, concerns with it. When you ask people to reveal the reasoning behind those decisions is not always their inner thoughts and actions to you, you may obvious. be opening them up to a degree of suffering: per- haps recalling troubling experiences, for example, As we’ve seen, field research is a potentially as in the earlier example of interviewing cancer powerful tool for social scientists, one that provides p atients. Moreover, you are also asking them to a useful balance to the strengths and weaknesses risk the public disclosure of what they have con- of experiments and surveys. Chapters 10 and 12 fided in you, and you are strictly obligated to honor of Part 3 present additional modes of observation their confidences. We have seen cases of research- available to social researchers. ers going to jail rather than reveal the private mat- ters they observed in confidence. Ethics and Qualitative Field Research Geoff Pearson (2009) examines the sticky ques- tion of how participant observers should behave As I’ve noted repeatedly, all forms of social research when studying people routinely engaged in criminal raise ethical issues. By bringing researchers into di- activities. The researcher’s refusal to join in such il- rect and often intimate contact with their subjects, legal behavior might very well alter what is being field research raises ethical concerns in a particu- studied and, in some cases, risk the researcher’s larly dramatic way. Here are some of the issues study and/or safety. On the other hand, are re- mentioned by John and Lyn Lofland (1995: 63): searchers justified in breaking the law in such cases? Obviously the severity of the crimes would affect • Is it ethical to talk to people when they do not your decisions, but when you examine such ethical questions in depth, you are likely to find yourself know you will be recording their words? entering numerous gray areas. Planning and con- ducting field research in a responsible way requires • Is it ethical to get information for your own attending to these and other ethical concerns. purposes from people you hate? Main Points • Is it ethical to see a severe need for help and Introduction not respond to it directly? • Field research involves the direct observation of so- • Is it ethical to be in a setting or situation but cial phenomena in their natural settings. Typically, field research is qualitative rather than quantitative. not commit yourself wholeheartedly to it? • In field research, observation, data processing, and analysis are interwoven, cyclical processes.
356 ■ Chapter 11: Paradigms, Methods, and Ethics of Qualitative Field Research Topics Appropriate for Field Research generally not appropriate for arriving at statistical descriptions of large populations. • Field research is especially appropriate for topics Ethics and Qualitative Field Research and processes that are not easily quantifiable, that are best studied in natural settings, or that change • Conducting field research responsibly involves over time. Among these topics are practices, epi- sodes, encounters, roles, relationships, groups, confronting several ethical issues that arise from organizations, settlements, social worlds, and life- the researcher’s direct contact with subjects. styles or subcultures. Key Terms Special Considerations in Qualitative Field Research The following terms are defined in context in the chapter and at the bottom of the page where the term • Among the special considerations involved in is introduced, as well as in the comprehensive glossary at the back of the book. field research are the various possible roles of the observer and the researcher’s relationships with case study institutional ethnography subjects. As a field researcher, you must decide emancipatory research naturalism whether to observe as an outsider or as a partici- ethnography participatory action pant, whether or not to identify yourself as a re- ethnomethodology research (PAR) searcher, and how to negotiate your relationships extended case method qualitative interview with subjects. focus group rapport grounded theory reactivity Some Qualitative Field Research Paradigms Proposing Social Research: Qualitative • Field research can be guided by any one of several Field Research paradigms, such as naturalism, ethnomethodol- This chapter has laid out a large number of different ogy, grounded theory, case studies and the ex- possibilities for conducting field research. If you’re tended case method, institutional ethnography, doing field research, you should indicate the kind of and participatory action research. study you plan to do. Will you be the sole observer in the study? If not, how will you select and train the Conducting Qualitative Field Research other observers? • Preparing for the field involves doing background Will you be a participant in the events you are observing and, if so, will you identify yourself as a research, determining how to make contact with researcher to those you are observing? You might say subjects, and resolving issues of what your rela- something about how these choices may affect what tionship to your subjects will be. you observe, as well as discussing the ethical issues involved. • Field researchers often conduct in-depth inter- In earlier exercises, you dealt with the variables views that are much less structured than those you’ll examine and the ways you’ll select informants conducted in survey research. Qualitative inter- and/or people to observe, as well as the times and viewing is more of a guided conversation than places for your observations. As this chapter has a search for specific information. Effective inter- demonstrated, there are other logistical issues to be viewing involves skills of active listening and the worked out. It may be appropriate to describe your ability to direct conversations unobtrusively. note-taking plans if that’s likely to be difficult (for example, if you’re a participant not identified as a • To create a focus group, researchers bring subjects researcher). together and observe their interactions as they ex- plore a specific topic. • Whenever possible, field observations should be recorded as they are made; otherwise, they should be recorded as soon afterward as possible. Strengths and Weaknesses of Qualitative Field Research • Among the advantages of field research are the depth of understanding it can provide, its flexibility, and (usually) its inexpensiveness. • Compared with surveys and experiments, field research measurements generally have more validity but less reliability. Also, field research is
Online Study Resources ■ 357 If you’ll be conducting in-depth interviews, you S P SS E x e r c i s e s should include an outline of the topics to be covered in those interviews. Are there topics or questions that See the booklet that accompanies your text for ex- must be addressed in each interview and others that ercises using SPSS (Statistical Package for the Social will be pursued only if appropriate? Sciences). There are exercises offered for each chap- ter, and you’ll also find a detailed primer on using Compared with experiments and surveys, field SPSS. research allows more flexibility as to the timing of the research. Depending on how things go, you may Online Study Resources find yourself concluding earlier or later than you had planned. Nevertheless, you should say something in Access the resources your instructor has assigned. For the proposal regarding the schedule you are planning. this book, you can access: Review Questions and Exercises CourseMate for The Practice of Social Research 1. Think of some group or activity you participate in or are very familiar with. In two or three para- Login to CengageBrain.com to access chapter-specific graphs, describe how an outsider might effectively learning tools including Learning Objectives, Practice go about studying that group or activity. What Quizzes, Videos, Internet Exercises, Flash Cards, Glossaries, should he or she read, what contacts should be Web Links, and more from your Sociology CourseMate. made, and so on? If your professor has assigned Aplia homework: 2. Choose any two of the paradigms discussed in 1. Sign into your account. this chapter. Then describe how your hypotheti- 2. After you complete each page of questions, click cal study from Exercise 1 might be conducted if you followed each. Compare and contrast the “Grade It Now” to see detailed explanations of way these paradigms might work in the context of every answer. your study. 3. Click “Try Another Version” for an opportunity to improve your score. 3. To explore the strengths and weaknesses of ex- Visit www.cengagebrain.com to access your account periments, surveys, and field research, choose a and purchase materials. general research area (such as prejudice, political orientation, education) and write brief descrip- tions of studies in that area that could be con- ducted using each of these three methods. In each case, explain why the chosen method is the most appropriate for the study you describe. 4. Return to the example you devised in response to Exercise 1 and list five ethical issues that you can imagine having to confront if you were to under- take your study. 5. Using InfoTrac College Edition on your Sociology CourseMate at www.cengagebrain.com, find a re- search report using the Grounded Theory Method. Summarize the study design and main findings.
Introduction ■ 359 Introduction married women who listened to the show said they now practiced family planning, compared with You may not be familiar with Twende na Wakati only 19 percent of the nonlisteners. There were (“Let’s Go with the Times”), but it was the most other impacts: popular radio show in Tanzania a few years back. It was a soap opera. The main character, Mkwaju, Some 72 percent of the listeners in 1994 said was a truck driver with some pretty traditional that they adopted an HIV/AIDS prevention ideas about gender roles and sexuality. By contrast, behavior because of listening to “Twende na Fundi Mitindo, a tailor, and his wife, Mama Waridi, Wakati,” and this percentage increased to had more-modern ideas regarding the roles of men 82 percent in our 1995 survey. Seventy-seven and women, particularly in relation to the issues of percent of these individuals adopted monogamy, overpopulation and family planning. 16 percent began using condoms, and 6 percent stopped sharing razors and/or needles. Twende na Wakati was the creation of Population Communications International (PCI) and other (Rogers et al. 1996: 21) o rganizations working in conjunction with the Tanzanian government in response to two prob- We can judge the effectiveness of the soap lems facing that country: (1) a population growth opera because of a particular form of social science. rate over twice that of the rest of the world and Evaluation research refers to a research purpose (2) an AIDS epidemic particularly heavy along rather than a specific method. This purpose is to the international truck route, where more than a evaluate the impact of social interventions such as fourth of the truck drivers and over half the com- new teaching methods or innovations in parole. mercial sex workers were found to be HIV positive Many methods—surveys, experiments, and so in 1991. The prevalence of contraceptive use was on—can be used in evaluation research. 11 percent (Rogers et al. 1996: 5–6). Evaluation research is appropriate whenever The purpose of the soap opera was to bring some social intervention occurs or is planned. about a change in knowledge, attitudes, and A social intervention is an action taken within a practices (KAP) relating to contraception and social context for the purpose of producing some family planning. Rather than instituting a conven- intended result. In its simplest sense, evaluation tional educational campaign, PCI felt it would be research is the process of determining whether more effective to illustrate the message through a social intervention has produced the intended entertainment. result. Peter Rossi, Mark Lipsey, and Howard F reeman (2002: 4) define it as follows: Between 1993 and 1995, 208 episodes of Twende na Wakati were aired, aiming at the 67 percent of Program evaluation is the use of social research pro- Tanzanians who listen to the radio. Eighty-four cedures to systematically investigate the effectiveness percent of the radio listeners reported listening of social intervention programs. More specifically, to the PCI soap opera, making it the most popu- evaluation researchers [evaluators] use social lar show in the country. Ninety percent of the research methods to study, appraise, and help show’s listeners recognized Mkwaju, the sexist improve social programs in all their important truck driver, and only 3 percent regarded him as aspects, including the diagnosis of the social a p ositive role model. Over two-thirds identified Mama Waridi, a businesswoman, and her tailor evaluation research Research undertaken for the husband as positive role models. purpose of determining the impact of some social intervention, such as a program aimed at solving a Surveys conducted to measure the impact of social problem. the show indicated it had affected knowledge, atti- tudes, and behavior. For example, 49 percent of the
CHAPTER 1 2 Evaluation Research: Types, Methods, and Issues chapter overview Introduction Types of Evaluation Research Designs Now you’re going to see one of the Topics Appropriate most rapidly growing uses of social for Evaluation Research Experimental Designs research: the evaluation of social Quasi-Experimental interventions. You’ll come away Formulating the Problem: Designs from this chapter able to judge Issues of Measurement Qualitative Evaluations whether social programs have succeeded or failed. Specifying Outcomes The Social Context Measuring Experimental Logistical Problems Contexts Use of Research Results Specifying Interventions Specifying the Population Social Indicators Research New versus Existing The Death Penalty Measures and Deterrence Operationalizing Success/ Computer Simulation Failure Ethics and Evaluation Research Aplia for The Practice of Social Research After reading, go to “Online Study Resources” at the end of this chapter for
360 ■ Chapter 12: Evaluation Research: Types, Methods, and Issues problems they address, their conceptualization It will be useful, therefore, to consider whether and design, their implementation and adminis- and how it’s actually applied. As you’ll see, the tration, their outcomes, and their efficiency. obvious implications of an evaluation research project do not necessarily affect real life. They Evaluation research is probably as old as social may become the focus of ideological, rather than research itself. Whenever people have instituted a scientific, debates. They may simply be denied out social reform for a specific purpose, they have paid of hand, for political or other reasons. Perhaps most attention to its actual consequences, even if they typically, they may simply be ignored and forgotten, have not always done so in a conscious, deliberate, left to collect dust in bookcases across the land. or sophisticated fashion. In recent years, however, the field of evaluation research has become an The chapter continues with a look at a par- increasingly popular and active research specialty, ticular resource for large-scale evaluation—social as reflected in textbooks, courses, and projects. indicators research. This type of research is also a Moreover, the growth of evaluation research points rapidly growing specialty. Essentially it involves the to a more general trend in the social sciences. As a creation of aggregated indicators of the “health” of researcher, you’ll likely be asked to conduct evalua- society, similar to the economic indicators that give tions of your own. diagnoses and prognoses of economies. The chapter then concludes with a look at the special ethical In part, the growth of evaluation research reflects concerns that arise in evaluation research. social researchers’ increasing desire to make a dif- ference in the world. At the same time, we can’t Topics Appropriate discount the influence of (1) an increase in federal for Evaluation Research requirements that program evaluations must ac- company the implementation of new programs and The topics appropriate for evaluation research are (2) the availability of research funds to fulfill those limitless. When the federal government abolished requirements. In any case, it seems clear that social the selective service system (the draft), military researchers will be bringing their skills into the real researchers began paying special attention to the world more than ever before. impact on enlistment. As individual states have liberalized their marijuana laws, researchers have This chapter looks at some of the key elements sought to learn the consequences, both for mari- in this form of social research. After considering the juana use and for other forms of social behavior. kinds of topics commonly subjected to evaluation, Do no-fault divorce reforms increase the number we’ll move through some of its main operational of divorces, and do related social problems decrease aspects: measurement, study design, and execu- or increase? Has no-fault automobile insurance tion. As you’ll see, formulating questions is as really brought down insurance policy premiums? important as answering them. Because it occurs Agencies providing foreign aid also conduct evalua- within real life, evaluation research has its own tions to determine whether the desired effects were problems, some of which we’ll examine. Logistical produced. Has the “No Child Left Behind” program problems arise from evaluation research generally improved the quality of education in America? and from its specific, technical procedures. The use Have “Just Say No” abstinence programs reduced of research results also presents certain concerns. rates of sexual activity and pregnancies among As you review reports of program evaluations, you young people? These are the kinds of questions should be especially sensitive to these issues. that evaluation research can address. Evaluation is a form of applied research—that There are many variations in the intent of eval- is, it’s intended to have some real-world effect. uation research. Needs assessment studies aim to determine the existence and extent of problems, needs assessment studies Studies that aim to typically among a segment of the population, such determine the existence and extent of problems, typically among a segment of the population, such as the elderly.
Topics Appropriate for Evaluation Research ■ 361 as the elderly. Cost-benefit studies determine First, it suggested that the apparent impact whether the results of a program can be justified by of drivers’ education was largely a matter of self- its expense (both financial and other). Monitoring selection. The kind of students who took drivers’ studies provide a steady flow of information education were less likely to have accidents and about something of interest, such as crime rates traffic violations—with or without driver training. or the outbreak of an epidemic. Sometimes the Students with high grades, for example, were more monitoring involves incremental interventions. likely to sign up for driver training, and they were Read this description of “adaptive management” also less likely to have accidents. by the Nature Conservancy, a public-interest group seeking to protect natural areas: More startling, however, was the suggestion that driver-training courses may have actually First, partners assess assumptions and set man- increased traffic accidents! The existence of drivers’ agement goals for the conservation area. Based education may have encouraged some students to on this assessment, the team takes action, then get their licenses earlier than if there were no such monitors the environment to see how it re- courses. In a study of ten Connecticut towns that sponds. After measuring results, partners refine discontinued driver training, about three-fourths their assumptions, goals and monitoring regi- of those who probably would have been licensed men to reflect what they’ve learned from past through their classes delayed getting licenses until experiences. With refinements in place, the they were 18 or older (Hilts 1981: 4). entire process begins again. As you might imagine, these results were not (2005: 3) well received by those most closely associated with driver training. This matter was complicated, Much of evaluation research is referred to as moreover, by the fact that the NHTSA study was p rogram evaluation or outcome assessment: also evaluating a new, more intensive training the determination of whether a social intervention p rogram—and the preliminary results showed that is producing the intended result. Here’s an example. the new program was effective. Some years ago, a project evaluating the n ation’s Here’s a very different example of evaluation drivers’ education programs, conducted by the Na- research. Rudolf Andorka, a Hungarian sociologist, tional Highway and Transportation Safety Admin- had been particularly interested in his country’s shift istration (NHTSA), stirred up a controversy. Philip to a market economy. Even before the dramatic Hilts (1981: 4) reported on the study’s findings: events in Eastern Europe in 1989, Andorka and his colleagues had been monitoring the nation’s For years the auto insurance industry has given “ second economy”—jobs pursued outside the large insurance discounts for children who take socialist economy. Their surveys followed the rise drivers’ education courses, because statistics and fall of such jobs and examined their impact show that they have fewer accidents. within Hungarian society. One conclusion was that “the second economy, which earlier probably The preliminary results of a new major tended to diminish income inequalities or at least study, however, indicate that drivers’ education does not prevent or reduce the incidence of cost-benefit studies Studies that determine traffic accidents at all. whether the results of a program can be justified by its expense (both financial and other). Based on an analysis of 17,500 young people in DeKalb County, Georgia (including Atlanta), the monitoring studies Studies that provide a steady preliminary findings indicated that students who flow of information about something of interest, took drivers’ education had just as many accidents such as crime rates or the outbreak of an epidemic. and traffic violations as those who didn’t take it. The study also seemed to reveal some subtle program evaluation/outcome assessment The aspects of driver training. determination of whether a social intervention is producing the intended result.
362 ■ Chapter 12: Evaluation Research: Types, Methods, and Issues improved the standard of living of the poorest part of threatens the survival of the program and perhaps the population, in the 1980s increasingly c ontributed even their jobs. to the growth of inequalities” (Andorka 1990: 111). The main problem I want to introduce, however, Whereas evaluation research is basically a has to do with the purpose of the intervention to matter of discovering whether social interven- be evaluated. The question “What is the intended tions make a difference, it is not surprising that it result of the new program?” often produced a vague is sometimes coupled with the intentions of par- response such as “Students will get an in-depth and ticipatory action research, discussed in Chapter 11. genuine understanding of mathematics, instead Since PAR has been particularly strong among of simply memorizing methods of calculations.” Australian researchers, it’s not surprising to find Fabulous! And how could we measure that “in- Wayne Miller and June Lennie (2005) speaking of depth and genuine understanding”? Often, I was “empowerment evaluation” to characterize their told that the program aimed at producing something assessment of a national school-breakfast program. that could not be measured by conventional apti- They say that this approach aims to include all tude and achievement tests. No problem there; that’s types of stakeholders—staff, funders, members to be expected when we’re innovating and being of the community—in the design and execution unconventional. What would be an unconventional of the evaluation And in the process, they intend measure of the intended result? Sometimes this that evaluation and improvement will “become a d iscussion came down to an assertion that the effects n ormal part of planning and managing programs.” of the program would be “unmeasurable.” (2005: 18) There’s the common rub in evaluation research: As you can see, the questions appropriate to eval- measuring the “unmeasurable.” Evaluation research uation research are of great practical significance: is a matter of finding out whether something is Jobs, programs, and investments as well as beliefs there or not there, whether something happened and values are at stake. Let’s now examine how or didn’t happen. To conduct evaluation research, these questions are answered—how evaluations we must be able to operationalize, observe, and are conducted. recognize the presence or absence of what is under study. Formulating the Problem: Issues of Measurement Often, outcomes can be derived from published program documents. Thus, when Edward Howard Several years ago, I headed an institutional and Darlene Norman (1981) evaluated the perfor- research office that conducted research directly mance of the Vigo County Public Library (VCPL) in relevant to the operation of the university. Often, Indiana, they began with the statement of purpose we were asked to evaluate new programs in the previously adopted by the library’s Board of Trustees. curriculum. The following description is fairly typical of the problem that arose in that context, To acquire by purchase or gift, and by recording and it points to one of the key barriers to good and production, relevant and potentially useful evaluation research. information that is produced by, about, or for the citizens of the community; Faculty members would appear at my office to say they’d been told by the university admin- To organize this information for efficient istration to arrange for an evaluation of the new delivery and convenient access, furnish the program they had permission to try. This points equipment necessary for its use, and provide to a common problem: Often the people whose assistance in its utilization; and programs are being evaluated aren’t thrilled at the prospect. For them, an independent evaluation To effect maximum use of this information toward making the community a better place in which to live through aiding the search for understanding by its citizens. (1981: 306)
Formulating the Problem: Issues of Measurement ■ 363 As the researchers said, “Everything that as young as seventh grade, or some combina- VCPL does can be tested against the Statement of tion of the dimensions of rate of use, nature P urpose.” They then set about creating operational of use, and age of user? . . . Applied research- measures for each of the purposes. ers should, to the degree possible, attempt to achieve consensus from research consumers Although “official” purposes of interventions in advance of the study (e.g., through advisory are often the key to designing an evaluation, they groups) or at least ensure that their studies are may not always be sufficient. Anna-Marie M adison able to produce data relevant to the standards (1992: 38), for example, warns that programs de- posited by all potentially interested parties. signed to help disadvantaged minorities do not al- ways reflect what the proposed recipients of the aid (Hedrick, Bickman, and Rog 1993: 27) may need and desire: In some cases you may find that the definitions The cultural biases inherent in how middle- of a problem and a sufficient solution are defined class white researchers interpret the experi- by law or by agency regulations; if so, you must ences of low-income minorities may lead to be aware of such specifications and accommo- erroneous assumptions and faulty propositions date them. Moreover, whatever the agreed-on concerning causal relationships, to invalid definitions, you must also achieve agreement on social theory, and consequently to invalid how the measurements will be made. Because program theory. Descriptive theories derived there are different possible methods for estimating from faulty premises, which have been legiti- the percentage of students “using drugs weekly,” mized in the literature as existing knowledge, for example, you’d have to be sure that all the may have negative consequences for program parties involved understood and accepted the participants. method(s) you’ve chosen. In setting up an evaluation, then, researchers Or on the other side of the coin, Yuet Wah must pay careful attention to issues of measurement. Cheung (2009) used “drug-free weeks” as the Let’s take a closer look at the types of measurements dependent variable in his evaluation of drug- that evaluation researchers must deal with. treatment programs in Hong Kong. This longi- tudinal study examined the role of positive and Specifying Outcomes negative “social capital” in determining success or failure. Positive social capital included degree of As I’ve already suggested, a key variable for evalu- family support and support from non-drug-using ation researchers to measure is the outcome, or friends, while negative social capital included stressful what is called the response variable. If a social events and association with drug-using friends. program is intended to accomplish something, we Cheung found, for example, that if recovering drug must be able to measure that something. If we users were able to establish networks of support- want to reduce prejudice, we need to be able to ive, non-drug-using friends, this made it less likely measure prejudice. If we want to increase marital that they would revert to associating with their old harmony, we need to be able to measure that. n etwork of drug users. It’s essential to achieve agreements on In the case of the Tanzanian soap opera, there definitions in advance: were several outcome measures. In part, the pur- pose of the program was to improve knowledge The most difficult situation arises when there about both family planning and AIDS. Thus, for is disagreement as to standards. For example, example, one show debunked the belief that the many parties may disagree as to what defines AIDS virus was spread by mosquitoes and could serious drug abuse—is it defined best as 15% be avoided by the use of insect repellant. Stud- or more of students using drugs weekly, 5% ies of listeners showed a reduction in that belief or more using hard drugs such as cocaine or (Rogers et al. 1996: 21). PCP monthly, students beginning to use drugs
364 ■ Chapter 12: Evaluation Research: Types, Methods, and Issues PCI also wanted to change Tanzanian attitudes control-group designs should allow you to pinpoint toward family size, gender roles, HIV/AIDS, and the effects of the program you’re evaluating. other related topics; the research indicated that the show had affected these as well. Finally, the Specifying Interventions program aimed at affecting behavior. We’ve already seen that radio listeners reported changing their Besides making measurements relevant to the behavior with regard to AIDS prevention. They outcomes of a program, researchers must measure reported a greater use of family planning as well. the program intervention—the experimental stimu- However, because there’s always the possibility of lus. In part, this measurement will be handled by a gap between what people say they do and what the assignment of subjects to experimental and con- they actually do, the researchers sought indepen- trol groups, if that’s the research design. A ssigning dent data to confirm their conclusions. a person to the experimental group is the same as scoring that person “yes” on the stimulus, and as- Tanzania’s national AIDS-control program had signment to the control group represents a score of been offering condoms free of charge to citizens. In “no.” In practice, however, it’s seldom that simple. the areas covered by the soap opera, the number of condoms given out increased sixfold between 1992 Let’s stick with the job-training example. Some and 1994. This far exceeded the increase of 1.4 times people will participate in the program; others will in the control area, where broadcasters did not carry not. But imagine for a moment what job-training the soap opera. programs are probably like. Some subjects will participate fully; others will miss a lot of sessions Measuring Experimental or fool around when they are present. So you may Contexts need measures of the extent or quality of participa- tion in the program. If the program is effective, you Measuring the dependent variables that are directly should find that those who participated fully have involved in the experimental program is only a higher employment rates than those who partici- beginning. As Henry Riecken and Robert Boruch pated less do. (1974: 120–21) point out, it’s often appropriate and important to measure those aspects of the context Other factors may further confound the ad- of an experiment researchers think might affect the ministration of the experimental stimulus. Suppose experiment. Though external to the experiment we’re evaluating a new form of psychotherapy de- itself, some variables may affect it. signed to cure sexual impotence. Several therapists administer it to subjects composing an experimental Suppose, for example, that you were conduct- group. We plan to compare the recovery rate of the ing an evaluation of a program aimed at training experimental group with that of a control group, unskilled people for employment. The primary which receives some other therapy or none at all. It outcome measure would be their success at gain- may be useful to include the names of the therapists ing employment after completing the program. treating specific subjects in the experimental group, You would, of course, observe and c alculate the because some may be more effective than others. subjects’ employment rate, but you should also If this turns out to be the case, we must find out determine what has happened to the employment/ why the treatment worked better for some thera- unemployment rates of society at large during the pists than for others. What we learn will further evaluation. A general slump in the job market develop our understanding of the therapy itself. should be taken into account in assessing what might otherwise seem a pretty low employment Specifying the Population rate for subjects. Or, if all the experimental sub- jects get jobs following the program, you should In evaluating an intervention, it’s important to consider any general increase in available jobs. define the population of possible subjects for whom Combining complementary measures with proper the program is appropriate. Ideally, all or a sample of appropriate subjects will then be assigned to
Formulating the Problem: Issues of Measurement ■ 365 experimental and control groups as warranted by on the head. However, creating our own measure the study design. Defining the population, how- will cost us the advantages to be gained from using ever, can itself involve specifying measurements. If preexisting measures. Creating good measures we’re evaluating a new form of psychotherapy, for takes time and energy, both of which could be example, it’s probably appropriate for people with saved by adopting an existing technique. Of greater mental problems. But how will “mental problems” scientific significance, measures that have been be defined and measured? The job-training pro- used frequently by other researchers carry a body gram mentioned previously is probably intended of possible comparisons that might be important to for people who are having trouble finding work, our evaluation. If the experimental therapy raises but what counts as “having trouble”? scores by an average of ten points on a standard- ized test, we’ll be in a position to compare that Beyond defining the relevant population, then, therapy with others that had been evaluated using the researcher should make fairly precise measure- the same measure. Finally, measures with a long ments of the variables considered in the definition. history of use usually have known degrees of valid- For example, even though the randomization of ity and reliability, but newly created measures will subjects in the psychotherapy study would ensure require pretesting or will be used with considerable an equal distribution of those with mild and those uncertainty. with severe mental problems into the experimental and control groups, we’d need to keep track of the Operationalizing Success/Failure relative severity of different subjects’ problems in case the therapy turns out to be effective only for Potentially one of the most taxing aspects of evalu- those with mild disorders. Similarly, we should ation research is determining whether the program measure such demographic variables as sex, age, under review succeeded or failed. The purpose of a race, and so forth in case the therapy works only foreign language program may be to help students for women, the elderly, or some other group. better learn the language, but how much better is enough? The purpose of a conjugal visit program at New versus Existing Measures a prison may be to raise morale, but how high does morale need to be raised to justify the program? In providing for the measurement of these different kinds of variables, the researcher must continually As you may anticipate, clear-cut answers to choose whether to create new measures or use questions like these almost never arrive. This ones already devised by others. If a study addresses dilemma has surely been the source of what is something that’s never been measured before, the generally called cost-benefit analysis. How much does choice is easy. If it addresses something that others the program cost in relation to what it returns in have tried to measure, the researcher will need benefits? If the benefits outweigh the cost, keep the to evaluate the relative worth of various existing program going. If the reverse, junk it. That’s simple measurement devices in terms of her or his specific enough, and it seems to apply in straightforward research situations and purpose. Recall that this is economic situations: If it costs you $20 to produce a general issue in social research that applies well something and you can sell it for only $18, there’s beyond evaluation research. Let’s briefly compare no way you can make up the difference in volume. creating new measures and using existing ones. Unfortunately, the situations faced by evaluation Creating measurements specifically for a study researchers are seldom amenable to straightforward can offer greater relevance and validity than using economic accounting. The foreign language program existing measures would. If the psychotherapy may cost the school district $100 per student, and we’re evaluating aims at a specific aspect of recov- it may raise students’ performances on tests by an ery, we can create measures that pinpoint that as- average of 15 points. Because the test scores can’t pect. We might not be able to find any standardized be converted into dollars, there’s no o bvious ground psychological measures that hit that aspect right for weighing the costs and benefits.
366 ■ Chapter 12: Evaluation Research: Types, Methods, and Issues Sometimes, as a practical matter, the criteria In summary, researchers must take measure- of success and failure can be handled through ment quite seriously in evaluation research, care- competition among programs. If a different foreign fully determining all the variables to be measured language program costs only $50 per student and and getting appropriate measures for each. However, produces an increase of 20 points in test scores, it such decisions are typically not purely scientific will undoubtedly be considered more successful ones, as we’ve seen. Evaluation researchers often than the first program—assuming that test scores must work out their measurement strategy with the are seen as an appropriate measure of the purpose people responsible for the program being evaluated. of both programs and the less expensive program It usually doesn’t make sense to determine whether has no unintended negative consequences. a program achieves Outcome X when its purpose is to achieve Outcome Y. (Realize, however, that When Connolly, Elmore, and Stein (2008) evaluation designs sometimes have the purpose of undertook a qualitative evaluation of a Jamaican testing for unintended consequences.) radio drama designed for youth, they utilized focus groups, in-depth interviews, and exercises in which There is a political aspect to these choices, also. respondents drew sketches to illustrate their answers. Because evaluation research often affects other The researchers described their aims thusly: people’s professional interests—their pet program may be halted, or they may be fired or lose profes- The purpose of the study was to assess how sional standing—the results of evaluation research listeners to the program engaged with the pro- are often argued about. gram and to what extent they found personal meaning and were influenced by the educa- Let’s turn now to some of the research designs tional messages and themes in the drama. commonly employed by evaluators. Unlike a quantitative evaluation, this Types of Evaluation report does not attempt to generalize the Research Designs findings to all Outta Road youth listeners in Jamaica. The findings do, however, provide As I noted at the start of this chapter, evaluation rich verbal and visual insights into how the research is not itself a method, but rather one program was incorporated into the lives of par- application of social research methods. As such, it ticipants, what personal meaning they derived can involve any of several research designs. Here from the content, and through reflection how we’ll consider three main types of research design youth listeners internalized the key messages that are appropriate for evaluations: experimental from the drama. designs, quasi-experimental designs, and qualita- tive evaluations. (2008: 2) Experimental Designs Ultimately, the criteria of success and failure are often a matter of agreement. The people responsi- Many of the experimental designs introduced in ble for the program may commit themselves in ad- Chapter 9 can be used in evaluation research. vance to a particular outcome that will be regarded By way of illustration, let’s see how the classical as an indication of success. If that’s the case, all you experimental model might be applied to our evalu- need to do is make absolutely certain that the re- ation of a new psychotherapy treatment for sexual search design will measure the specified outcome. I impotence. mention this obvious requirement simply because researchers sometimes fail to meet it, and there’s In designing our evaluation, we should begin by little or nothing more embarrassing than that. So, identifying a population of patients appropriate for for example, it is agreed that higher scores on the the therapy. This identification might be made by SAT is the desired result of an educational capstone researchers experimenting with the new therapy. program, you should ask “how high” and make Let’s say we’re dealing with a clinic that a lready has certain your research design includes SAT scores.
Types of Evaluation Research Designs ■ 367 100 patients being treated for sexual impotence. We some patients watch the movies eagerly but others might take that group and the clinic’s definition of look away from the screen? We’d have to ask these sexual impotence as a starting point, and we should kinds of questions and create specific measurements maintain any existing a ssessments of the severity of to address them. the problem for each specific patient. Having thus designed the study, all we have For purposes of evaluation research, however, to do is “roll ’em.” The study is set in motion, the we would need to develop a more specific measure observations are made and recorded, and the mass of impotence. Maybe it would involve whether of data is accumulated for analysis. Once the study patients have sexual intercourse at all within a has run its course, we can determine whether the specified time, how often they have intercourse, new therapy had its intended—or perhaps some or whether and how often they reach orgasm. unintended—consequences. We can tell whether A lternatively, the outcome measure might be based the movies were most effective for mild problems on the assessments of independent therapists not or severe ones, whether they worked for young involved in the therapy who interview the patients subjects but not older ones, and so forth. later. In any event, we would need to agree on the measures to be used. This simple illustration shows how the stan- dard experimental designs presented in Chapter 9 In the simplest design, we would assign the can be used in evaluation research. Many, perhaps 100 patients randomly to experimental and control most, of the evaluations reported in the research groups; the former would receive the new therapy, literature don’t look exactly like this illustration, and the latter would be taken out of therapy al- however. Because it’s nested in real life, evaluation together during the experiment. Because ethical research often calls for quasi-experimental designs. practice would probably prevent withdrawing ther- Let’s see what this means. apy altogether from the control group, however, it’s more likely that the control group would continue Quasi-Experimental Designs to receive their conventional therapy. Quasi experiments are distinguished from “true” Having assigned subjects to the experimental experiments primarily by the lack of random and control groups, we would need to agree on a ssignment of subjects to an experimental and a the length of the experiment. Perhaps the design- control group. In evaluation research, it’s often im- ers of the new therapy feel it ought to be effective possible to achieve such an assignment of subjects. within two months, and an agreement could be Rather than forgo evaluation altogether, researchers reached. The duration of the study doesn’t need to sometimes create designs that give some evaluation be rigid, however. One purpose of the experiment of the program in question. This section describes and evaluation might be to determine how long it some of these designs. actually takes for the new therapy to be effective. Conceivably, then, an agreement could be struck to Time-Series Designs measure recovery rates weekly, say, and let the ul- timate length of the experiment rest on a continual To illustrate the time-series design—which review of the results. involves measurements taken over time—I’ll begin Let’s suppose the new therapy involves show- quasi experiments Nonrigorous inquiries some- ing pornographic movies to patients. We’d need what resembling controlled experiments but lacking to specify that stimulus. How often would patients key elements such as pre- and posttesting and/or see the movies, and how long would each session control groups. be? Would they see the movies in private or in groups? Should therapists be present? Perhaps we time-series design A research design that involves should observe the patients while the movies are measurements made over some period, such as the being shown and include our observations among study of traffic accident rates before and after lower- the measurements of the experimental stimulus. Do ing the speed limit.
368 ■ Chapter 12: Evaluation Research: Types, Methods, and Issues Figure 12-1 Figure 12-2 presents three possible patterns of class participation over time, both before and after Two Observations of Class Participation: Before and After an Open the open discussion on Wednesday. Which of these Discussion patterns would give you some confidence that the discussion had the impact I contend it had? by asking you to assess the meaning of some h ypothetical data. Suppose I come to you with If the time-series results looked like the first what I say is an effective technique for getting pattern in Figure 12-2, you’d probably conclude students to participate in classroom sessions of that the process of greater class participation had a course I’m teaching. To prove my assertion, begun on the Wednesday before the discussion and I tell you that on Monday only four students had continued, unaffected, after the day devoted asked questions or made a comment in class; on to the discussion. The long-term data suggest that Wednesday I devoted the class time to an open the trend would have occurred even without the discussion of a controversial issue raging on cam- discussion on Wednesday. The first pattern, then, pus; and on Friday, when we returned to the contradicts my assertion that the special discussion subject matter of the course, eight students asked increased class participation. questions or made comments. In other words, I contend, the discussion of a controversial issue on The second pattern contradicts my assertion by Wednesday has doubled classroom participation. indicating that class participation has been bounc- This simple set of data is presented graphically in ing up and down in a regular pattern throughout Figure 12-1. the semester. Sometimes it increases from one class to the next, and sometimes it decreases; the open Have I persuaded you that the open discussion discussion on that Wednesday simply came at a on Wednesday has had the consequence I claim for time when the level of participation was about to it? Probably you’d object that my data don’t prove increase. More to the point, we note that class par- the case. Two observations (Monday and Friday) ticipation decreased again at the class following the aren’t really enough to prove anything. Ideally alleged postdiscussion increase. I should have had two classes, with students as- signed randomly to each, held an open discussion Only the third pattern in Figure 12-2 sup- in only one, and then compared the two on Friday. ports my contention that the open discussion But I don’t have two classes with random assign- mattered. As depicted there, the level of discus- ment of students. Instead, I’ve been keeping a re- sion before that Wednesday had been a steady cord of class participation throughout the semester four students per class. Not only did the level of for the one class. This record allows you to conduct participation double following the day of the dis- a time-series evaluation. cussion, but it continued to increase afterward. Although these data do not protect us against the possible influence of some extraneous fac- tor (I might also have mentioned that participa- tion would figure into students’ grades), they do exclude the possibility that the increase results from a process of maturation (indicated in the first pattern) CorefrnogmargegeulaLreflaucrtnuiantiogns (indi- cated in the sBeacobnbdi)e.: The Practice of Social Research, 13/e Nonequiva1le-1n3t3C-0o4n97t9ro-6l GroFiugp. 1s2-1 The time-series design just described involves only an “experimental” group; it doesn’t provide the value to be gained from having a control group. Sometimes, when researchers can’t create
Types of Evaluation Research Designs ■ 369 Figure 12-2 Three Patterns of Class Participation in a Longer Historical Period
370 ■ Chapter 12: Evaluation Research: Types, Methods, and Issues experimental and control groups by random as- very similar. The rate of parental smoking signment from a common pool, they can find an reported by the students was just above existing “control” group that appears similar to 40 percent in both schools. the experimental group. Such a group is called a nonequivalent control group. If an innovative (McAlister et al. 1980: 720) foreign language program is being tried in one class in a large high school, for example, you may be In the initial set of observations, the experi- able to find another foreign language class in the mental and control groups reported virtually the same school that has a very similar student popula- same (low) frequency of smoking. Over the tion: one that has about the same composition in 21 months of the study, smoking increased in both terms of grade in school, sex, ethnicity, IQ, and so groups, but it increased less in the experimental forth. The second class, then, could provide a point group than in the control group, suggesting that of comparison even though it is not formally part the program affected students’ behavior. of the study. At the end of the semester, you could give both classes the same foreign language test Multiple Time-Series Designs and then compare performances. Sometimes the evaluation of processes occurring Here’s how two junior high schools were se- outside of “pure” experimental controls can be lected for purposes of evaluating a program aimed made easier by the use of more than one time- at discouraging tobacco, alcohol, and drug use: series analysis. Multiple time-series designs are an improved version of the nonequivalent control The pairing of the two schools and their group design just described. Carol Weiss (1972: 69) a ssignment to “experimental” and “control” presents a useful example: conditions was not random. The local Lung Association had identified the school where An interesting example of multiple time se- we delivered the program as one in which ries was the evaluation of the Connecticut administrators were seeking a solution to crackdown on highway speeding. Evaluators admitted problems of smoking, alcohol, and collected reports of traffic fatalities for several drug abuse. The “control” school was chosen as periods before and after the new program went a convenient and nearby demographic match into effect. They found that fatalities went where administrators were willing to allow our down after the crackdown, but since the series surveying and breath-testing procedures. The had had an unstable up-and-down pattern for principal of that school considered the exist- many years, it was not certain that the drop was ing program of health education to be e ffective due to the program. They then compared the and believed that the onset of smoking was statistics with time-series data from four neigh- relatively uncommon among his students. The boring states where there had been no changes communities served by the two schools were in traffic enforcement. Those states registered no equivalent drop in fatalities. The comparison nonequivalent control group A control group lent credence to the conclusion that the crack- that is similar to the experimental group but is not down had had some effect. created by the random assignment of subjects. This sort of control group differs significantly from the ex- Although this study design is not as good as perimental group in terms of the dependent variable one in which subjects are assigned randomly, it’s or variables related to it. nonetheless an improvement over assessing the experimental group’s performance without any multiple time-series designs The use of more comparison. That’s what makes these designs quasi than one set of data that were collected over time, as experiments instead of just fooling around. The in accident rates over time in several states or cities, key in assessing this aspect of evaluation studies is so that comparisons can be made. c omparability, as the following example illustrates.
Types of Evaluation Research Designs ■ 371 Rural development, a growing concern in activities designed to improve their situation. the poor countries of the world, has captured the Six types were studied. “Active initiative,” for attention and support of many rich countries. e xample, was defined as “active effort to influence Through national foreign-assistance programs persons or events affecting group members ver- and through international agencies such as the sus passive response or withdrawal” (Tandon and World Bank, the developed countries are in the Brown 1981: 180). The data for evaluation came process of sharing their technological knowledge from the journals that the peer-group leaders had and skills with the developing countries. Such been keeping since their initial technological train- programs have had mixed results, however. Often, ing. The researchers read through the journals and modern techniques do not produce the intended counted the number of initiatives taken by mem- results when applied in traditional societies. bers of the peer groups. Two researchers coded the journals independently and compared their work Rajesh Tandon and L. Dave Brown (1981) to test the reliability of the coding process. undertook an experiment in which technological training would be accompanied by instruction in Figure 12-3 compares the number of active village organization. They felt it was important for initiatives by members of the two experimental poor farmers to learn how to organize and exert groups with those coming from the control groups. collective influence within their villages—getting Similar results were found for the other outcome needed action from government officials, for ex- measures. ample. Only then would their new technological skills bear fruit. Notice two things about the graph. First, there is a dramatic difference in the number of initiatives Both intervention and evaluation were attached by the two experimental groups as compared with to an ongoing program in which 25 villages had the eleven controls. This would seem to confirm been selected for technological training. Two poor the effectiveness of the special training program. farmers from each village had been trained in new Second, notice that the number of initiatives also agricultural technologies. Then they had been increased among the control groups. The research- sent home to share their new knowledge with ers explain this latter pattern as a result of conta- their village and to organize other farmers into gion. Because all the villages were near each other, “peer groups” who would assist in spreading that the lessons learned by peer-group members in the knowledge. Two years later, the authors randomly experimental groups were communicated in part to selected two of the 25 villages (subsequently called members of the control villages. Group A and Group B) for special training and 11 other untrained groups as controls. A careful This example illustrates the strengths of mul- comparison of demographic characteristics showed tiple time-series designs in situations where true the experimental and control groups to be strik- experiments are inappropriate to the program ingly similar, suggesting they were sufficiently being evaluated. comparable for the study. Qualitative Evaluations The peer groups from the two experimental villages were brought together for special training Although I’ve laid out the steps involved in tightly in organization building. The participants were structured, mostly quantitative evaluation research, given some information about organizing and evaluations can also be less structured and more making demands on the government, and they qualitative. For example, Pauline Bart and Patricia were also given opportunities to act out dramas O’Brien (1985) wanted to evaluate different ways similar to the situations they faced at home. The to stop rape, so they undertook in-depth interviews training took three days. with rape victims and with women who had suc- cessfully fended off rape attempts. As a general The outcome variables considered by the rule, they found that resistance (e.g., yelling, kick- evaluation all had to do with the extent to which ing, running away) was more likely to succeed members of the peer groups initiated group
372 ■ Chapter 12: Evaluation Research: Types, Methods, and Issues Figure 12-3 Active Initiatives over Time Source: Rajesh Tandon and L. Dave Brown,“Organization-Building for Rural Development: An Experiment in India,” Journal of Applied Behavioral Science 17, no. 2 (April 1981): 182. Copyright © 1981 by Sage Publications. Reprinted by permission of SAGE Publications. than to make the situation worse, as women some- one important piece of information: that under- times fear it will. sized babies were a bad thing. Many of the young Sometimes even structured quantitative evalu- ations can yield unexpected qualitative results. Paul women Steel interviewed thought that smaller Steel is a social researcher specializing in the evalu- ation of programs aimed at pregnant drug users. b abies would mean easier deliveries. One program he evaluated involved counseling by public-health nurses, who warned pregnant drug In another program, a local district attorney users that continued drug use would likely result in underweight babies whose skulls would be an had instituted what would generally be regarded as average of 10 percent smaller than normal. In his in-depth interviews with program participants, a progressive, enlightened program. If a pregnant however, he discovered that the program omitted drug user were arrested, she could avoid prosecu- tion if she would (1) agree to stop using drugs and (2) successfully complete a drug-rehabilitation pro- gram. Again, in-depth interviews suggested that the program did nporitnacliwplaey. sSpoepceirfiactaelloyn, SthteeelgrdoiCsucnoedvnetrhgeeda g e Learning way it did in
The Social Context ■ 373 that whenever a young woman was arrested for These experiences, along with earlier comments drug use, her fellow inmates would advise her to in previous sections, hint at the possibility of prob- get pregnant as soon as she was released on bail. lems in the actual execution of evaluation research That way, she would be able to avoid prosecution projects. Of course, all forms of research can run (personal communication, November 22, 1993). into problems, but evaluation research has a special propensity for it, as we shall now explore further. The most effective evaluation research is one that combines qualitative and quantitative com- The Social Context ponents. Making statistical comparisons is useful, and so is gaining an in-depth understanding of This section looks at some of the logistical problems the p rocesses producing the observed results—or in evaluation research and presents some observa- p reventing the expected results from appearing. tions about using evaluation research results. The social context also raises special ethical issues; we’ll The evaluation of the Tanzanian soap opera, discuss these at the end of the chapter. presented earlier in this chapter, employed several research techniques. I’ve already mentioned the Logistical Problems listener surveys and data obtained from clinics. In addition, the researchers conducted numerous In a military context, logistics refers to moving focus groups to probe more deeply into the impact supplies around—making sure people have food, the shows had on listeners. Also, content analyses guns, and tent pegs when they need them. Here, I were done on the soap opera episodes themselves use it to refer to getting subjects to do what they’re and on the many letters received from listeners. supposed to do, getting research instruments Both quantitative and qualitative analyses were d istributed and returned, and other seemingly undertaken (Swalehe et al. 1995). simple tasks. These tasks are more challenging than you might guess! The soap opera research also offers an op- portunity to see the impact of different cultures Motivating Sailors on the conduct of research. I had an opportunity to experience this firsthand when I consulted on When Kent Crawford, Edmund Thomas, and the evaluation of soap operas being planned in Jeffrey Fink (1980) set out to find a way to motivate Ethiopia. In contrast to the Western concern for “low performers” in the U.S. Navy, they found out confidentiality in social research, respondents se- just how many problems can occur. The purpose of lected for interviews in rural Ethiopian villages often the research was to test a three-pronged program took a special pride at being selected and wanted for motivating sailors who were chronically poor their a nswers broadly known in the community. performers and often in trouble aboard ship. First, a workshop was to be held for supervisory person- Or, sometimes, local researchers’ desires to nel, training them in the effective leadership of low please the client got in the way of the evaluation. performers. Second, a few supervisors would be For example, some pilot episodes were tested in selected and trained as special counselors and role focus groups to determine whether listeners would models—people the low performers could turn recognize any of the social messages being com- to for advice or just as sounding boards. Finally, municated. The results were more encouraging the low performers themselves would participate than could have been expected. When I asked in workshops aimed at training them to be more how the focus group subjects had been selected, motivated and effective in their work and in their the researcher described his introductory con- lives. The project was to be conducted aboard a versation: “We would like you to listen to some particular ship, with a control group selected from radio programs designed to encourage people to sailors on four other ships. have small families, and we’d like you to tell us whether we’ve been successful.” Not surprisingly, the small-family theme came through clearly to the focus group.
374 ■ Chapter 12: Evaluation Research: Types, Methods, and Issues To begin, the researchers reported that the su- with 8 subjects. Although the evaluation pointed to pervisory personnel were not exactly thrilled with positive results, the very small number of subjects the program. warranted caution in any generalizations from the experiment. Not surprisingly, there was considerable resis- tance on the part of some supervisors toward The special, logistical problems of evaluation dealing with these issues. In fact, their reluc- research grow out of the fact that it occurs within tance to assume ownership of the problem was the context of real life. Although evaluation re- reflected by “blaming” any of several factors search is modeled after the experiment—which that can contribute to their personnel problem. suggests that the researchers have control over The recruiting system, recruit training, parents, what happens—it takes place within frequently and society at large were named as influencing uncontrollable daily life. Of course, the participant- low performance—factors that were well observer in field research doesn’t have control over b eyond the control of the supervisors. what is observed either, but that method doesn’t strive for control. Given the objectives of evalua- (Crawford et al. 1980: 488) tion research, lack of control can create real dilem- mas for the researcher. Eventually, the reluctant supervisors came around and “this initial reluctance gave way to guarded op- Administrative Control timism and later to enthusiasm” (1980: 489). As suggested in the previous example, the The low performers themselves were even logistical details of an evaluation project often more of a problem, however. The research design fall to program administrators. Let’s suppose called for pre- and posttesting of attitudes and per- you’re evaluating the effects of a “conjugal visit” sonalities, so that changes brought about by the program on the morale of married prisoners. program could be measured and evaluated. The program allows inmates periodic visits from their spouses during which they can have sexual Unfortunately, all of the LPs (Low Performers) relations. On the fourth day of the program, a were strongly opposed to taking these so-called male prisoner dresses up in his wife’s clothes and personality tests and it was therefore concluded escapes. Although you might be tempted to as- that the data collected under these circum- sume that his morale was greatly improved by stances would be of questionable validity. Ethi- escaping, that turn of events would complicate cal concerns also dictated that we not force your study design in many ways. Perhaps the “testing” on the LPs. warden will terminate the program altogether, and where’s your evaluation then? Or, if the war- (Crawford et al. 1980: 490) den is brave, he or she may review the files of all those prisoners you selected randomly for the ex- As a consequence, the researchers had to rely perimental group and veto the “bad risks.” There on interviews with the low performers and on the goes the comparability of your e xperimental and judgments of supervisors for their measures of at- control groups. As an alternative, stricter security titude change. The subjects continued to present measures may be introduced to prevent further problems, however. escapes, but the security measures may have a dampening effect on morale. So the experimen- Initially, the ship’s command ordered 15 low tal stimulus has changed in the middle of your performers to participate in the experiment. Of research project. Some of the data will reflect the 15, however, one went into the hospital, an- the original stimulus; other data will reflect the other was assigned duties that prevented partici- modification. Although you’ll probably be able to pation, and a third went “over the hill” (absent sort it all out, your carefully designed study has without leave). Thus, the experiment began with become a logistical snake pit. 12 subjects. But before it was completed, three more subjects completed their tour of duty and left the Navy, and another was thrown out for disci- plinary reasons. The experiment concluded, then,
The Social Context ■ 375 Or suppose you’ve been engaged to evaluate and obvious policy implications—that were ig- the effect of race-relations lectures on prejudice nored, as Research in Real Life feature “The Impact in the army. You’ve carefully studied the soldiers of ‘Three Strikes’ Laws” illustrates. available to you for study, and you’ve randomly assigned some to attend the lectures and others to There are three important reasons why the stay away. The rosters have been circulated weeks implications of the evaluation research results are in advance, and at the appointed day and hour, not always put into practice. First, the implications the lectures begin. Everything seems to be going may not always be presented in a way that the smoothly until you begin processing the files: The nonresearchers can understand. Second, evalu- names don’t match. Checking around, you discover ation results sometimes contradict deeply held that military field exercises, KP duty, and a variety beliefs. People thought Copernicus was crazy of emergencies required some of the experimental when he said the earth revolved around the sun. subjects to be elsewhere at the time of the lectures. Anybody could tell the earth was standing still. That’s bad enough, but then you learn that help- The third barrier to the use of evaluation results is ful commanding officers sent others to fill in for vested interests. If I’ve devised a new rehabilitation the missing soldiers. And whom do you suppose program that I’m convinced will keep ex-convicts they picked to fill in? Soldiers who didn’t have from returning to prison, and if people have taken anything else to do or who couldn’t be trusted to to calling it “The Babbie Plan,” how do you think do anything important. You might learn this bit of I’m going to feel when your evaluation suggests information a week or so before the deadline for the program doesn’t work? I might apologize for submitting your final report on the impact of the misleading people, fold up my tent, and go into race-relations lectures. another line of work. More likely, I’d call your re- search worthless and begin intense lobbying with These are some of the logistical problems con- the appropriate authorities to have my program fronting evaluation researchers. You need to be continue. familiar with the problems to understand why some research procedures may not measure up to In the earlier example of the evaluation of the design of the classical experiment. As you read drivers’ education, Philip Hilts reported some of the reports of evaluation research, however, you’ll find reactions to the researchers’ preliminary results: that—my earlier comments notwithstanding—it is possible to carry out controlled social research in Ray Burneson, traffic safety specialist with conjunction with real-life experiments. the National Safety Council, criticized the study, saying that it was a product of a group Use of Research Results (NHTSA) run by people who believe “that you can’t do anything to train drivers. You can only One more facts-of-life aspect of evaluation research improve medical facilities and build stronger concerns how evaluations are used. Because the cars for when the accidents happen. . . . This purpose of evaluation research is to determine the knocks the whole philosophy of education.” success or failure of social interventions, you might think it reasonable that a program would auto- (1981: 4) matically be continued or terminated based on the results of the research. By its nature, evaluation research takes place in the midst of real life, affecting it and being affected Reality isn’t that simple and reasonable, how- by it. Here’s another example, well known to social ever. Other factors intrude on the assessment of researchers. evaluation research results, sometimes blatantly and sometimes subtly. Undoubtedly every evalu- Rape Reform Legislation ation researcher can point to studies he or she conducted—studies providing clear research results For years, many social researchers and other observers have noted certain problems with the prosecution of rape cases. All too often, it is felt, the victim ends up suffering almost as much on the
Search
Read the Text Version
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
- 18
- 19
- 20
- 21
- 22
- 23
- 24
- 25
- 26
- 27
- 28
- 29
- 30
- 31
- 32
- 33
- 34
- 35
- 36
- 37
- 38
- 39
- 40
- 41
- 42
- 43
- 44
- 45
- 46
- 47
- 48
- 49
- 50
- 51
- 52
- 53
- 54
- 55
- 56
- 57
- 58
- 59
- 60
- 61
- 62
- 63
- 64
- 65
- 66
- 67
- 68
- 69
- 70
- 71
- 72
- 73
- 74
- 75
- 76
- 77
- 78
- 79
- 80
- 81
- 82
- 83
- 84
- 85
- 86
- 87
- 88
- 89
- 90
- 91
- 92
- 93
- 94
- 95
- 96
- 97
- 98
- 99
- 100
- 101
- 102
- 103
- 104
- 105
- 106
- 107
- 108
- 109
- 110
- 111
- 112
- 113
- 114
- 115
- 116
- 117
- 118
- 119
- 120
- 121
- 122
- 123
- 124
- 125
- 126
- 127
- 128
- 129
- 130
- 131
- 132
- 133
- 134
- 135
- 136
- 137
- 138
- 139
- 140
- 141
- 142
- 143
- 144
- 145
- 146
- 147
- 148
- 149
- 150
- 151
- 152
- 153
- 154
- 155
- 156
- 157
- 158
- 159
- 160
- 161
- 162
- 163
- 164
- 165
- 166
- 167
- 168
- 169
- 170
- 171
- 172
- 173
- 174
- 175
- 176
- 177
- 178
- 179
- 180
- 181
- 182
- 183
- 184
- 185
- 186
- 187
- 188
- 189
- 190
- 191
- 192
- 193
- 194
- 195
- 196
- 197
- 198
- 199
- 200
- 201
- 202
- 203
- 204
- 205
- 206
- 207
- 208
- 209
- 210
- 211
- 212
- 213
- 214
- 215
- 216
- 217
- 218
- 219
- 220
- 221
- 222
- 223
- 224
- 225
- 226
- 227
- 228
- 229
- 230
- 231
- 232
- 233
- 234
- 235
- 236
- 237
- 238
- 239
- 240
- 241
- 242
- 243
- 244
- 245
- 246
- 247
- 248
- 249
- 250
- 251
- 252
- 253
- 254
- 255
- 256
- 257
- 258
- 259
- 260
- 261
- 262
- 263
- 264
- 265
- 266
- 267
- 268
- 269
- 270
- 271
- 272
- 273
- 274
- 275
- 276
- 277
- 278
- 279
- 280
- 281
- 282
- 283
- 284
- 285
- 286
- 287
- 288
- 289
- 290
- 291
- 292
- 293
- 294
- 295
- 296
- 297
- 298
- 299
- 300
- 301
- 302
- 303
- 304
- 305
- 306
- 307
- 308
- 309
- 310
- 311
- 312
- 313
- 314
- 315
- 316
- 317
- 318
- 319
- 320
- 321
- 322
- 323
- 324
- 325
- 326
- 327
- 328
- 329
- 330
- 331
- 332
- 333
- 334
- 335
- 336
- 337
- 338
- 339
- 340
- 341
- 342
- 343
- 344
- 345
- 346
- 347
- 348
- 349
- 350
- 351
- 352
- 353
- 354
- 355
- 356
- 357
- 358
- 359
- 360
- 361
- 362
- 363
- 364
- 365
- 366
- 367
- 368
- 369
- 370
- 371
- 372
- 373
- 374
- 375
- 376
- 377
- 378
- 379
- 380
- 381
- 382
- 383
- 384
- 385
- 386
- 387
- 388
- 389
- 390
- 391
- 392
- 393
- 394
- 395
- 396
- 397
- 398
- 399
- 400
- 401
- 402
- 403
- 404
- 405
- 406
- 407
- 408
- 409
- 410
- 411
- 412
- 413
- 414
- 415
- 416
- 417
- 418
- 419
- 420
- 421
- 422
- 423
- 424
- 425
- 426
- 427
- 428
- 429
- 430
- 431
- 432
- 433
- 434
- 435
- 436
- 437
- 438
- 439
- 440
- 441
- 442
- 443
- 444
- 445
- 446
- 447
- 448
- 449
- 450
- 451
- 452
- 453
- 454
- 455
- 456
- 457
- 458
- 459
- 460
- 461
- 462
- 463
- 464
- 465
- 466
- 467
- 468
- 469
- 470
- 471
- 472
- 473
- 474
- 475
- 476
- 477
- 478
- 479
- 480
- 481
- 482
- 483
- 484
- 485
- 486
- 487
- 488
- 489
- 490
- 491
- 492
- 493
- 494
- 495
- 496
- 497
- 498
- 499
- 500
- 501
- 502
- 503
- 504
- 505
- 506
- 507
- 508
- 509
- 510
- 511
- 512
- 513
- 514
- 515
- 516
- 517
- 518
- 519
- 520
- 521
- 522
- 523
- 524
- 525
- 526
- 527
- 528
- 529
- 530
- 531
- 532
- 533
- 534
- 535
- 536
- 537
- 538
- 539
- 540
- 541
- 542
- 543
- 544
- 545
- 546
- 547
- 548
- 549
- 550
- 551
- 552
- 553
- 554
- 555
- 556
- 557
- 558
- 559
- 560
- 561
- 562
- 563
- 564
- 565
- 566
- 567
- 568
- 569
- 570
- 571
- 572
- 573
- 574
- 575
- 576
- 577
- 578
- 579
- 580
- 581
- 582
- 583
- 584
- 585
- 586
- 587
- 588
- 589
- 590
- 591
- 592
- 593
- 594
- 595
- 596
- 597
- 598
- 599
- 600
- 601
- 602
- 603
- 604
- 605
- 606
- 607
- 608
- 609
- 1 - 50
- 51 - 100
- 101 - 150
- 151 - 200
- 201 - 250
- 251 - 300
- 301 - 350
- 351 - 400
- 401 - 450
- 451 - 500
- 501 - 550
- 551 - 600
- 601 - 609
Pages:
