Research design TABLE 14.2 COMPARING RELIABILITY IN QUANTITATIVE AND QUALITATIVE RESEARCH Bases of reliability in quantitative research Bases of reliability in qualitative research Reliability ←→ Dependability Demonstrability ←→ Trustworthiness Stability ←→ Stability Isolation, control and manipulation of required Fidelity to the natural situation and real life variables ←→ Identification, control and manipulation of key Thick description and high detail on required or important variables ←→ aspects Singular, objective truths ←→ Multiple interpretations/perceptions Replicability ←→ Replicability Parallel forms ←→ Parallel forms Generalizability ←→ Generalizability Context-freedom ←→ Context-specificity Objectivity ←→ Authenticity Coverage of domain ←→ Comprehensiveness of situation Verification of data and analysis ←→ Honesty and candour Answering research questions ←→ Depth of response Meaningfulness to the research ←→ Meaningfulness to respondents Parsimony ←→ Richness Objectivity ←→ Confirmability Fidelity to ‘etic’ research ←→ Fidelity to ‘emic’ research Internal consistency ←→ Credibility Generalizability ←→ Transferability Parallel forms ←→ Parallel forms Inter-rater reliability ←→ Inter-rater reliability Accuracy ←→ Accuracy Precision ←→ Accuracy Replication ←→ Replication Neutrality ←→ Multiple interests represented Consistency ←→ Consistency Theoretical relevance ←→ Applicability Triangulation ←→ Triangulation Alternative forms (equivalence) Split-half Inter-item correlations (alphas) of bias are: the characteristics of the interviewer and the Studies have also shown that ethnicity, religion, respondent; and the substantive content of the questions. gender, sexual orientation, status, social class and age Researcher bias (Maxwell, 2005, p. 108), which has an in certain contexts can be potent sources of bias, i.e. effect on the interview (e.g. reactivity), can include, for interviewer effects (Lee, 1993; Scheurich, 1995). Inter- example: viewers and interviewees alike bring their own, often unconscious experiential and biographical baggage OO the attitudes, opinions and expectations of the with them into the interview situation. Indeed Hitch- interviewer; cock and Hughes (1989) argue that because interviews are interpersonal, humans interacting with humans, it OO a tendency for the interviewer to see the respondent is inevitable that the researcher will have some influ- in her own image; ence on the interviewee and, thereby, on the data. Interviewer neutrality is a chimera (Denscombe, 1995, OO a tendency for the interviewer to seek answers that 2014). support her preconceived notions or theory; Lee (1993) indicates problems of reliability in con- ducting interviews very sharply, where the researcher is OO misperceptions on the part of the interviewer of researching sensitive subjects, i.e. research that might what the respondent is saying; OO misunderstandings on the part of the respondent of what is being asked. 272
Validity and reliability pose a significant threat to interviewers and interview- OO poor rapport between interviewer and interviewee; ees. Here the interview might be seen as an intrusion OO changes to question wording (e.g. in attitudinal and into private worlds, or the interviewer might be regarded as someone who can impose sanctions on the factual questions); interviewee, or as someone who can exploit the power- OO poor prompting and biased probing; less; the interviewee is in the interviewer’s searchlight OO poor use and management of support materials (e.g. (see also Scheurich, 1995), so may be cautious in what is revealed. Indeed Gadd (2004) reports that an inter- show cards); viewee may reduce his/her willingness to ‘open up’ to OO alterations to the sequence of questions; an interviewer if the dynamics of the interview situa- OO inconsistent coding of responses; tion are too threatening, taking the role of the ‘defended OO selective or interpreted recording of data/transcripts; subject’. This raises issues of transference and coun- OO poor handling of difficult interviews. ter‑transference, which have their basis in psychoanal- ysis. In transference, interviewees project onto the One can add to this the issue of ‘acquiescence’ (Break- interviewer their feelings, fears, desires, needs and atti- well, 2000, p. 254), the tendency of respondents to say tudes that derive from their own experiences (Scheu- ‘yes’, regardless of the question or, indeed, regardless rich, 1995). In counter‑transference the process is of what they really feel or think. reversed. Both affect reliability. There is also the issue of leading questions. A One way of addressing reliability is to have a highly leading question is one which makes assumptions about structured interview, with the same format and interviewees or ‘puts words into their mouths’, i.e. sequence of words and questions for each respondent where the question influences the answer perhaps ille- (Silverman, 1993), though Scheurich (1995, pp. 241–9) gitimately. For example (Morrison, 1993, pp. 66–7), suggests that this is to misread the infinite complexity the question ‘when did you stop complaining to the and open-e ndedness of social interaction. Controlling headteacher?’ assumes that the interviewee had been a the wording is no guarantee of controlling the inter- frequent complainer, and the question ‘how satisfied view. Oppenheim (1992, p. 147) argues that wording is are you with the new Mathematics scheme?’ assumes a a particularly important factor in attitudinal rather than degree of satisfaction with the scheme. The leading factual questions. He suggests that changes in wording, questions here might be rendered less leading by context and emphasis undermine reliability, because it rephrasing, for example: ‘how frequently do you have ceases to be the same question for each respondent. conversations with the headteacher?’ and ‘what is your Indeed he argues that error and bias can stem from opinion of the new Mathematics scheme?’ respectively. alterations to wording, procedure, sequence, recording In discussing the issue of leading questions we are and rapport, and that training for interviewers is essen- not necessarily suggesting that there is not a place for tial to minimize this. Silverman (1993) suggests that it them. Indeed Kvale (1996, p. 158) makes a powerful is important for each interviewee to understand the case for leading questions, arguing that they may be question in the same way. He suggests that the reliabil- necessary in order to obtain information that the inter- ity of interviews can be enhanced by: careful piloting viewer suspects the interviewee might be withholding. of interview schedules; training of interviewers; Here it might be important to put the ‘burden of denial’ inter‑rater reliability in the coding of responses; and the onto the interviewee (e.g. ‘when did you stop cheating extended use of closed questions. in examinations?’). Leading questions, frequently used On the other hand, Silverman (1993) argues for the in police interviews, may be used for reliability checks importance of open-ended interviews, as this enables with what the interviewee has already said, or may be respondents to demonstrate their unique way of looking deliberately used to elicit particular non-verbal behav- at the world – their definition of the situation. It recog- iours that provide an indication of the sensitivity of the nizes that what is a suitable sequence of questions for interviewee’s remarks. one respondent might be less suitable for another, and The researcher must also be aware of possible bias open‑ended questions enable important but unantici- in interviewees giving what they think are socially pated issues to be raised. desirable answers to questions (Fowler, 2009), i.e. Oppenheim (1992, pp. 96–7) suggests several causes answers to please the interviewer or not to appear dif- of bias in interviewing: ferent from what is socially acceptable or desirable. Reducing bias in interviews requires: (a) careful for- OO biased sampling (sometimes created by the mulation of questions so that the meaning is crystal researcher not adhering to sampling instructions); clear; (b) thorough training procedures so that an inter- viewer is more aware of the possible problems; (c) probability sampling of respondents; and (d) matching 273
Research design interviewer characteristics with those of the sample game-like connotations. He suggests that powerful being interviewed (where appropriate). Oppenheim people control the agenda and course of the interview, (1992, p. 148) argues, for example, that interviewers and are usually very adept at this because they have seeking attitudinal responses have to ensure that people both a personal and professional investment in being with known characteristics are included in the sample – interviewed (see also Batteson and Ball, 1995; Phillips, the criterion group. Researchers must recognize that the 1998; Walford, 2012). interview is a shared, negotiated and dynamic social The effect of power can be felt even before the inter- moment. view commences. Neal (1995) instances being kept Power is significant in the interview situation, for waiting, and subsequently being interrupted, patronized the interview is not simply a data-collection situation and interviewed by the interviewee (see also Walford, but a social and frequently a political situation. Liter- 1994b, 2012). Scheurich (1995) suggests that many ally the word ‘inter-view’ is a view between people, powerful interviewees will rephrase or not answer the mutually, not the interviewer alone, extracting data one question. Limerick et al. (1996) report interviews in way from the interviewee. Power resides with inter- which interviewers have felt themselves to be passive, viewer and interviewee alike (Thapar-Björkert and vulnerable, helpless and indeed manipulated. One way Henry, 2004), though Scheurich (1995, p. 246) argues of overcoming this is to have two interviewers conduct- that, typically, more power resides with the interviewer ing each interview (Walford, 1994c, p. 227). On the (see also Lee, 1993; Morrison, 2013a): the interviewer other hand, Hitchcock and Hughes (1989) observe that generates the questions and the interviewee answers if the researchers are known to the interviewees and them; the interviewee is under scrutiny whilst the inter- they are peers, however powerful, then a degree of reci- viewer is not. Kvale (1996, p. 126), too, suggests that procity might be taking place, with interviewees giving there are definite asymmetries of power as the inter- answers that they think the researchers might want viewer tends to define the situation, the topics and the to hear. course of the interview. Of course, the interviewee is The issue of power features in feminist research powerful as he/she has data that the interviewer wants, (e.g. Thapar-B jörkert and Henry, 2004), i.e. research and has power to withhold such data (discussed below). which emphasizes subjectivity, equality, reciprocity, Cassell (cited in Lee, 1993) and Walford (2012) collaboration, non-h ierarchical relations and emancipa- suggest that elites and powerful people might feel tory potential (catalytic and consequential validity) demeaned or insulted when being interviewed by those (Neal, 1995), echoing the comments on research that is with a lower status or less power. Further, those with influenced by critical theory (Chapter 3). Here feminist power, resources and expertise might be anxious to research addresses a dilemma of interviews which are maintain their reputation, and so will be more guarded constructed in the dominant, male paradigm of pitching in what they say, wrapping this up in well-chosen, questions that demand answers from a passive articulate phrases (Walford, 2012). Interviewers need respondent. to be aware of the potentially distorting effects of Limerick et al. (1996) suggest that, in fact, it is power, a significant feature of critical theory (see wiser to regard the interview as a gift (and ‘data’ means Chapter 3). ‘things that are given’), as interviewees have the power Neal (1995) draws attention to the feelings of pow- to withhold information, to choose the location of the erlessness and anxieties about the physical presentation interview, to choose how seriously to attend to the and status of interviewers when interviewing powerful interview, how long it will last, when it will take place, people. This is particularly so for frequently lone, low- what will be discussed – and in what and whose terms status research students interviewing powerful people; – what knowledge is important, even how the data will a low-s tatus female research student might find that an be analysed and used (see also Thapar-B jörkert and interview with a male in a position of power (e.g. a uni- Henry, 2004). Echoing Foucault, they argue that power versity vice‑chancellor, a senior politician or a senior is fluid and is discursively constructed through the manager) might turn out to be very different from an interview rather than being the province of either party. interview with the same person if conducted by a male Miller and Cannell (1997) identify particular prob- university professor, which is perceived by the inter- lems in conducting telephone interviews, where reduc- viewee to be more of a dialogue between equals (see ing the interview situation to just auditory sensory cues also Connell et al., 1996; Gewirtz and Ozga, 1993, can be challenging (see Chapter 25). For example, the 1994). Ball (1994b) comments that, when powerful interviewee can only retain a certain amount of infor- people are being interviewed, interviews must be mation in her/his short-term memory, so bombarding seen as an extension of the ‘play of power’ – with its the telephone interviewee with too many choices (the 274
Validity and reliability non-w ritten form of ‘show cards’ of possible responses) In educational circles interviewing might be a partic- becomes unworkable. Here the reliability of responses ular problem in working with children (Morrison, 2013a) is subject to the memory capabilities of the interviewee (see also Chapters 13 and 25). Simons (1982), McCor- – how many scale points and descriptors, for example, mick and James (1988) and Greig and Taylor (1999) can an interviewee retain in her head about a single comment on particular problems involved in interview- item? Further, the absence of non-verbal cues, for ing children which affect reliability, for example: example, facial expression, gestures, posture, the sig- nificance of silences and pauses (Robinson, 1982), is OO establishing trust; important, as interviewees may be unclear about the OO overcoming shyness and reticence; meaning behind words and statements. This problem is OO maintaining informality; compounded if the interviewer is unknown to the OO avoiding assuming that children ‘know the interviewee. Miller and Cannell (1997) report important research answers’; evidence to support the significance of the non-verbal OO overcoming the problems of inarticulate children; mediation of verbal dialogue. As discussed earlier, the OO pitching the question at the right level; interview is a social situation; in telephone interviews OO choice of vocabulary; the absence of essential social elements could under- OO non-verbal cues; mine the salient conduct of the interview, and hence its OO moving beyond the institutional response or receiv- reliability and validity. Non-v erbal paralinguistic cues affect the conduct, pacing and relationships in the inter- ing what children think the interviewer wants view and the support, threat and confidence felt by the to hear; interviewees. Telephone interviews can easily slide into OO avoiding the interviewer being seen an authority spy becoming mechanical and cold. Further, the problem of or plant; loss of non-v erbal cues is compounded by the asym- OO keeping to the point; metries of power that often exist between interviewer OO breaking silences on taboo areas and those which and interviewee; the interviewer will need to take are reinforced by peer-g roup pressure; immediate steps to address these issues (e.g. by putting OO children being seen as of lesser importance than interviewees at their ease, as the interviewee might adults (maybe in the sequence in which interviews simply put down the telephone). are conducted, e.g. the headteacher, then the teach- On the other hand, Nias (1991), Miller and Cannell ing staff, then the children). (1997) and Ary et al. (2002) suggest that the very fact that interviews are not face-to-face may strengthen their These are not new matters. Studies by Labov in the reliability, as the interviewee might disclose informa- 1960s and 1970s showed how students reacted very tion that may not be so readily forthcoming in a face- strongly to contextual matters in an interview situation to-face, more intimate situation. Hence, telephone (Labov, 1969). The language of children varied accord- interviews have their strengths and weaknesses; their ing to the ethnicity of the interviewee, the friendliness of use should be governed by the criterion of fitness-for- the surroundings, the opportunity for the children to be purpose. They tend to be shorter, more focused and interviewed with friends, the ease with which the scene useful for contacting busy people (Harvey, 1988; was set for the interview, the demeanour of the adult (e.g. Miller, 1995). whether the adults was standing or sitting), the nature of A cluster of problems surround the person being the topics covered. The interview is a social encounter, interviewed. Tuckman (1972), for example, observed and children may be very sensitive to the social context that, when formulating questions, an interviewer has to of the interview (Morison et al., 2000; Morrison, 2013a); consider the extent to which a question might influence Maguire (2005, p. 4) suggests that ‘children have good the respondent to show herself in a good light; or the social radar’. The differences can be significant, varying extent to which a question might influence the respond- from monosyllabic responses by children in unfamiliar ent to be unduly helpful by attempting to anticipate and uncongenial surroundings to extended responses in what the interviewer wants to hear; or the extent to the more congenial and less threatening surroundings – which a question might be asking for information about more sympathetic to the children’s everyday world. The a respondent that she is not certain or likely to know language, argot and jargon, social and cultural factors of herself. Interviews may be based on the assumption the interviewer and interviewee all exert a powerful that the person interviewed has insight into the cause of influence on the interview situation. her behaviour, and this may not be possible. Lee (1993) raises the further issue of whether there should be a single interview that maintains the detach- ment of the researcher (perhaps particularly useful in 275
Research design addressing sensitive topics), or whether there should be method of recording replies. One way is to summarize repeated interviews to gain depth and to show fidelity responses in the course of the interview. This has the dis- to the collaborative nature of research (a feature, as advantage of breaking the continuity of the interview and noted above, which is significant for feminist research may result in bias because the interviewer may uncon- (Oakley, 1981)). sciously emphasize responses that agree with her expec- Kvale (1996, pp. 148–9) suggests that, in order to tations and fail to note those that do not. It is sometimes obtain reliable and valid data, a skilled interviewer possible to summarize an individual’s responses at the should: end of the interview. Although this preserves the conti- nuity of the interview, it is likely to induce greater bias OO know his/her subject matter in order to conduct an because the delay may lead to the interviewer forgetting informed conversation; some of the details. It is these forgotten details that are most likely to be those which disagree with his or her OO structure the interview well, so that each stage of the own expectations. We advise the reader also to review interview is clear to the participant; Chapter 25 of the present volume. OO be clear in the terminology and coverage of the 14.12 Validity and reliability in material; experiments OO allow participants to take their time and answer in One fundamental purpose of experimental design is to their own way; impose control over conditions that would otherwise cloud the true effects of the independent variables on OO be sensitive and empathic, using active listening and the dependent variables, so that causality can be attrib- being sensitive to how something is said and the uted to the intervention in question. Clouding condi- non-v erbal communication involved; tions that threaten to jeopardize the validity of experiments have been identified by Campbell and OO be alert to those aspects of the interview which may Stanley (1963), Bracht and Glass (1968), Lewis-Beck hold significance for the participant; (1993), Shadish et al. (2002) and Torgerson and Torg- erson (2008), conditions that are of greater consequence OO keep to the point and the matter in hand, steering the to the validity of quasi-e xperiments (more typical in interview where necessary in order to address this; educational research) than to true experiments in which random sampling and assignment to treatments occurs OO check the reliability, validity and consistency of and where both treatment and measurement can be responses by well-placed questioning; more adequately controlled by the researcher. The following summaries distinguish between OO be able to recall and refer to earlier statements made ‘internal validity’ and ‘external validity’. Internal valid- by the participant; ity is concerned with the question, do the experimental treatments, in fact, make a difference in the specific OO be ready to clarify, confirm and modify the partici- experiments under scrutiny? External validity, on the pant’s comments with the participant. other hand, asks the question, given these demonstrable effects, to what populations or settings can they be Walford (1994c, 2012) adds to this the need for the generalized? interviewer to have done her homework when inter- Threats to internal validity were introduced earlier viewing powerful people, as such people could well in this chapter, and comprise: interrogate the interviewer – they will assume up-to- dateness, competence and knowledge in the inter- OO history viewer. Powerful interviewees are usually busy people OO maturation and will expect the interviewer to have read relevant OO statistical regression material in the public domain. OO testing The issues of reliability do not reside solely in the OO instrumentation preparations for and conduct of the interview; they OO selection extend to the ways in which interview data are ana- OO experimental mortality lysed. For example, Lee (1993) and Kvale (1996, OO instrument reactivity p. 163) comment on the issue of ‘transcriber selectiv- OO selection–maturation interaction. ity’. Here transcripts of interviews, however detailed and full they might be, remain selective, since they are interpretations of social situations. They become decon- textualized, abstracted, even if they record silences, intonation, non-v erbal behaviour etc. The issue, then, is how useful they are to researchers overall rather than whether they are completely reliable. One problem in using open-e nded questions in inter- views is that of developing a satisfactory, reliable 276
Validity and reliability Shadish et al. (2002), Torgerson and Torgerson (2008) controlled trials, with its focus on average effects and and Creswell (2012) add to these: (a) contamination, in its risk of overlooking the importance of outliers that control and experimental groups may communicate and sub-groups. This suggests that, to ensure validity with each other, affecting what happens with each and reliability in educational experiments, attention group; and (b) compensatory rivalry: a control group must be paid to: the whole person; contexts and other may feel resentful about being deprived of the interven- interventions and practices that are taking place in the tion (if they are told what the intervention comprises) educational experiences of students at the time of the and this may affect their behaviour. experiment; duration, intensity and differential inputs Several threats to external validity were discussed into, operations of and responses to the intervention; earlier in this chapter (Section 14.5) and the reader is and to the often non-linear and dynamical systems advised to review these. As in clinical trials, educa- nature of the ‘careers’ of participants. tional experiments require attention to the educational An experiment can be said to be internally valid to equivalents of: effects on different sub-groups of a the extent that, within its own confines, its results are sample; side effects; contra-indications; effects of per- credible; but for those results to be useful, they must be sonal characteristics of participants; dose‑response (e.g. generalizable beyond the confines of the particular attention to amount, quality, strength, frequency, inten- experiment; in a word, they must be externally valid sity, duration of the intervention); recognition that a also (for a critique of randomized controlled experi- person is a complex system which combines and con- ments and the problems of generalizability, see Morri- nects very many elements whose interactions and out- son, 2001; Cartwright and Hardie, 2012). Without comes change over time (with commensurate changes internal validity an experiment cannot be externally to interventions over time); patients who forget to, or valid, but the converse does not necessarily follow; an refuse to, take medicine or take it irregularly (i.e. internally valid experiment may or may not have exter- changes to the intervention affect reliability); comor- nal validity. bidity (other problems may exist at the time of the It follows, then, that the way to reliable and valid intervention); patients taking multiple medications educational experimentation lies in maximizing both (several other events may be happening at the same internal validity and, where relevant, external validity. time as the intervention); and doctor–patient (teacher– student) relations. 14.13 Validity and reliability in In clinical trials, some treatments may require initial questionnaires intensive medication, tailing off to a maintenance level (e.g. many people start antibiotics with a double dose Questionnaires feature in many types of research, from and then maintain a single dose for several days); some surveys to action research. Validity of questionnaires medication may require a gentle start, with an increas- (however administered, e.g. face-to-face, postal, tele- ing, cumulative dosage as the body responds to treat- phone, Internet) can be seen from two viewpoints ment. The equivalents in educational research also (Belson, 1986). First, whether respondents who com- apply. Further, some drugs may require ongoing, plete questionnaires do so accurately, honestly and cor- regular, very close monitoring, whilst others require rectly; and second, whether those who fail to return less frequent monitoring. The point here is that medica- their questionnaires would have given the same distri- tion is not a single event but, as the career of a disease bution of answers as did the returnees. and a patient changes over time, so does the treatment. As more and more questionnaires are conducted Translating these practices from clinical research to online, this brings with it the problem of honesty: are educational research suggests that educational experi- respondents really telling the truth about themselves ments need greater sensitivity to many confounding and about the matters to which they have been asked to factors and to the need to address many kinds of threats respond? Fowler (2009) gives the example of people to validity and reliability. under-reporting how many cigarettes they smoke each Though the analogy between clinical trials and edu- day or how much alcohol they drink (p. 16). Fowler cational experiments may not hold too strongly, for also reports that respondents may not understand or example, the former operating on a one-to-one patient– may misunderstand a question, or they may not know doctor relation and the latter typically operating on a the answer, or they may not be able to recall accurately, one-to-many basis, and the former adopting a patholog- or they may not want to disclose information, or they ical model and the latter adopting a non-pathological may give what they believe to be the socially desirable model, nevertheless in educational experiments, this answer rather than a ‘true’ answer, and all of these argues against a too-s implistic operation of randomized affect reliability (p. 105). Here the piloting of the 277
Research design questionnaire and the guarantees of anonymity and et al., 2011; Ramírez and Palu-a y, 2015). Further, a non-traceability might attenuate such problems. questionnaire is often more economical than the inter- The question of accuracy can be checked by the view in terms of time and money, for example, it can intensive interview method, a technique consisting of be mailed or conducted online. tactics that include familiarization, temporal recon- Its disadvantages, on the other hand, are: there is struction, probing and challenging (cf. Belson, 1986, often too low a percentage of returns; the interviewer is pp. 35–8). unable to answer questions concerning both the purpose The problem of non-response is addressed in Chap- of the interview and any misunderstandings experienced ters 17 and 40. Hudson and Miller (1997) suggest by the interviewee, for it sometimes happens in the case several strategies for maximizing the response rate to of the latter that the same questions have different mean- questionnaires (and, thereby, to increase reliability), ings for different people; if only closed items are used, which involve: the questionnaire may lack coverage or authenticity; if only open items are used, respondents may be unwilling OO including stamped addressed envelopes (for postal to write their answers for one reason or another; ques- questionnaires); tionnaires present problems to people of limited literacy; and an interview can be conducted at an appropriate OO multiple rounds of follow-u p to request returns speed whereas questionnaires are often filled in hur- (maybe up to three follow-u ps); riedly. There is a need, therefore, to pilot questionnaires and refine their contents, wording, length, etc. as appro- OO stressing the importance and benefits of the priate for the sample being targeted. questionnaire; One central issue in considering the reliability and validity of questionnaire surveys is that of sampling. OO stressing the importance of, and benefits to, the An unrepresentative, skewed sample, one that is too client group being targeted (particularly if it is a small or too large, can easily distort the data, and minority group that is struggling to have a voice); indeed, in the case of very small samples, prohibit sta- tistical analysis. We address sampling in Chapter 12. OO providing interim data from returns to non-returners to involve and engage them in the research; 14.14 Validity and reliability in observations OO checking addresses and changing them if necessary; OO following up questionnaires with a personal There are questions about two types of validity in observation-b ased research. Comments about the sub- telephone call; jective and idiosyncratic nature of the participant obser- OO tailoring follow-u p requests to individuals (with vation study are about its external validity. How do we know that the results of this one piece of research are indications to them that they are personally known applicable to other situations? Fears that observers’ and/or important to the research – including provid- judgements will be affected by their close involvement ing respondents with clues by giving some personal in the group relate to the internal validity of the method. information to show that they are known) rather How do we know that the results of this one piece of than blanket generalized letters; research represent the real thing? In Chapter 12 on OO features of the questionnaire itself (ease of comple- sampling, we refer to a number of techniques (quota tion, time to be spent, sensitivity of the questions sampling, snowball sampling, purposive sampling) that asked, length of the questionnaire); researchers employ as a way of checking on the repre- OO invitations to a follow-u p interview (face-to-face or sentativeness of the events that they observe and of by telephone); cross-c hecking their interpretations of the meanings of OO encouragement to participate by a friendly third those events. party; In addition to external validity, participant observa- OO understanding the nature of the sample population in tion should have rigorous internal validity checks. depth, so that effective targeting strategies can There are several threats to validity and reliability here, be used. for example: The advantages of the questionnaire over interviews, OO the researcher, in exploring the present, may be for instance, are: it tends to be more reliable; because it unaware of important antecedent events; is anonymous, it may encourage greater honesty (though, of course, dishonesty and falsification might not be able to be discovered in a questionnaire, and this is a particular issue in Internet questionnaires, where even the factual details about a respondent may be false). For example, an online respondent can create a different persona and make up responses (Shulman 278
Validity and reliability OO informants may be unrepresentative of the sample in tests.) Additionally, there are several ways in which the study; reliability might be compromised in tests. Feldt and Brennan (1993) suggest four types of threat to OO the presence of the observer might bring about dif- reliability: ferent behaviours (reactivity); OO individuals (e.g. their motivation, concentration, for- OO the researcher might ‘go native’, becoming too getfulness, health, carelessness, guessing, their attached to the group to see it sufficiently related skills, e.g. reading ability, their usedness to dispassionately. solving the type of problem set, the effects of practice); To address this, Denzin (1989) suggests triangulation of data sources and methodologies. Chapter 26 dis- OO situational factors (e.g. the psychological and physi- cusses the principal ways of overcoming problems of cal conditions for the test – the context); reliability and validity in observational research in nat- uralistic inquiry. In essence it is suggested that ‘trust- OO test marker factors (e.g. idiosyncrasy and worthiness’ (Lincoln and Guba, 1985) replaces more subjectivity); conventional views of reliability and validity, and that this is devolved on issues of credibility, confirmability, OO instrument variables (e.g. poor domain sampling, transferability and dependability. errors in sampling tasks, the realism of the tasks and If observational research is more structured in its relatedness to the experience of the testees, poor nature, yielding quantitative data, then the conven- question items, the assumption or extent of unidi- tions of intra- and inter-rater reliability apply. Here mensionality in item response theory, length of the steps are taken to ensure that observers enter data into test, mechanical errors, scoring errors, computer the appropriate categories consistently (i.e. intra- and errors). inter-rater reliability) and accurately. Further, to ensure validity, a pilot should have been conducted to Sources of unreliability ensure that the observational categories themselves are appropriate, exhaustive, discrete, unambiguous There are several threats to reliability in tests and and effectively operationalize the purposes of the examinations, particularly tests of performance and research. achievement, for example (Cunningham, 1998; Aira- sian, 2001; Cohen et al., 2010; Creswell, 2012). 14.15 Validity and reliability in tests For example, with respect to examiners and markers these concern: The researcher will have to judge the place and signifi- cance of test data, not forgetting the problem of the OO errors in marking (e.g. attributing, adding and trans- Hawthorne effect operating negatively or positively on fer of marks); students who undertake the tests. There is a range of issues which might affect the reliability of the test – for OO inter-rater reliability (different markers giving dif- example, the time of day, the time of the school year, ferent marks for the same or similar pieces of work); the temperature in the test room, the perceived impor- tance of the test, the degree of formality of the test situ- OO inconsistency in the marker (e.g. being harsh in the ation, ‘examination nerves’, the amount of guessing of early stages of the marking and lenient in the later answers by the students, how the test is administered stages of the marking of many scripts); and marked, the degree of closure or openness of test items. Hence the researcher who is considering using OO variations in the award of grades for work that is testing for acquiring research data must ensure that it is close to grade boundaries (some markers placing the appropriate, valid and reliable (Linn, 1993; Borsboom score in a higher or lower category than other et al., 2004). markers); Wolf (1994) suggests four main factors that might affect reliability: the range of the group that is being OO the Halo effect, wherein a student who is judged to tested; the group’s level of proficiency; the length of do well or badly in one assessment is given unde- the measure (the longer the test, the greater the chance served favourable or unfavourable assessment of errors); and the way in which reliability is calcu- respectively in other areas. lated. (Fitz-G ibbon (1997, p. 36) argues that, ceteris paribus, longer tests are more reliable than shorter With reference to the students and teachers themselves, there are several sources of unreliability: OO Motivation and interest in the task has a consider able effect on performance. Clearly, students need to be motivated if they are going to make a serious attempt at any test that they are required to 279
Research design undertake, where motivation is intrinsic (doing OO Students respond to the tester in terms of their something for its own sake) or extrinsic (doing perceptions of what he/she expects of them (Hala- something for an external reason, e.g. obtaining a dyna, 1997; Tombari and Borich, 1999; Stiggins, certificate or employment or entry into higher edu- 2001). cation). The results of a test completed in a desul- tory fashion by resentful pupils are hardly likely to OO The time of the day, week or month will exert an supply the researcher with reliable information influence on performance. Some students are fresher about the students’ capabilities (Wiggins, 1998). in the morning and more capable of concentration Motivation to participate in test-taking sessions is (Stiggins, 2001). strongest when students have been helped to see its purpose, and where the examiner maintains a warm, OO Students are not always clear on what they think is purposeful attitude towards them during the testing being asked in the question; they may know the session (Airasian, 2001). right answer but not infer that this is what is required OO The relationship (positive to negative) between the in the question. assessor and the testee exerts an influence on the assessment. This takes on increasing significance in OO The students may vary from one question to another teacher assessment, where the students know the – a student may have performed better with a differ- teachers personally and professionally – and vice ent set of questions which tested the same matters. versa – and where the assessment situation involves Black (1998) argues that two questions which, to the face-to-face contact between the teacher and the expert, may seem to be asking the same thing but in student. Both test-takers and test-givers mutually different ways, to the students might well be seen as influence one another during examinations, oral completely different questions. assessments and the like (Harlen, 1994). During a face-to-face test, students respond to such character- OO Students (and teachers) practice test-like materials, istics of the assessor as the person’s sex, age and which, even though scores are raised, might make personality. them better at taking tests, but the results might not OO The conditions – physical, emotional, social – exert indicate increased performance. an influence on the assessment, particularly if they are unfamiliar. Wherever possible, students should OO A student may be able to perform a specific skill in take tests in familiar settings, preferably in their own a test but not be able to select or perform it in the classrooms under normal school conditions. Distrac- wider context of learning. tions in the form of extraneous noise, walking about the room by the examiner, and intrusions into the OO Cultural, ethnic and gender background affect room, all have significant impact upon the scores of how meaningful an assessment task or activity is the test-takers, particularly when they are younger to students, and meaningfulness affects their pupils (Gipps, 1994). An important factor in reduc- performance. ing students’ anxiety and tension during an exami- nation is the extent to which they are quite clear OO Students’ personalities may make a difference to about what exactly they are required to do. Simple their test performance. instructions, clearly and calmly given by the exam- iner, can significantly lower the general level of OO Students’ learning strategies and styles may make a tension in the test-room. Teachers who intend to difference to their test performance. conduct testing sessions may find it beneficial in this respect to rehearse the instructions they wish to give OO Marking practices are not always reliable, markers to pupils before the actual testing session. Ideally, maybe being too generous, marking by effort and test instructions should be simple, direct and as brief ability rather than performance. as possible. OO The Hawthorne effect, wherein, in this context, OO The context in which the task is presented affects simply informing a student that this is an assessment performance: some students can perform the task in situation will be enough to disturb her performance everyday life but not under test conditions. – for better or worse (either case not being a fair reflection of her usual abilities). With regard to the test items themselves, there may be OO Distractions (including superfluous information). problems (e.g. test bias), for example: OO The task itself may be multi-dimensional, for example, testing ‘reading’ may require several com- ponents and constructs. Students can execute a mathematics operation in the mathematics class but they cannot perform the same operation in, for example, a physics class; students will disregard English grammar in a science class but observe it in an English class. This raises the issue of the number of contexts in which the behaviour must be 280
Validity and reliability demonstrated before a criterion is deemed to have OO the conditions operating at the time of the test (e.g. been achieved (Cohen et al., 2010). The question of noise, distractions, ambient, temperature, time of transferability of knowledge and skills is also raised day/week); in this connection. The context of the task affects the student’s performance. OO the test administration, with unstandardized proce- OO The validity of the items may be in question. dure in the timing, duration, teacher intervention, OO Clarity of the test items (Creswell, 2012): the avoid- instructions given, distribution and collection of ance of ambiguity. materials/test papers, monitoring etc. OO The language of the assessment and the assessor exerts an influence on the testee, for example, if the Hence specific contextual factors can exert a significant assessment is carried out in the testee’s second lan- influence on learning and this has to be recognized in guage or in a ‘middle-c lass’ code (Haladyna, 1997). conducting assessments, to render an assessment as OO The readability level of the task can exert an influ- unthreatening and natural as possible. ence on the test, for example, a difficulty in reading Harlen (1994, pp. 140–2) suggests that inconsist- might distract from the purpose of a test which is of ency and unreliability in teacher- and school-b ased the use of a mathematical algorithm. assessment may derive from differences in: (a) inter- OO The size and complexity of numbers or operations preting the assessment purposes, tasks and contents, by in a test (e.g. of mathematics) might distract the teachers or assessors; (b) the actual task set, or the con- testee who actually understands the operations and texts and circumstances surrounding the tasks (e.g. time concepts. and place); (c) how much help is given to the test-takers OO The number and type of operations and stages to a during the test; (d) the degree of specificity in the task – a student might know how to perform each marking criteria; (e) the application of the marking cri- element, but when they are presented in combina- teria and the grading or marking system that accompa- tion the size of the task can be overwhelming. nies it; (f ) how much additional information about the OO The form and presentation of questions affect the student or situation is being referred to in the results, giving variability in students’ performances. assessment. OO A single error early on in a complex sequence may Harlen advocates the use of a range of moderation confound the later stages of the sequence (within a strategies, both before and after the tests, including: question or across a set of questions), even though the student might have been able to perform the later OO statistical reference/scaling tests; stages of the sequence, thereby preventing the student OO inspection of samples (by post or by visit); from gaining credit for all she or he can, in fact, do. OO group moderation of grades; OO Questions might favour boys more than girls or vice OO post hoc adjustment of marks; versa. OO accreditation of institutions; OO Essay questions favour boys if they concern imper- OO visits of verifiers; sonal topics and girls if they concern personal and OO agreement panels; interpersonal topics (Haladyna, 1997; Wedeen et al., OO defining marking criteria; 2002). OO exemplification; OO Boys may perform better than girls on multiple- OO group moderation meetings. choice questions and girls perform better than boys on essay-type questions (perhaps because boys are Whilst moderation procedures are essentially post hoc more willing than girls to guess in multiple-choice adjustments to scores, agreement trials and practice items), and girls may perform better in written work marking can be undertaken before the administration of than boys. a test, which is particularly important if there are large OO Questions and assessment may be culture-b ound: numbers of scripts or several markers. what is comprehensible in one culture may be The issue here is that results as well as instruments incomprehensible in another. should be reliable. Reliability is also addressed by: OO The test may be so long, in order to ensure cover- age, that boredom and loss of concentration may OO calculating coefficients of reliability, split-h alf tech- impair reliability. niques, the Kuder‑Richardson formula, parallel/ equivalent forms of a test, test/re-test methods, the With regard to the operational procedures of the test, alpha coefficient; there may be variability in: OO calculating and controlling the standard error of measurement; 281
Research design OO increasing the sample size (to maximize the range test items, a key feature of domain sampling); and spread of scores in a norm-referenced test), content validity is achieved by ensuring that the though criterion-referenced tests recognize that content of the test fairly samples the class or fields scores may bunch around the high level (in mastery of the situations or subject matter in question. learning, for example), i.e. the range of scores might Content validity is achieved by making professional be limited, thereby lowering the correlation coeffi- judgements about the relevance and sampling of the cients that can be calculated; contents of the test to a particular domain. It is con- cerned with coverage and representativeness rather OO increasing the number of observations made and than with patterns of response or scores. It is a items included in the test (in order to increase the matter of judgement rather than measurement (Ker- range of scores); linger, 1986). Content validity will need to ensure several features of a test (Wolf, 1994): (a) test cov- OO ensuring effective domain sampling of items in tests erage (the extent to which the test covers the rele- based on item response theory (a particular issue in vant field); (b) test relevance (the extent to which Computer adaptive testing, introduced below the test items are taught through, or are relevant to, (Thissen, 1990)); a particular programme); (c) programme coverage (the extent to which the programme covers the OO ensuring effective levels of item discriminability overall field in question); and item difficulty. OO criterion-related validity (where a high correlation coefficient exists between the scores on the test and Reliability not only has to be achieved but has to be the scores on other accepted tests of the same per- seen to be achieved, particularly in ‘high-s takes’ testing formance); criterion-related validity is achieved by (where a lot hangs on the results of the test, e.g. comparing the scores on the test with one or more entrance to higher education or employment). Hence variables (criteria) from other measures or tests that the procedures for ensuring reliability must be transpar- are considered to measure the same factor. Wolf ent. The difficulty here is that the more one moves (1994) argues that a major problem facing test devis- towards reliability as defined above, the more the test ers addressing criterion-related validity is the selec- will become objective, the more students will be meas- tion of the suitable criterion measure. He cites the ured as though they are standardized objects, and the example of the difficulty of selecting a suitable crite- more the test will become decontextualized. rion of academic achievement in a test of academic An alternative form of reliability which is premised aptitude. The criterion must be: (a) relevant (and on a more constructivist psychology emphasizes the agreed to be relevant); (b) free from bias (i.e. where significance of context, the importance of subjectivity external factors that might contaminate the criterion and the need to engage and involve the testee more are removed); (c) reliable – precise and accurate; (d) fully than a simple test. This rehearses the tension capable of being measured or achieved; between positivism, post-p ositivism and more interpre- OO construct validity (e.g. the clear relatedness of a test tive approaches outlined in the first chapter of this item to its proposed construct/unobservable quality book. Objective tests, as described in the present or trait, demonstrated by both empirical data and chapter, lean strongly towards the positivist/post- logical analysis and debate, i.e. the extent to which positivist paradigm, whilst more phenomenological and particular constructs or concepts can account for interpretive paradigms of social science research will performance on the test); construct validity is emphasize the importance of settings, of individual per- achieved by ensuring that performance on the test is ceptions, of attitudes, in short, of ‘authentic’ testing fairly explained by particular appropriate constructs (e.g. by using non-contrived, non-artificial forms of test or concepts. As with content validity, it is not based data, e.g. portfolios, documents, course work, tasks that on test scores, but is more a matter of whether the are stronger in realism and more ‘hands on’). Though test items are indicators of the underlying, latent this latter adopts a view which is closer to assessment construct in question. In this respect construct valid- rather than narrowly ‘testing’, nevertheless the two ity also subsumes content and criterion-related overlap, both can yield marks, grades and awards, both validity. Construct validity is threatened by (a) can be formative as well as summative, both can be under-representation of the construct, i.e. the test is criterion-r eferenced. too narrow and neglects significant facets of a con- With regard to validity, it is important to note here struct, and (b) the inclusion of irrelevancies – excess that an effective test will ensure adequate: variance; OO content validity (e.g. adequate and representative coverage of programme and test objectives in the 282
Validity and reliability OO concurrent validity (where the results of the test To ensure test validity, then, the test must demonstrate concur with results on other tests or instruments that fitness for purpose as well as addressing the several types are testing/assessing the same construct/perform- of validity outlined above. The most difficult for ance – similar to predictive validity but without the researchers to address, perhaps, is construct validity, as it time dimension. Concurrent validity can occur argues for agreement on the definition and operationali- simultaneously with another instrument rather than zation of an unseen, half-g uessed-at construct or phe- after some time has elapsed); nomenon. The community of scholars has a role to play here. We also refer readers here to Chapter 27 on testing. OO face validity (that, superficially, the test appears – at face value – to test what it is designed to test); 14.16 Validity and reliability in life histories OO jury validity (an important element in construct validity, where it is important to agree on the con- Three central issues underpin the quality of data gener- ceptions and operationalization of an unobservable ated by life history methodology. They concern repre- construct); sentativeness, validity and reliability. Plummer (1983) draws attention to a frequent criticism of life history OO predictive validity (where results on a test accurately research, namely that its cases are atypical rather than predict subsequent performance – akin to criterion- representative. To avoid this charge, he urges research- related validity); ers to clarify and make explicit the life history in ques- tion’s relationship to a wider population (Plummer, OO consequential validity (where the inferences that can 1983) by way of appraising the subject on a continuum be made from a test are sound); of representativeness and non-representativeness. Reliability in life history research hinges on identi- OO systemic validity (Fredericksen and Collins, 1989) fying sources of bias and applying techniques to reduce (where programme activities both enhance test per- them. Bias arises from the informant, the researcher formance and enhance performance of the construct and the interactional encounter itself. Box 14.1, adapted that is being addressed in the objective). Cunning- ham (1998) gives an example of systemic validity where, if the test and the objective of vocabulary performance leads to testees increasing their vocab- ulary, then systemic validity has been addressed. Box 14.1 Principal sources of bias in life history research Source: Informant Is misinformation (unintended) given? Has there been evasion? Is there evidence of direct lying and deception? Is a ‘front’ being presented? What may the informant ‘take for granted’ and hence not reveal? How far is the informant ‘pleasing you’? How much has been forgotten? How much may be self-d eception? Source: Researcher Attitudes of researcher: age, gender, class, race, religion, politics etc. Demeanour of researcher: dress, speech, body language etc. Personality of researcher: anxiety, need for approval, hostility, warmth etc. Scientific role of researcher: theory held (etc.), researcher expectancy. Source: The interaction The encounter needs to be examined. Is bias coming from: The physical setting – ‘social space’? The prior interaction? Non-verbal communication? Vocal behaviour? Source: Adapted from Plummer (1983, table 5.2, p. 103) 283
Research design from Plummer (1983), provides a checklist of some an observed operational pattern and any plausible alter- aspects of bias arising from these principal sources. native theories are removed. Pattern matching here can correlate or compare the extent to which what actually Several validity checks are available to researchers occurred reflects the theoretical predictions or explana- here. Plummer identifies the following: tions (Trochim, 2006). Internal validity, Yin (2009, p. 41) suggests, can be 1 The subject of the life history may present an aut- addressed by: (a) pattern matching; (b) building explana- ocritique of it, having read the entire product. tions and considering rival, alternative explanations, and using logic models (where the dependent variable at one 2 A comparison may be made with similar written stage becomes the independent variable in the next stage sources by way of identifying points of major diver- of the causal research in a time sequence and in which gence or similarity. predicted and observed events are compared (pp. 149ff.)). Case studies, given their context-s pecificity and 3 A comparison may be made with official records by emphasis on subjectivity, may have limited or no exter- way of imposing accuracy checks on the life history. nal validity; that is, they may not be generalizable. However, external validity, Yin observes, can be 4 A comparison may be made by interviewing other addressed by careful use of theory in the single-c ase informants. studies, such that replication can be conducted (2009, p. 41). He suggests that ‘analytic generalizability’ Essentially, the validity of any life history lies in its (p. 43) may be possible, i.e. where a research strives to ability to represent the informant’s subjective reality, generalize from a particular set of findings to some that is to say, his or her definition of the situation. broader or more enduring theory (p. 43). In this he notes the importance of replication studies (p. 44). 14.17 Validity and reliability in case External validity here also has to consider the likeli- studies hood of transferability of the case study from one context to another, and whether those contexts and the Case studies can use quantitative, qualitative and mixed causal connections (Cartwright and Hardie, 2012) methods research, and we have indicated earlier in this between elements in those contexts differ. chapter the canons of validity and reliability for these. Reliability, Yin suggests, benefits from a case study Yin (2009) focuses on three main types of validity in database (2009, p. 41) as this can provide the eviden- case studies: construct, internal and external. Construct tiary source for checking, together with respondent val- validity, he avers, can be addressed by using: (a) multi- idation. Yin underlines the importance of keeping ple sources of evidence (which can lead to convergent careful documentary evidence here (p. 45) and of docu- lines of enquiry) (pp. 41–2); (b) a chain of evidence menting all aspects and stages of the research so that (from case study questions to the protocols for the case they can be checked (i.e. transparency). study – those protocols which link case study questions In essence, the points that Yin makes for addressing to the topic in hand, to the evidentiary base, to the con- validity and reliability in case study research can apply clusions and reporting) (p. 123); and (c) key informants to other types of educational research, and they act as a to review drafts of the case study report (p. 41). useful conclusion to this chapter. Pattern matching can also be used for establishing construct validity (Yin, 2009, p. 41). Here a predicted pattern (a theoretical pattern) in the data is matched to Companion Website The companion website to the book provides additional materials and PowerPoint slides for this chapter, which list the structure of the chapter and then provide a summary of the key points in each of its sections. This resource can be found online at: www.routledge.com/cw/cohen. 284
Part 3 Methodologies for educational research It is important to distinguish between design, method- of key issues and features of the different research ology and instrumentation. Too often methods of data styles that researchers can address in planning and collection are confused with methodology, and method- implementing their research. ology is confused with design. Part 2 provided an intro- duction to design issues and this part examines different Although we recognize that these are by no means methodologies of research, different styles, kinds of, exhaustive, we suggest that they cover the major styles and approaches to, research, separating them from of research methodology. These take in quantitative as methods – instruments for data collection. We identify well as qualitative research, mixed methods, the emerg- key styles of educational research in this section: a ing f ield of virtual worlds, social networking and neto- bundle of approaches that come under the umbrella of graphy, together with small-scale and large-scale naturalistic/qualitative/ethnographic types of research; approaches. Selecting research approaches is not a historical and documentary research (entirely rewritten matter simply of preference, arbitrary or automatic by Jane Martin); different kinds of survey, with an decision making, but, like other aspects of research, is a entirely new chapter on Internet surveys; case studies; deliberative process in which the key is the application different kinds of experiment, an extended review of of the notion of fitness for purpose. We do not advocate randomized controlled trials and coverage of ex post slavish adherence to a single methodology in research; facto research; a chapter on meta-analysis, research indeed combining methodologies may be appropriate syntheses and systematic reviews (entirely rewritten by for the research in hand. The intention here is to shed Harsh Suri), which takes account of the increased light on the different styles of research, locating them prominence given to these in the research community; in the paradigms of research introduced in Part 1. action research; and an entirely rewritten and updated chapter on virtual worlds, social network software and The companion website to the book provides addi- netography in educational research (written by Stewart tional materials and PowerPoint slides for the chapters Martin). These chapters include more extended analysis in this part, which can be found online at: www. routledge.com/cw/cohen. 285
Qualitative, naturalistic CHAPTER 15 and ethnographic research The title of this chapter indicates that a wide range of a form of social inquiry that tends to adopt a flexible types and kinds of qualitative research are addressed and data-driven research design, to use relatively here. The chapter addresses several key issues in plan- unstructured data, to emphasize the essential role of ning and conducting qualitative research: subjectivity in the research process, to study a number of naturally occurring cases in detail, and to OO foundations of qualitative, naturalistic and ethno- use verbal rather than statistical forms of approach. graphic inquiry (theoretical bases of these kinds of research) (Hammersley, 2013, p. 12) OO naturalistic research There are several purposes of qualitative research, for OO ethnographic research example, description, explanation, reporting, creation OO critical ethnography of key concepts, theory generation and testing. It is OO autoethnography important to stress, at the outset, that, though there are OO virtual ethnography many similarities and overlaps between naturalistic/ OO phenomenological research ethnographic and qualitative methods, there are also OO planning qualitative, naturalistic and ethnographic differences between them. The former connotes long- term residence with an individual, group or specific research comm unity (cf. Swain, 2006, p. 206), whilst the latter, OO reflexivity often being concerned with the nature of the data and OO doing qualitative research the kinds of research question to be answered, is OO some challenges in qualitative, ethnographic and an approach that need not require naturalistic approaches or principles. That said, there are sufficient naturalistic approaches areas of commonality to render it appropriate to con- sider them in the same chapter, and we tease out differ- There is no single blueprint for naturalistic, qualitative ences between them where relevant. The intention of or ethnographic research, because there is no single this chapter is to provide guidance for qualitative picture of the world. Rather, there are many worlds researchers who are conducting either long-term ethno- and many ways of investigating them. In this chapter graphic research or small-s cale, short-term qualitative we set out a range of key issues in understanding these research. worlds. There are many varieties of qualitative research, ‘Qualitative research’ is a loosely defined term that indeed Preissle (2006, p. 686) remarks that qualitative includes a vast range of kinds of research, has a wide researchers cannot agree on its purposes, boundaries, range of meanings and covers a heterogeneity of fields disciplinary fields or, indeed, its terminology (interpre- (Preissle, 2006; Hammersley, 2013, p. 9), so much so tive, naturalistic, qualitative, ethnographic, phenomeno- that Hammersley (2013) suggests that, given this range, logical, anthropological, symbolic interactionist, critical the term may no longer be a ‘genuine or useful cate- theoretical, case study, grounded theory etc.). However, gory’ (p. 99). He quotes Bryman’s (2008) view that she does indicate that qualitative research is character- qualitative research connotes the use of words rather ized by a ‘loosely defined’ group of designs that elicit than numbers (p. 366), and Sandelowski’s (2001) view verbal, aural, observational, tactile, gustatory and olfac- that it focuses on the attitudes towards understanding, tory information from a range of sources including experiences and interpretations by humans of the social audio, film, documents and pictures, that it draws world, and how to enquire about all of these (p. 893) strongly on direct experience and meanings, and that (though, as Hammersley (2013, p. 2) notes, these are not exclusive to qualitative research). Hammersley defines qualitative research as: 287
Methodologies for educational research these may vary according to the style of qualitative conjunctions and disjunctions. It is multilayered and research undertaken. not easily susceptible to the atomization or aggregation Qualitative research provides an in-depth, intricate processes inherent in much numerical research. It has and detailed understanding of meanings, actions, non- to be studied in total rather than in fragments if a true observable as well as observable phenomena, attitudes, understanding is to be reached. Chapter 1 indicated that intentions and behaviours, and these are well served by several approaches to educational research are con- naturalistic enquiry (Gonzales et al., 2008, p. 3). It tained in the paradigm of qualitative, naturalistic and gives voices to participants and it probes issues that lie ethnographic research. The characteristics of that para- beneath the surface of presenting behaviours and digm (Boas, 1943; Blumer, 1969; Lincoln and Guba, actions. 1985; Woods, 1992; LeCompte and Preissle, 1993; Ary Qualitative research can be used in systematic et al., 2002; Flick, 2009; Larsson, 2009; Hammersley, reviews (Dixon-W oods et al., 2001; see also Chapter 2013, 2014; Pring, 2015; Wellington, 2015) can be set 21 of the present volume), to: out ontologically (what is it we are trying to under- stand?), epistemologically (how can we know about OO identify and refine questions, fields, foci and topics something?) and methodologically (how can we of the review, i.e. to act as a precursor to a full research something?). review; Ontology of qualitative research OO provide data in their own right for a research synthesis; OO Qualitative research regards people as anticipatory, meaning-m aking beings who actively construct their OO indicate and identify the outcomes that are of inter- own meanings of situations and make sense of their est, and for whom; world and act in it through such interpretations (the constructivist/constructionist premise). People are OO complement and augment data from quantitative deliberate, intentional and creative in their actions, reviews; and meaning arises out of social situations, interac- tions and negotiations, and is handled through the OO fill gaps in quantitative reviews; interpretive processes of the humans involved. OO explain the findings from quantitative reviews OO Meanings used by participants to interpret situations and data; are culture- and context‑bound, and there are multi- OO provide alternative perspectives on topics; ple realities, not single truths in interpreting a situ OO contribute to the drawing of conclusions from the ation. History and biography intersect – we create our own futures but not necessarily in situations of review; our own choosing. OO be part of a multi-m ethods research synthesis; OO suggest how to turn evidence into practice. OO Realities are multiple, constructed and holistic, capable of sustaining multiple interpretations, Whilst the lack of controls in much qualitative research including those of all parties involved. People, situ- renders it perhaps unattractive for research syntheses, ations, events and objects are unique and have this is possibly unjustified, as it suggests that qualita- meaning conferred upon them rather than possess- tive research has to abide by the rules of the game of ing their own intrinsic meaning. Knower and known quantitative approaches. Qualitative methods have their are interactive, inseparable. own tenets, and these complement very well those of numerical research. As with quantitative studies, quali- Epistemology of qualitative research tative studies have to be weighted, downgraded, upgraded or excluded according to the quality of the OO Behaviour and, thereby, data are socially situated, evidence and sampling that they contain. They also context-related, context‑dependent and context-rich. have to overcome the problem that, by stripping out the To understand a situation researchers need to under- context in order to obtain themes and key concepts, stand the context both specifically and holistically – they destroy the heart of qualitative research – context the whole picture – because situations affect behaviour (Dixon-W oods et al., 2001, p. 131). and perspectives and vice versa. One task of the researcher is to understand, describe and explain the 15.1 Foundations of qualitative, multiple and differing interpretations of situations, naturalistic and ethnographic their distinctiveness, causes and consequences. inquiry OO All factors, rather than a limited number of vari The social and educational world is a messy place, full ables, have to be taken into account in understanding of contradictions, richness, complexity, connectedness, 288
Qualitative, naturalistic and ethnographic research a phenomenon. Research looks at relationships its framing and bounding, method of working and between elements in a whole system. As all entities data collection, analysing and reporting findings. are in a state of mutual simultaneous shaping, it is Research is influenced by the choice of the para- difficult, if not impossible, or inappropriate to dis- digm that guides the investigation into the problem, tinguish causes from effects. The attribution of and the choice of the substantive theory utilized to meaning is continuous and evolving over time. guide the collection and analysis of data and in the OO Social reality, experiences and social phenomena interpretation of findings. Research is influenced by are capable of multiple, sometimes contradictory the values that inhere in the context, which may be interpretations and are available to us through social congruent or dissonant within and between the interaction. Researchers focus on subjective accounts, parties involved. views and interpretations of a phenomenon by the OO Researchers are the instruments of the research participants (including the researcher): their ‘defini- (Eisner, 1991), blurring the distinction between the tion of the situation’, which is typically reported researcher and other participants and between sub- verbally rather than numerically. Social research jective and objective facts, as ‘objective facts’ are examines situations through the eyes of the partici- mediated through subjective interpretations. pants; the task of ethnographies, as Malinowski OO Generalizability is interpreted as generalizability to (1922, p. 25) observed, is to grasp the point of view identifiable, specific settings and subjects rather than of the native [sic], his [sic] view of the world and in universally. The context-s pecificity of the phenome- relation to his [sic] life. non being research often precludes generalization. Larsson (2009) suggests that generalization can be Methodology of qualitative research addressed through maximizing variation, context similarity and recognition of patterns (p. 28). OO Research must include ‘thick descriptions’ (Geertz, 1973) of the contextualized behaviour; for descrip- Hammersley (2013, pp. 29–34) notes the ‘critical’ tra- tions to be ‘thick’ requires inclusion not only of dition in qualitative research, in which situations are detailed observational data and data on meanings, observed and interpreted through wide-a ngle lenses that participants’ interpretations of situations and unob- include a focus on the multiple, intersecting, wider served factors. Observational data are important, factors that bear on a situation, utilizing ideology cri- acquired from the natural, undisturbed setting, with tique with an interest in emancipation from oppression, participants speaking in their own terms and exploitation, inequality, power and powerlessness, and behaving ‘naturally’. To understand and research un‑freedoms, i.e. research with an overtly political a situation often requires long-term immersion in intent to expose the deforming interests (ideologies) at the system, not least as researchers do not know work in a situation and to bring about a more just in advance what they will see or what they will society. look for. Lincoln and Guba (1985, pp. 39–43), Ary et al. (2002, pp. 451–7) and Polkinghorne (2007) tease out OO Social research should be conducted in natural, implications of these axioms: uncontrived, real-world settings with as little intru- siveness as possible by the researcher. Here data are OO studies must be set in their natural settings as collected systematically, analysed inductively and context is heavily implicated in meaning; abductively, with constructs and findings deriving and inferred from the data during the research. OO humans are the research instrument; Human phenomena seem to require even more con- OO utilization of tacit knowledge is inescapable; ditional stipulations than do other kinds of phenom- OO qualitative methods sit more comfortably than quan- ena, and meanings and understandings replace proof. Only time- and context-bound working titative methods with the notion of the human-a s- hypotheses and idiographic statements are possible, instrument; and researchers generate rather than test hypotheses. OO purposive sampling enables the full scope of issues Theory generation is derivative – grounded (Glaser to be explored; and Strauss, 1967) – the data suggest the theory OO data analysis is inductive rather than a priori and rather than vice versa. deductive; OO theory emerges rather than being pre-o rdinate. A OO The processes of research and behaviour are as priori theory is replaced by grounded theory; important as the outcomes. Research is value-b ound OO research designs emerge over time (and as the sam- and is influenced by the researcher’s values as pling changes over time); expressed in the choice of the focus of the research, 289
Methodologies for educational research OO the outcomes of the research are negotiated; respect Tracy (2010) notes (p. 844) that triangula- OO the natural mode of reporting is the case study; tion and crystallization may sit together uncomfort- OO nomothetic interpretation is replaced by idiographic ably, as triangulation suggests a single correct conclusion (accuracy) whilst crystallization (looking interpretation; through a crystal from many different viewpoints OO applications are tentative and pragmatic; to see many refracted images) yields different OO the focus of the study determines its boundaries; findings. OO trustworthiness, credibility, theoretical adequacy, 5 Resonance: readers find points of resonance with themselves (empathy, identification and reverb corroboration, interpretive adequacy, dependability eration) and are affected by the research through and confirmability replace more conventional views evocative, engaging, vivid and carefully worded of reliability and validity. Here a statement or claim representations, transferable findings to their own for knowledge is not ‘intrinsically valid’; rather, situation and ‘naturalistic generalizations’ (applic validity is a matter of ‘intersubjective judgement’ as ability to their own situation in improving practice). determined by the community of participants 6 Significant contribution: moving forward the field (widely defined) and the force of the argument and conceptually, theoretically, methodologically, prac- evidence (Polkinghorne, 2007, pp. 474–5). tically, heuristically, morally and practically, which includes catalytic validity (see Chapter 14). Whilst these points above suggest considerations in 7 Ethical: attention to procedural ethics (see the dis- addressing quality in qualitative research (Hammersley, cussion of ethics later in this chapter), situational 2007), researchers will need to note that it is invidious (see Chapter 7), cultural, relational (awareness of to produce simplistic, single, permanent or universal the researcher’s influence on other participants) and lists of criteria for quality in qualitative research (Guba the ethics of leaving the research field. and Lincoln, 2005), as they may all too easily be a poor 8 Meaningful coherence: achievement of the study’s fit to the range of types and methodologies of qualita- purposes, fitness for purposes in methods and proce- tive research. Tracy (2010, p. 840), whilst being dures used, linkages between literature, research mindful of the dangers in this enterprise (p. 838), sets questions, methods, findings and interpretations. out and defends ‘eight “big tent” criteria for excellent qualitative research’: 1 A worthy topic (one which is timely, relevant, sig- Lincoln and Guba (1985, pp. 226–47) set out ten ele- nificant and interesting, pointing out surprises that ments in research design for qualitative studies: challenge common-sense assumptions). Pelias (2015) notes that the researcher has to ask whose 1 Determining a focus for the inquiry; interests are being served by the research. 2 Determining the fit of paradigm to focus; 3 Determining the ‘fit’ of the inquiry paradigm to the 2 ‘Rich rigor’, with attention to the theoretical con- structs used (matching the complexity of the phe- substantive theory selected to guide the inquiry; nomenon with the complexity of theoretical 4 Determining where and from whom data will be constructs), contexts, sampling, sufficient data to support the claims made and the levels of analysis collected; applied, data collection and analysis, time in the 5 Determining successive phases of the inquiry; field and transparency of data and analysis. 6 Determining instrumentation; 7 Planning data collection and recording modes; 3 Sincerity: addressing self-reflexivity (discussed 8 Planning data-a nalysis procedures; below) and introspection, honesty, vulnerability, 9 Planning the logistics: authenticity and transparency concerning the research process, from entry to the field to exit from a prior logistical considerations for the project as it, together with data analysis. a whole 4 Credibility: ‘trustworthiness, verisimilitude and b the logistics of field excursions prior to going plausibility of the research findings’ (Tracy, 2010, into the field p. 842), addressed by thick description, triangula- tion, multivocality, autoethnographies, member c the logistics of field excursions while in reflections, crystallization, demonstration of how the the field findings and conclusions were reached, persuasive- ness of the claims in light of the warrants, concrete d the logistics of activities following field details and disclosure of tacit knowledge. In this excursions e the logistics of closure and termination; 10 Planning for trustworthiness. 290
Qualitative, naturalistic and ethnographic research These elements can be set out into a sequential, staged deductive paradigm (e.g. Meinefeld, 2004, p. 153), not approach to planning naturalistic research (e.g. Schatz- least because there is a recognition that the researcher man and Strauss, 1973; Delamont, 1992). Spradley influences the research and because the research is (1979) sets out the stages of: (i) selecting a problem; much more open and emergent in qualitative (ii) collecting cultural data; (iii) analysing cultural data; approaches. Indeed, Meinefeld, citing classic studies (iv) formulating ethnographic hypotheses; (v) writing like Whyte’s (1955) Street Corner Society, suggests the ethnography. We offer a fuller, twelve-s tage model that it is impossible to predetermine hypotheses, later in this chapter. whether one wishes to or not, as prior knowledge Like other styles of research, naturalistic and quali- cannot be presumed. Glaser and Strauss (1967) suggest tative methods can formulate research questions which that researchers should deliberately free themselves should be clear and unambiguous but open to change as from all prior knowledge, even suggesting that it is the research develops. Strauss (1987) terms these ‘gen- impossible to read up in advance, as it is not clear what erative questions’: they stimulate the line of investiga- reading will turn out to be relevant – the data speak for tion, suggest initial hypotheses and areas for data themselves. Theory is the end point of the research, not collection, yet they do not foreclose the possibility of its starting point. modification as the research develops. A balance must One has to be mindful that the researcher’s own be struck between having research questions that are so background interest, knowledge and biography precede broad that they do not steer the research in any particu- the research and that though initial hypotheses may not lar direction, and so narrow that they block new be foregrounded in qualitative research, nevertheless avenues of enquiry (Flick, 2004b, p. 150). the initial establishment of the research presupposes a Miles and Huberman (1994) identify two types of particular area of interest, i.e. the research and data for qualitative research design: loose and tight. Loose focus are not theory-free; knowledge is not theory-free. research designs have broadly defined concepts and Indeed Glaser and Strauss (1967) acknowledge that areas of study, and, indeed, are open to changes of they brought their own prior knowledge to their methodology. These are suitable, they suggest, when research on dying. researchers are experienced and when the research is The resolution of this apparent contradiction – the investigating new fields or developing new constructs, call to reject an initial hypothesis in qualitative akin to the flexibility and openness of theoretical sam- research, yet a recognition that all research commences pling (see Chapter 37). By contrast, a tight research with some prior knowledge or theory that gives rise to design has narrowly restricted research questions and the research, however embryonic – lies in several fields. predetermined procedures, with limited flexibility. These include: an openness to data (Meinefeld, 2004, These, the authors suggest, are useful when researchers pp. 156–7); a preparedness to modify one’s initial pre- are inexperienced, when the research is intended to suppositions and position; a declaration of the extent to look at particular specified issues, constructs, groups or which the researcher’s prior knowledge may be influ- individuals, or when the research brief is explicit. encing the research (i.e. reflexivity); a recognition of Even though, in qualitative research, issues and the- the tentative nature of one’s hypothesis; a willingness ories emerge from the data, this does not preclude the to use the research to generate a hypothesis; and an value of having research questions. Flick (1998, p. 51) acknowledgement that having a hypothesis may be just suggests three types of research questions in qualitative as much a part of qualitative research as it is of quanti- research, namely, those concerned with: (a) describing tative research. states, their causes and how these states are sustained; An alternative to research hypotheses in qualitative (b) describing processes of change and consequences of research is a set of research questions, and we consider those states; (c) suitability for supporting or not sup- these below. For qualitative research, Miles and Huber- porting hypotheses and assumptions or for generating man (1994, p. 74) also suggest the replacement of new hypotheses and assumptions (the ‘generative ques- ‘hypotheses’ with ‘propositions’, as this indicates that tions’ referred to above). the qualitative research is not necessarily concerned with testing a predetermined hypothesis as such but, Should one have a hypothesis in qualitative nevertheless, is concerned to be able to generate and research? test a theory (e.g. grounded theory). We mentioned in Chapter 1 that positivist approaches typically test pre-formulated hypotheses and that a dis- tinguishing feature of naturalistic and qualitative approaches is its reluctance to enter the hypothetico- 291
Methodologies for educational research 15.2 Naturalistic research made reference in Chapter 1. LeCompte and Preissle (1993, pp. 39–44) suggest that ethnographic approaches Main kinds of naturalistic enquiry are (Arsenault and are concerned with description rather than prediction, Anderson, 1998, p. 121; Flick, 2004a, 2004b, 2009): induction rather than deduction, generation rather than verification of theory, construction rather than enumer- OO case study (an investigation into a specific instance ation, and subjectivities rather than objective know or phenomenon in its real-life context); ledge. With regard to the latter, they distinguish between emic approaches (as in the term ‘phonemic’, where the OO comparative studies (where several cases are com- concern is to catch the subjective meanings placed on pared on the basis of key areas of interest); situations by participants) and etic approaches (as in the term ‘phonetic’, where the intention is to identify and OO retrospective studies (which focus on biographies of understand the objective or researcher’s meaning and participants or which ask participants to look back constructions of a situation) (p. 45). on events and issues); Woods (1992), however, argues that some differ- ences between quantitative and qualitative research are OO snapshots (analyses of particular situations, events exaggerated, that the epistemological contrast between or phenomena at a single point in time); the two is overstated, as qualitative techniques can be used for both generating and testing theories. OO longitudinal studies (which investigate issues or people over time); 15.3 Ethnographic research OO ethnography (a portrayal and explanation of social As the ‘interview society’ is losing ground to the groups and situations in their real-life contexts); ‘observation society’ (Gobo, 2011), ethnography, being largely observation-b ased, is coming into prominence. OO grounded theory (developing theories to explain LeCompte and Preissle (1993) suggest that ethno- phenomena, the theories emerging from the data graphic research seeks to create as vivid and analytical rather than being prefigured or predetermined); a reconstruction as possible of the culture or groups being studied (p. 235). An ethnography is a descriptive, OO biography (individual or collective); analytical and explanatory study of the culture (and its OO phenomenology (seeing things as they are really like components), values, beliefs and practices of one or more groups (e.g. Creswell, 2012, p. 462; Bhatti, 2012; and establishing the meanings of things through illu- Denscombe, 2014). It can study a small group (a few mination and explanation rather than through taxo- people: micro-e thnography) (p. 463) or a larger group/ nomic approaches or abstractions, and developing society/community, and, in autoethnography, an indi- theories through the dialogic relationships of vidual in a social setting. Though it typically uses qual- researcher to researched). itative data, it does not preclude the use of relevant quantitative data (Hamersley, 2006). The main methods for data collection in naturalistic LeCompte and Preissle (1993) and Denscombe enquiry are (Hammersley and Atkinson, 1983): (2014) indicate several key elements of ethnographic approaches: OO participant observation OO interviews and conversations OO the world view of the participants is investigated OO documents and field notes and represented – their ‘definition of the situation’ OO accounts (Thomas, 1923); OO notes and memos. OO data are elicited and gathered; Ary et al. (2002) add to these grounded theory and his- OO researchers spend considerable amounts of time in torical research. Lofland (1971) suggests that naturalistic methods the field – immersion – to research the everyday as are intended to address three major questions: well as the non-n ormal aspects of the culture, group, etc. (though Hammersley (2006) notes that, in com- OO What are the characteristics of a social phenomenon? parison to earlier times, much fieldwork includes OO What are the causes of the social phenomenon? shorter rather than longer stays in the field, and this OO What are the consequences of the social may risk loss of important historical contextual information and the danger of assuming that the data phenomenon? These include: (a) the environment; (b) people and their relationships; (c) behaviour, actions and activities; (d) verbal behaviour; (e) psychological stances; (f ) histo- ries; and (g) physical objects (Baker, 1994, pp. 241–4). There are several key differences between the natu- ralistic approach and that of the positivists to whom we 292
Qualitative, naturalistic and ethnographic research collected are a fair representation of the entire why the group, culture or society is acting as it does situation); and what can be learned from this. OO meanings are accorded to phenomena by both the In educational research, Walford (2009) notes that researcher and the participants; the process of ethnographies can use multiple data types to focus in research, therefore, is hermeneutic, uncovering depth on cultural formations and maintenance: how the meanings; culture works (p. 273). Knowledge about these is OO the constructs of the participants are used to struc- obtained through sustained, long-term immersion and ture the investigation; involvement in the group, giving importance to the OO empirical data are gathered in their naturalistic ‘accounts of participants’ perspectives and understand- setting (unlike laboratories or in controlled settings ings’ (p. 272), all of which lead to the formation of as in other forms of research where variables are hypotheses and theory testing which can provide the manipulated); basis for theoretical generalization and, indeed, subse- OO observational techniques are used extensively (both quent data collection (p. 272). Like Hammersley participant and non-participant) to acquire data on (2006), Walford does not rule out quantitative data. real-life settings; Ethnography, Walford avers (p. 275), is story-telling, OO the research is holistic, that is, it seeks a description with the researcher centrally involved in the generation and interpretation of ‘total phenomena’; and telling of the story (p. 275). OO there is a move from description and data to infer- For Denscombe (2014, p. 90), the attractions of eth- ence, explanation, suggestions of causation, and nographies are that they use detailed and direct obser- theory generation; vational data; they focus on holism in the research; OO methods are ‘multimodal’ and the ethnographer is a bring a fresh eye to the obvious, ordinary, taken-for- ‘methodological omnivore’ (LeCompte and Preissle, granted and everyday behaviour; take seriously the par- 1993, p. 232). ticipants’ views; have strong ecological validity; and are self-aware (see section below, ‘Reflexivity’). On Hitchcock and Hughes (1989, pp. 52–3) suggest that the other hand, he notes that ethnographies have to ethnographies involve: balance objective descriptions of events and cultures with the researcher’s own interpretation of these, and OO the production of descriptive cultural knowledge of they have to avoid the risk of creating isolated ‘pic- a group; tures’ of situations which lack an overall structure. Eth- nographers and ethnographies must not sacrifice OO the description of activities in relation to a particular analysis to telling a story. Ethnographies are difficult – cultural context from the point of view of the if not impossible – to replicate or to check, and they members of that group themselves; raise difficult ethical issues more prominently than other approaches (discussed below), and researchers OO the production of a list of features constitutive of may experience difficulties in gaining access to the membership in a group or culture; research setting (p. 91). Dobbert and Kurth-S chai (1992) urge not only that OO the description and analysis of patterns of social ethnographic approaches become more systematic but interaction; that they study and address regularities in social behav- iour and social structure (pp. 94–5). The task of ethnog- OO the provision as far as possible of ‘insider accounts’; raphers (p. 150) is to balance a commitment to catching OO the development of theory. the diversity, variability, creativity, individuality, uniqueness and spontaneity of social interactions (e.g. Bryman (2008) notes that ethnographic researchers by ‘thick descriptions’; Geertz, 1973) with a commit- immerse themselves in the group or society which they ment to the task of social science to seek regularities, are studying in order to collect field data which may order and patterns within such diversity. As Durkheim comprise descriptive notes and analytical comments (1982) noted, there are ‘social facts’. about the culture of the members of the society or Following this line, it is possible to suggest that eth- group which they are studying, including the views and nographic research can address issues of generalizabil- definitions of the situation of the members themselves, ity – a tenet of positivist research – interpreted as which are then written up in way that is amenable and ‘comparability’ and ‘translatability’ (LeCompte and accessible to the target audience or readership. An Preissle, 1993, p. 47). For comparability, the character- ethnography moves beyond description to data analy- istics of the group that is being studied need to be made sis, to theory generation and, if appropriate, to hypothe- sis generation, to explain what is happening and observed in a situation, group, culture or society and why, what are its key dynamics, in short to understand 293
Methodologies for educational research explicit so that readers can compare them with other journey along the road of knowledge creation for both similar or dissimilar groups. For translatability, the ana- interviewer and interviewee, i.e. as part of a mutually lytic categories used in the research as well as the char- empowering relationship (e.g. Edwards and Holland, acteristics of the groups are made explicit so that 2013, p. 32). meaningful comparisons can be made to other groups With ‘mutual shaping and interaction’ between the and disciplines. researcher and participants taking place (Lincoln and Spindler and Spindler (1992, pp. 72–4) put forward Guba, 1985, p. 155), the researcher becomes, as it were, several hallmarks of effective ethnographies: the ‘human instrument’ in the research (p. 187), building on her tacit knowledge and her propositional OO Observations have contextual relevance, both in the knowledge, using methods that sit comfortably with immediate setting in which behaviour is observed human inquiry, for example, observations, interviews, and in further contexts beyond. documentary analysis and ‘unobtrusive’ methods (p. 187). The advantage of the ‘human instrument’ is OO Hypotheses emerge in situ as the study develops in her adaptability, responsiveness, knowledge, ability the observed setting. to handle sensitive matters, ability to see the whole picture and ability to clarify, summarize, explore, OO Observation is prolonged and often repetitive. Events analyse and examine atypical or idiosyncratic responses and series of events are observed more than once to (pp. 193–4). Here Hammersley (1992b) comments on establish reliability in the observational data. the risk of researcher bias (see section below on ‘Reflexivity’). OO Inferences from observation and various forms of Denscombe (2014) notes that ethnographies can ethnographic inquiry are used to address insiders’ include life histories, and they need to provide thick views of reality. descriptions and use both idiographic and nomothetic approaches, the former to produce a detailed picture of OO A major part of the ethnographic task is to elicit the unique situation/culture/group and the latter to socio-cultural knowledge from participants, render- produce theories that can apply beyond the situation in ing social behaviour comprehensible. question. A key concern for ethnographers is how far out to OO Instruments, schedules, codes, agenda for inter- go in order to understand a situation (macro issues views, questionnaires, etc. should be generated in affecting, contextualizing, locating or contributing to situ, and should derive from observation and ethno- the situation in hand) or how far in to go in focusing on graphic inquiry. a situation (micro ethnography). In other words, if eth- nography celebrates holism, what is the whole and how OO A transcultural, comparative perspective is usually are data about the whole to be gathered (Hammersley, present, although often it is an unstated assumption, 2006, pp. 6–7)? and cultural variation (over space and time) is natural. 15.4 Critical ethnography OO Some socio-c ultural knowledge that affects behav- One branch of ethnography that resonates with the criti- iour and communication under study is tacit/ cal paradigm outlined in Chapter 3 is critical ethnogra- implicit, and may not be known even to participants phy – ‘critical theory in action’ (Madison, 2005, p. 13), or known ambiguously to others. It follows that one which, as Thomas (1993, p. vii) suggests, adopts a task for an ethnographer is to make explicit to ‘subversive worldview’ to conventional traditions readers what is tacit/implicit to informants. of research. Marshall and Rossman (2016) note that critical ethnography has a wide embrace, taking in OO The ethnographic interviewer should not frame or different kinds of critical theory, queer theory, critical predetermine responses by the kinds of questions race theory, autoethnography, feminist theories, criti- that are asked, because the informants themselves cal discourse analysis, participatory action research, have the emic, native cultural knowledge. cultural studies, post-c olonial theories and Internet studies. OO In order to collect as much live data as possible, any Whereas conventional ethnography is concerned technical device may be used. with what is, critical ethnography concerns itself with what could or should be (Thomas, 1993, p. 4). Here OO The ethnographer’s presence should be declared and his or her personal, social and interactional position in the situation should be described. Ethnographic researchers will need to consider whether to employ interviewing at all, as it is a non-natural situ ation, or, if it is to be used, what form it will take – away from the interviewer as ‘miner’ (Kvale, 1996) – seeking nuggets of information (see Chapter 25) – and moving towards the interviewer as ‘traveller’ in a collaborative 294
Qualitative, naturalistic and ethnographic research qualitative, anthropological, participant, observer-based OO research and thinking are mediated by power research has its theoretical basis in critical theory relations; (Quantz, 1992, p. 448; Carspecken, 1996; Creswell, 2012). As outlined in Chapter 3, this paradigm is con- OO these power relations are socially and historically cerned with the exposure of oppression and inequality located; in society with a view to emancipating individuals and groups towards collective empowerment. In this OO facts and values are inseparable; respect, research is an inherently political enterprise; it OO relationships between objects and concepts are fluid is ethnography with a political intent (cf. Thomas, 1993, p. 4). Madison (2005, p. 5) indicates that critical and mediated by the social relations of production; ethnography has an explicit agenda and an ‘ethical OO language is central to perception; responsibility’ to promote freedom, social justice, OO certain groups in society exert more power than equity and well-b eing. This, he avers, inevitably involves disturbing accepted meanings and disrupting others; the status quo and purported neutrality of research, OO inequality and oppression are inherent in capitalist together with exposing taken-for-granted, ‘domesti- cated’ (Thomas, 1993, p. 7) assumptions that perpetu- relations of production and consumption; ate the power of the already powerful at the expense of OO ideological domination is strongest when oppressed the powerless and the dominated. Critical ethnography takes power, control, empow- groups see their situation as inevitable, natural or erment, privilege, repression, hegemony, victimization, necessary; marginalization and social exploitation as problematic OO forms of oppression mediate each other and must be and to be changed rather than simply to be interrogated considered together (e.g. race, gender, class). and discovered (Thomas, 1993, p. 6; Creswell, 2012, p. 467). Like ethnography, it catches ethnographic data, Quantz (1992, pp. 473–4) argues that research is ines- but, beyond this, exposes the data to the ideology cri- capably value-laden in that it serves some interests, and tique (see Chapter 3). that in critical ethnography researchers must expose Like Habermas’s emancipatory interest (see Chapter these interests and move participants towards emanci- 3 of this volume), research is not simply a scientific, pation and freedom. The focus and process of research, technical exercise, nor is it simply a hermeneutic matter at heart political, concern issues of power, domination, of understanding and interpreting a situation; it does voice and empowerment (cf. Lather, 1991). In critical not reject these, but it requires the researcher to move ethnography the cultures, groups and individuals being beyond them to engage change (Thomas, 1993, p. 19) studied are located in contexts of power and interests. as a political act, and it must play its part as activism These contexts must be exposed, their legitimacy inter- against hegemonic oppression. Here researchers have rogated and the value base of the research itself to consider their own ‘positionality’ in this enterprise exposed. Reflexivity is high in critical ethnography. (Madison, 2005, p. 7), i.e. how their research will help What separates critical ethnography from other forms to break domination and inequality. Researchers and of ethnography is that in the former, questions of legiti- their research are neither neutral nor innocent. Both macy, power, values in society and domination and subjectivity and objectivity have to be interrogated for oppression are foregrounded. their political stances and effects (p. 8) in relation to How does the critical ethnographer proceed? This is those being researched (the ‘Others’) (p. 9); the not an easy task, as critical ethnography focuses on, research has to make a positive difference to the worlds and challenges, taken-for-granted assumptions and of the ‘Others’ (the participants). This moves the eth- meanings, and these may be difficult to expose simply nographer beyond simply being reflexive to being an because they are so taken-for-granted, i.e. embedded in activist. our daily lifeworlds and behaviour. In this sense a criti- This is contentious: on the one hand it suggests that cal ethnography is untidy, the study emerges rather the researcher is an ideologue (rather than, say, a cool than being planned in advance; areas of focus emerge theorist); on the other hand, the claim made is that, like as meanings are revealed and challenged from the posi- it or not, research is a political act, and that this has tion of ideology critique (Thomas, 1993, p. 35). It starts been hidden in much research. with unsettling issues in society and explores them Carspecken (1996, pp. 4ff.) suggests several key further (Thomas gives the examples of prisons, the premises of critical ethnography: social construction of deviance, racism, prejudice and repressive legislation). Carspecken and Apple (1992, pp. 512–14) and Carspecken (1996, pp. 41–2) identify five stages in crit- ical ethnography (Figure 15.1). 295
Methodologies for educational research Stage 1 (that the utterance is comprehensible) and sincerity (of Compiling the primary record through the speaker’s intentions). Carspecken (1996, pp. 104–5) takes this further in suggesting several categories the collection of monological data of reference in objective validity: (i) that the act is comprehensible, socially legitimate and appropriate; Stage 2 (ii) that the actor has a particular identity and Preliminary reconstructive analysis particular intentions or feelings when the action takes place; (iii) that objective, contextual factors are Stage 3 acknowledged. Dialogical data collection Stage 2: Preliminary reconstructive analysis Stage 4 Discovering system relations Reconstructive analysis attempts to uncover the taken- for-granted components of meaning or abstractions that Stage 5 participants have of a situation. Such analysis is Using system relations to explain findings intended to identify the value systems, norms and key concepts that are guiding and underpinning situations. FIGURE 15.1 Five stages in critical ethnography Carspecken (1996, p. 42) suggests that the researcher go back over the primary record from stage one to Stage 1: Compiling the primary record examine patterns of interaction, power relations, roles, through the collection of monological data sequences of events, and meanings accorded to situa- At this stage the researcher is comparatively passive tions. He asserts that what distinguishes this stage as and unobtrusive: a participant observer. The task here ‘reconstructive’ is that cultural themes and social and is to acquire objective data and it is ‘monological’ system factors that are not usually articulated by the in the sense that it concerns only the researcher participants themselves are, in fact, reconstructed and writing her own notes to herself. Lincoln and Guba articulated, turning the undiscursive into discourse. (1985) suggest that validity checks at this stage will Moving to higher-level abstractions, this stage can include: utilize high-level coding (see the discussion of coding below). 1 using multiple devices for recording together with In critical ethnography, Carspecken (p. 141) recom- multiple observers; mends several ways to ensure validity at this stage: 2 using a flexible observation schedule in order to 1 use interviews and group discussions with the sub- minimize biases; jects themselves; 3 remaining in the situation for a long time in order to 2 conduct member checks on the reconstruction in overcome the Hawthorne effect; order to equalize power relations; 4 using low-inference terminology and descriptions; 3 use peer debriefing (a peer is asked to review the 5 using peer-debriefing; data to suggest if the researcher is being too selec- 6 using respondent validation. tive, e.g. of individuals, of data, of inference) to check biases or absences in reconstructions; Echoing Habermas’s (1979, 1982, 1984) work on speech-a ct validity claims, validity here includes truth 4 employ prolonged engagement to heighten the (the veracity of the utterance), legitimacy (rightness researcher’s capacity to assume the insider’s and appropriateness of the speaker), comprehensibility perspective; 5 use ‘strip analysis’ – checking themes and segments of extracted data with the primary data, for consistency; 6 use negative case analysis. Stage 3: Dialogical data collection Here data are generated by, and discussed with, the par- ticipants (Carspecken and Apple, 1992). The authors argue that this is non-n aturalistic in that the participants are being asked to reflect on their own situations, cir- cumstances and lives and to begin to theorize about 296
Qualitative, naturalistic and ethnographic research their lives. This is a crucial stage because it enables the 15.5 Autoethnography participants to have a voice, to democratize the research. It may be that this stage produces new data Autoethnography, a derivative of ethnography, is a which challenge the preceding two stages. process, method and product that ‘seeks to describe and In introducing greater subjectivity by participants systematically analyze (graphy) personal experience into the research at this stage, Carspecken (1996, (auto) in order to understand cultural experience pp. 164–5) proffers several validity checks, for (ethno)’ (Ellis et al., 2011, p. 1; cf. Reed‑Danahay, example: (a) consistency checks on interviews that 1997) and to ‘extend sociological understanding’ (Wall, have been recorded; (b) repeated interviews with par- 2008, p. 39) by looking at oneself in a wider context. ticipants; (c) matching observation with what partici- Autoethnographies are ‘highly personalized accounts pants say is happening or has happened; (d) avoiding that draw upon the experience of the author/researcher leading questions at interview, reinforced by having for the purposes of extending sociological understand- peer debriefers check on this; (e) respondent validation; ing’ (Sparkes, 2000, p. 21). For examples of this, see (f ) asking participants to use their own terms in describ- Reed-D anahay (1997), Ellis (2004) and Chang (2008). ing naturalistic contexts, and encouraging them to An autoethnography places the self – the researcher explain these terms. – at the centre of research about himself/herself in a social context; it is self-focused (Ngunjiri et al., 2010), Stage 4: Discovering system relations though it can engage collaborative as well as individual study (Denshire, 2014). It examines the relationship of This stage relates the group being studied to other the researcher to others through the eyes of the factors that impinge on that group, for example: local researcher and connects the personal, autobiographic to community groups, local sites that produce cultural the social and cultural (Ellis, 2004, p. xix). products. At this stage Carspecken (1996, p. 202) notes Autoethnography often has a deliberate political, that validity checks will include: (i) maintaining the social, critical theoretical and emancipatory or trans- validity requirements of the earlier stages; (ii) seeking a formative agenda (Belbase et al., 2008; Bettez, 2015; see match between the researcher’s analysis and the com- Chapter 3 and the comments above on critical ethnogra- mentaries that are provided by the participants and phy) and it focuses on ‘things that matter a great deal to other researchers; (iii) using peer debriefers and the autoethnographer’ (Delamont, 2009, p. 57). It prob- respondent validation. lematizes and interrogates the socially constructed self and the situatedness and relationship of self to others Stage 5: Using system relations to explain (Starr, 2010, p. 3). For educationists, it has been likened findings to a form of critical pedagogy in its commitment to transformative and emancipatory processes and the This stage seeks to examine and explain the findings in social construction of knowledge (Starr, 2010, p. 4). light of macro-s ocial theories (Carspecken, 1996, Autoethnography concerns studying and writing p. 202). In part, this is a matching exercise to fit the about our personal and socio‑cultural selves, identities research findings within a social theory. and the human condition (Dyson, 2007; Nicol, 2013; In critical ethnography, therefore, the move is from Marshall and Rossman, 2016, p. 24), on the assumption describing a situation to understanding it, to question- that the individual mirrors a social group (Walford, ing it and to changing it. This parallels the stages of 2009, p. 276). Autoethnography differs from autobiog- ideology critique set out in Chapter 3: raphy in that in the latter the focus is only on the self, whilst in the former the focus is on the self in context, Stage 1: a description of the existing situation – a typically a socio-c ultural context. Denshire (2014) hermeneutic exercise; notes that autoethnography moves beyond autobiogra- phy ‘whenever writers critique the depersonalizing ten- Stage 2: a penetration of the reasons that brought the dencies that can come into play in social and cultural situation to the form that it takes; spaces that have asymmetrical relations of power’ (p. 833). Autoethnography here concerns how one is Stage 3: an agenda for altering the situation; ‘othered’ (Hamilton et al., 2008, p. 22) and how one’s Stage 4: an evaluation of the achievement of the new ‘positionality’ (discussed later in this chapter) affects the researcher and what is researched (Starr, 2010). situation. Here emphasis is placed on the writing of the research in a personal, authentic, vivid, engaging and Critical ethnographies can also be conducted online (Evans, 2010), and we turn to this later in the present chapter. 297
Methodologies for educational research evocative style, ‘writing from the heart’ (Denzin, 2006, distort vivid experiences, raising issues of the credibil- p. 422) and celebrating the researcher’s ‘voice’ (Wall, ity and trustworthiness of the report, and these are 2006, p. 3). As Ellis and Bochner (2006) note, auto ethical matters in themselves (cf. Wall, 2006, 2008; ethnography catches passion, feelings, struggles, i.e. to Bochner, 2007; Walford, 2009). evoke the empathy, emotions and sympathy of the Further, in seeking an expressive, evocative style of reader, indeed for the reader to take action (p. 433), writing with the intent of reconstructing the authenticity with ideas grounded in the personal experiences of their of a lived experience and persuading, touching or moving author, and written in a way that promotes empathy the reader, there is the danger of subordination of the between readers and the ‘other’ in research. facts of the case to the emotional response; whether this By contrast, Anderson (2006) and Atkinson (2006) is legitimate is a moot point – it may be acceptable or out argue for an ‘analytic’ rather than ‘evocative’ stance to of court. Anderson (2006), for example, as mentioned doing and writing autoethnography, and Anderson above, argues for a more analytical than evocative (2006) sets out five features of analytic autoethnogra- approach to writing autoethnography, whilst Denshire phy: complete member research status (the researcher (2014) argues that autoe thnography is an essential part of is a member of the social world being studied); analytic a ‘fictive tradition’ (p. 836). reflexivity (an awareness of, and introspection about, In terms of method, autoethnography combines the reciprocal influence of settings, data and researcher) autobiography with ethnography (Ellis et al., 2011, (p. 382); narrative visibility of the researcher’s self; p. 2), as the researcher reviews personal experience dialogue with informants beyond the self; and commit- reflexively, usually retrospectively, and from this anal- ment to theoretical analysis. yses and distils key issues about that autobiography Autoethnography recognizes the unavoidable influ- from an ethnographic stance, i.e. what the personal ence of the researcher on the research process, and experiences say to the reader about culture, values, raises reflexivity (discussed below), subjectivity, emo- relations and society in relation to the topic of research tionality, personal characteristics of the researcher and interest. This may include writing about moments of autobiography to new prominence in the research (cf. existential crisis, turning points (‘epiphanies’) and life- Wall, 2006, 2008; Nicol, 2013). It focuses on, and changing moments. reflects on, the views, ‘confessional tales’ (Van Autoethnography uses the common tools of ethnog- Maanen, 1988) and analyses of the author on the per- raphy, such as field notes, documents, self‑observation sonal experiences of self and others included in his or and observation of others, interviews, dialogues and her experiences. The author is the participant, looking conversations (though see the comments later in this at himself/herself in socio-c ultural locations and terms. chapter about interviews), thick descriptions, reflexiv- In implicating others (often family members, friends ity, grounded theory, long-term attachment and obser- and social contacts) in that personal ethnography, vations of events, times, locations, personal ethical issues are raised concerning the confidentiality, accoutrements such as clothing and artefacts, relation- anonymity, privacy, safety and protection, non- ships, power and social life. identifiability and non-traceability of those others and An autoethnography is often written in the first their communities, and relations (sometimes intimate) person and uses emotional terms, in contrast to much between the researcher and his/her circle of contacts, standard academic writing which deliberately adopts a social circles and workplace groups. Not only are the third-p erson, passive voice and neutral, objective tone. ‘others’ vulnerable, but so is the researcher, the subject It is often written as a story or narrative and as a per- of the autoethnography, as self‑disclosure about sonal experience (Marshall and Rossman, 2016, p. 24), sensitive personal issues can be damaging (Ngunjiri et with an aesthetic sense as well as a factual basis (Ellis al., 2010). et al., 2011). Such storied texts may focus on inequal- The consequences of the written autoethnography ity, oppression, exploitation, subordination, lack of for the author and others included have to be consid- understanding or acceptance (e.g. issues of gender, sex- ered (e.g. Wall, 2006, 2008). This may lead to the need uality, race), injustice, marginalization, stigmatization for respondent validation (raising issues of what and ‘dominant discourses’ (Ellis and Bochner, 2000; happens if the respondent wishes to veto data) (Bettez, Wall, 2008). Less politically or critically, autoethnog- 2015), or for the removal of identifying features, or raphy may concern issues or experiences that are changing identifying features (e.g. gender, race, loca- important to the researchers (e.g. Dyson, 2007; Nicol, tion, appearance) (Ellis et al., 2011). The researcher also 2013). Writing an autoethnography can be therapeutic has to consider the danger of selective memory on his/ and cathartic as well as constituting a method of her part, for example, we may recall but unconsciously enquiry (Richardson, 2000; Roth, 2009). 298
Qualitative, naturalistic and ethnographic research Autoethnography is not without its critics (Ander- communication is a routine and integral part of peo- son, 2006; Delamont, 2007, 2009; Ellis et al., 2011). ple’s everyday lives, and, given this, its part in ethno- For example, it is accused of being an indulgence of the graphic research is unsurprising. Researchers can enter writer, conflating autobiography with research, lacking the Internet and study what is happening in and through analytical and theoretical rigour, failing to generate or it with respect to cultures and communities; the Internet to test hypotheses or theories, bringing emotions into is ‘a place where cultural and social phenomena what should be neutral work, and making reflexivity a happen’ (Webster and da Silva, 2013, p. 123) and thing in itself rather than a tool of ethnography (e.g. where ethnographic interviews can be conducted online Atkinson, 1997; Sparkes, 2000; Wall, 2006). It might (Hanna, 2012). be good for the writer, privileging the self (Hamilton et The Internet is a ‘socially constructed space’ (Mar- al., 2008, p. 17), but of little use to others. shall and Rossman, 2016, p. 30), albeit a virtual space Delamont (2007) argues trenchantly that ‘autoeth- (Hine, 2000, 2004), peopled by interacting participants nography is essentially lazy, literally lazy and also intel- with real and virtual lives, their own cultures, online lectually lazy’ (p. 1), that it cannot fight familiarity, that communities, groups, rites of passage, negotiated roles, it violates ethical standards of privacy and permissions group membership and behaviours, and these can be for identifying individuals in published research, that it researched as one would conduct an ethnography. The sacrifices analytical outcomes to reporting of experi- virtual, online environment is the site for the research ence, that it focuses on the ‘wrong side of the power (Evans, 2010), requiring different, computer-b ased divide’ (p. 5), that it abrogates the duty of the social sci- tools for conducting the research. The computer screen entist to go out and gather data rather than ‘sit in our is the on-s creen location of the research, and the major- offices obsessing about ourselves’ (p. 5), that we are not ity of the data is likely to be text-b ased, though this sufficiently interesting to warrant attention by others, does not preclude other data types which are increas- and that it is antithetical to two tenets of social science, ingly available on the Internet, for example, Skype, which are to study the social world and to move the dis- Blackboard Collaborate (Webster and da Silva, 2013). cipline forward. In short, as she writes (Delamont, ‘Virtual ethnography’ (Hine, 2004), netnography 2009), it is an ‘intellectual cul de sac’ (p. 51). (Kozinets, 2010), netography and ‘webnography’ Critics contend that it lacks genuine fieldwork and is (Evans, 2010) can be conducted though social network- the apotheosis of navel-g azing, narcissism and ing media, email, online interviews, message boards self‑absorption (e.g. Atkinson, 2006; Madison, 2006; and messaging, bulletin boards, blogs, chat rooms, Delamont, 2007, 2009; Ellis et al., 2011), i.e. it is more forums, discussion boards (see Chapters 23 and 25). about the ‘auto’ than the ‘ethnography’ (Atkinson, The researcher, as in traditional ethnography, is still a 2006, p. 402; Roth, 2009, p. 5). It stands accused of an participant observer or non-p articipant observer (Evans, absence of social context, social action and interaction, 2010), permanently or intermittently immersed in and of not being sufficiently social to qualify as social observing the virtual environment and what is happen- science, and of operating in a social vacuum (Atkinson, ing in it, keeping systematic field notes (Hine, 2000). 1997, p. 339). Delamont (2009), noting that ethno- Because virtual ethnography works with virtual graphic research is demanding and hard, derides people and alter egos (e.g. avatars), the researcher is autoethnography as ‘an abrogation of the honourable often deprived of assurances of honesty and of several trade of the scholar’ (p. 61). features of face‑to-face ethnography conducted in the Such criticisms have been roundly refuted, arguing physical presence of the ethnographer and the ‘real’ that differences of views of research should be participants in their real, physical, ‘natural habitat’ celebrated rather than proscribed (e.g. Ellis and (Hallett and Barber, 2014, p. 306) (e.g. knowledge of Bochner, 2000; Bochner, 2001; Denzin, 2006; Wall, gender, race, age, social status) (Hammersley, 2006, 2006, 2008; Starr, 2010; Ellis et al., 2011; Denshire, p. 8); it works as if participants are real – which they 2014). may or may not be – and the ‘natural habitat’ is now the ‘online habitat’ (Hallett and Barber, 2014, p. 308). 15.6 Virtual ethnography Netographies overcome problems of time, location and space; they enable the anonymity, privacy and As the Internet is a means of searching a repository of security of the real people to be respected, though this knowledge, a means of communication and a venue for renders problematic the issues of identity and authen- connecting people – real or virtual (Marshall and ticity of the world being investigated. In short, the Rossman, 2016, p. 30) – so the cyberworld has, itself, virtual world is a projection, true or false, of the face- become a source of ethnographic research. Online to-face world; it may be no more or less real because of 299
Methodologies for educational research this (Boellstorff, 2015). The ethnographer proceeds as Then the researcher will need to write the ethnogra- if the Internet world is the real thing, working with the phy and the report, and seek respondent validation and data provided on the Internet, with few, if any, checks member checks. This sequence echoes Kozinets’s on the correctness or authenticity of the actual people (2010) comments that the methods of traditional eth- behind the avatars. As with other forms of online nographers – gaining entry to the field and community, research, virtual ethnographies raise ethical issues of collecting data, careful analysis and interpretation of confidentiality, privacy, anonymity, disclosure, protec- data, and reporting, all couched within ethical princi- tion from harm to self and others, and informed consent ples – apply to netographers. (see also Chapter 8). Whilst online research catches some of the social For educationists, virtual ethnography can focus on space and topical issues in the community of partici- ‘real people’ in their virtual communities (Hallett and pants, whether this is sufficient to be called a true, fully Barber, 2014, p. 310), and on the data which they fledged, genuine ethnography in the traditional sense of provide online rather than focusing on virtual people or catching the all-round, overall, holistic picture of par- avatars. This questions how far these are real, full- ticipants and their socio-c ultural settings, is an open blooded ethnographies or simply partial and selective question. They are communities united by, or formed data posted online about specific topics of communal or by, a common interest rather than having any other shared interest by interested parties, i.e. extended dis- connections. cussions or sharing of opinions. Indeed Evans (2010) Whilst traditional ethnography sees participants in question whether a virtual ethnography is, in fact, more many settings, presenting many faces and aspects of like an extended online survey than an ethnography self to many parties, and whilst participants may defined as a ‘faithful reproduction of a particular cul- present different faces in virtual ethnographies, whether tural setting’ (p. 7). this happens sufficiently in a virtual ethnography for it In conducting a virtual ethnography, Evans (2010) to be counted as a full-b looded ethnography (rather suggests that researchers identify relevant ‘proactive than, for example, differing views on a topic or communities’ (p. 9), raising issues of access, gatekeep- different stories from participants) is another open ers and ethical issues of privacy, anonymity, informed question. consent, covert or overt research, and permissions. Webster and da Silva (2013) and Hallett and Barber Then researchers can identify key informants and key (2014) suggest that, in reality, to conduct a full ethnog- participants, negotiating access to people and groups raphy could require researchers to study the same par- and addressing the same ethical issues, with informed ticipants both online and offline, as the online world is consent including both the process and product of the as much part of their ‘real’ daily lives as the offline, ethnography, and the audience and dissemination of the face-to-face, physical interactions. It is a false dualism ethnography. Kulavuz-Onal and Vásquez (2013) to separate the online and offline worlds of participants. remind researchers that they may need to register as a member of an online community in order to gain access 15.7 Phenomenological research and may need to be an active participant in some events whilst being able to be less active in others (p. 229). Phenomenological research is based on the view that After this, the researcher can make further contact in our knowledge of the world is rooted in our (immediate) order to commence the research, engaging in interac- experiences, and the task of the researcher is to describe, tion with the participants (if participant observation is understand, interpret and explain these experiences selected) or being a non‑participant observer (though (Hammersley, 2013, p. 27; Denscombe, 2014, pp. 94–5). Kozinets (2010) advocates participant observation). This type of research aims to describe, explain and The researcher gathers ongoing systematically collected interpret a phenomenon, situation or experience by and systematically reviewed data and field notes identifying the meaning of these as understood by the (Kulavuz-O nal and Vásquez, 2013); indeed, online, participants, often at an individual as well as a group digital data (including online interviews, see Chapter level (Marshall and Rossman, 2016, pp. 16–17). 25) may lend themselves to software for data analysis. As there are many participants involved, each of Kulavuz-O nal and Vásquez (2013) comment that whom has his/her own authentic meaning and interpre- fieldwork practices in ethnographies of online com tation, there will be multiple realities and accounts; the munities need to be ‘reconceptualised’ (p. 237) researcher has to put to one side any prior concepts or because they differ from practices in ‘in-person ethno- suppositions (pp. 27–8) and seek to understand how graphic fieldwork’, being software based and computer everyday events and ‘commonsense knowledge’ mediated. (p. 28) are as they are, how they are perceived and 300
Qualitative, naturalistic and ethnographic research sustained by the participants, and what are the attitudes 15.8 Planning qualitative, of participants towards them. In this enterprise, empha- naturalistic and ethnographic sis is placed on the fully described subjective experi- research ences, perceptions, interpretations, attitudes, beliefs, values, feelings and meanings of agentic individuals In many ways the planning issues in qualitative (Denscombe, 2014, p. 94). In full depiction of lived research are not exclusive; they apply to other forms of experiences through the eyes of participants, in research, for example: identifying the problem and come rich description and fidelity to the original research purposes; deciding the focus of the study; experience and out go categorization, abstraction, identifying the research questions; selecting the over‑interpretation, quantification and even theoriza- research design and instrumentation; addressing valid- tion (pp. 95–6). ity and reliability; ethical issues; approaching data Ary et al. (2002, p. 447) note that, whilst this is analysis and interpretation. These are common to all common to much qualitative research, the distinctive research. More specifically, Wolcott (1992, p. 19) sug- feature of phenomenological research is its focus on the gests that naturalistic researchers should address the subjective experiences of the participants, which are at stages of watching, asking and reviewing, or, as he puts the heart of the research; what they mean for the partic- it, experiencing, enquiring and examining. It is possible ipants rather than, for example, the objective ‘status of to formulate stages in planning naturalistic research experiences’ (p. 447). Not only is there an individual (Hitchcock and Hughes, 1989, pp. 57–71; Bogdan and construction of reality, but a social construction of Biklen, 1992; LeCompte and Preissle, 1993). These are reality (Berger and Luckman, 1967), i.e. there is a presented in Figure 15.2 and are subsequently dealt shared rather than a solipsistic understanding of the with in the later pages of this chapter. ‘real’, with shared and multiple realities. One has to be cautious here: Figure 15.2 suggests a To understand the meanings that participants give to linearity in the sequence; in fact, the process is often the experiences typically requires in-depth, open-e nded more complex that this, with a backwards-and-forwards and often unstructured interviews with the participants movement between the several stages over the course (Marshall and Rossman, 2016, p. 18), which seek to of the planning and conduct of the research. The grasp the essence of the meaning(s) of a situation as process is iterative and recursive, as different elements given by each participant, with detailed descriptions come into focus and interact with each other in differ- figuring highly here and a recognition that complexity ent ways at different times. Indeed Flick (2009, p. 133) rather than unity or simplicity may be the hallmarks of suggests a circularity or mutually informing nature of the meanings given. The researcher has to strive to set elements of a qualitative research design. In this aside any of his/her own values, beliefs, taken-for- instance the stages of Figure 15.2 might be better pre- granted conceptual frameworks, predispositions and sented as interactive elements as in Figure 15.3. everyday background and to see the experience for Further, in some smaller-s cale qualitative research what it is in the eyes of the participants, freed from not all of these stages may apply, as the researcher may such researcher preconceptions, in other words to act as not always be staying for a long time in the field but a ‘stranger’ (Denscombe, 2014, p. 99). might only be gathering qualitative data on a ‘one-s hot’ Denscombe writes that phenomenology is suited to basis (e.g. a qualitative survey, qualitative interviews). small-s cale research, descriptive detail of authentic However, for several kinds of naturalistic and ethno- experiences and sympathy to humanistic research graphic study in which the researcher intends to remain which focuses on ‘lived experiences’. On the other in the field for some time, the several stages set out hand, he notes that, in its pursuit of rich, individualized here, and commented upon in the following pages, may descriptions, phenomenological research may lack the apply. scientific tenets of ‘objectivity, analysis and measure- These stages are shot through with a range of issues ment’ (p. 103), may not move beyond description (e.g. that affect the research, for example: to analysis and explanation), may not be generalizable and may focus on trivial everyday events to the neglect OO personal issues (the disciplinary sympathies of the of bigger issues (p. 103). researcher, researcher subjectivities and characteris- tics, personal motives and goals of the researcher). Hitchcock and Hughes (1989, p. 56) indicate that there are serious strains in conducting fieldwork because the researcher’s own emotions, attitudes, beliefs, values, characteristics enter the research; indeed, the more this 301
Methodologies for educational research Stage 1 OO role relationships; Locating a field of study OO boundary maintenance in the research; OO the maintenance of the balance between distance Stage 2 Formulating research questions and involvement; OO ethical issues; OO reflexivity. Stage 3 15.9 Reflexivity Addressing ethical issues Reflexivity is a central component of, and a ‘crucial Stage 4 strategy’ in, qualitative research (Berger, 2015). Deciding the sampling Researchers have a central role in the creation of knowledge in qualitative enquiry, hence they need to Stage 5 look at themselves and their ‘positionality’ (discussed Finding a role and managing entry into the context later) as part of the research process. Reflexivity recog- nizes that researchers are inescapably part of the social Stage 6 world that they are researching (Hammersley and Finding informants Atkinson, 1983, p. 14; Atkinson, 2006, p. 402), and, indeed, that this social world is one already interpreted Stage 7 by the actors, undermining the notion of objective Developing and maintaining relations in the field reality. Researchers are in the world and of the world that they research. They bring their own biographies Stage 8 and values to the research situation and participants Data collection in situ behave in particular ways in their presence. As Dens- combe (2014, p. 88) notes, the researcher does not Stage 9 commence the research ‘with a clean sheet’, but uses Data collection outside the field conceptual tools which derive from several sources, including culture and values. What we focus on, what Stage 10 we see, how we understand, describe, interpret and Data analysis explain are shaped by ourselves and what we bring to the situation. We cannot stand outside these. Stage 11 Qualitative inquiry is not a neutral activity, and Leaving the field researchers are not neutral; they have their own values, biases and world views, and these are lenses Stage 12 through which they look at and interpret the already- Writing the report interpreted world of participants (cf. Preissle, 2006, p. 691). Researcher bias is a key issue in qualitative FIGURE 15.2 Stages in the planning of naturalistic, research (as it is with quantitative research) (Hammers- qualitative and ethnographic research ley, 1992a). Researchers, then, have to self-a ppraise their role in happens the less will be the likelihood of gaining the the research process and product (Berger, 2015, p. 220). participants’ perspectives and meanings; Pillow (2010) and Bettez (2015) note that reflexive OO the kinds of participation that the researcher will researchers bring their own personal characteristics, undertake; experiences, knowledge, backgrounds, values, beliefs, OO issues of advocacy (where the researcher may be theories, age, gender, sexuality, politics, theories, race, expected to identify with the same emotions, con- ethnicity, conceptual frameworks and prejudices to the cerns and crises as the members of the group being research and that these are often mediated through, and studied and wishes to advance their cause, often a are in conjunction with, issues of power and status. feature that arises at the beginning and the end of They influence every stage of the research and affect the research when the researcher is considered to be the rapport and conduct of the research. They can affect a legitimate spokesperson for the group); the formulation of the research topic and questions, access to the field, relationships with participants, data collection, analysis and interpretation, insider and 302
Qualitative, naturalistic and ethnographic research Research Deciding questions role and managing Deciding field of entry study Ethical Sampling issues Data QUALITATIVE Data collection RESEARCH collection outside in the field DESIGN field Data analysis Locating informants Field Leaving relations the field Writing the report FIGURE 15.3 Elements of a qualitative research design outsider research, and so on. In short, the researcher ground, values and inductive processes, frames and para- may project something or a lot about themselves onto digms shape the research. They are research instruments. the research (Berger, 2015). McCormick and James (1988, p. 191) argue that combat- Reflexivity suggests that researchers should con- ing reactivity through reflexivity requires researchers to sciously and deliberately acknowledge, interrogate and monitor closely and continually their own interactions disclose their own selves in the research, seeking to with participants, their own reactions, roles and biases, understand their part in, and influence on, the research. and any other matters that might affect the research. This Rather than trying to eliminate researcher effects is addressed more fully in Chapter 14 on validity, encom- (which is impossible, as researchers are part of the passing issues of triangulation and respondent validity. world that they are investigating), researchers should hold themselves up to the light, echoing Cooley’s 15.10 Doing qualitative research (1902) notion of the ‘looking glass self ’, and research- ers should go beyond private reflection on how their An effective qualitative study has several features own biographies and backgrounds have influenced the (Creswell, 1998, pp. 20–2), and these can be addressed research and disclose this publicly as part of the neces- in evaluating qualitative research: sary transparency of the research. Highly reflexive researchers will be acutely aware of OO it uses rigorous procedures and multiple methods for the ways in which their selectivity, perception, back- data collection; 303
Methodologies for educational research OO the study is framed within the assumptions and wrong clothes: it inhibits comfort and the ability to nature of qualitative research; work properly. Maxwell notes that theoretical premises may not always be clear at the outset of the research; OO enquiry is a major feature, and can follow one or they may emerge, change, be added to etc. over time more different traditions (e.g. biography, ethnogra- as the qualitative research progresses (see Chapters 1 phy, phenomenology, case study, grounded theory); and 4 on paradigms and theories). Theory, Maxwell avers (p. 43), can provide a supporting set of prin OO the project commences with a focus on an issue, a ciples, world view or sense-m aking referent, and it can group, a problem rather than having a hypothesis or be used as a ‘spotlight’, illuminating something very the supposition of a causal relationship of variables; specific in a particular event or phenomenon. He advo- relationships may emerge later, but that is open; cates a cautious approach to the use of theory (p. 46), steering between, on the one hand, having it unneces- OO criteria for verification are set out, together with sarily constrain and narrow a field of investigation and rigour in writing the report; being accepted too readily and uncritically, and, on the other hand, not using it enough to ground rigorous OO verisimilitude is required, such that readers can research. Theories in qualitative research should be imagine being in the situation; those of the researcher and the participants. He suggests that theory can provide the conceptual and OO data are analysed at different levels; they are justificatory basis for the qualitative research being multilayered; undertaken, and it can also inform the methods and data sources for the study (p. 55). OO the writing engages the reader and is replete with Against this background, we set out a twelve-s tage unexpected insights, whilst maintaining believabil- process for doing qualitative research. ity and accuracy. Stage 1: locating a field of study Maxwell (2005, p. 21) argues that qualitative research should have both practical goals (e.g. that can be Bogdan and Biklen (1992, p. 2) suggest that research accomplished, that deliver a specific outcome and meet questions in qualitative research are not framed by a need) and intellectual goals (e.g. to understand or simply operationalizing variables as in the positivist explain something). His practical goals (p. 24) are: (a) paradigm. Rather, they are formulated in situ and in to generate ‘results and theories’ that are credible and response to situations observed, i.e. topics are investi- comprehensible to participants and other readers; (b) to gated in all their complexity in the naturalistic context. conduct formative evaluation in order to improve prac- In some qualitative studies, the selection of the tice; and (c) to engage in ‘collaborative and action research field will be informed by the research purposes, research’ with different parties. His intellectual goals the need for the research, what gave rise to the research, (pp. 22–3) are: (a) to understand the meanings attrib- the problem to be addressed and the research questions uted to events and situations by participants; (b) to and sub-questions. In other qualitative studies these ele- understand particular contexts in which participants are ments may only emerge after the researcher has been located; (c) to identify unanticipated events, situations immersed for some time in the research site itself. and phenomena and to generate grounded theories that incorporate these; (d) to understand processes that con- Stage 2: formulating research questions tribute to situations, events and actions; and (e) to develop causal explanations of phenomena. Research questions are an integral and driving feature He suggests that, whilst quantitative research is of qualitative research. They must be able to be interested in discovering the variance and regularity in answered concretely, specifically and with evidence the effects of one or more particular independent vari (see Chapter 10). They must be achievable and finite ables on an outcome, qualitative research is interested (cf. Maxwell, 2005, pp. 65–78) and are often character- in the causal processes at work in understanding how ized by being closed rather than open questions. one or more interventions or factors lead to an outcome, Whereas research purposes can be open and less finite, the mechanisms of their causal linkages. Quantitative motivated by a concern for ‘understanding’, research research can tell us correlations, how much, whether questions, by contrast, though they are informed by and ‘what’, whilst qualitative research can tell us the research purposes, are practical and able to be accom- ‘how’ and ‘why’ – the processes involved in under- plished (Maxwell, 2005, pp. 68–9). standing and explaining how things occur. Hence, instead of asking a non-d irectly answerable Maxwell also argues that qualitative research question such as ‘how should we improve online should be based on a suitable theoretical basis or para- digm. Quoting Becker (1986), he argues that if a researcher bases his or her research in an inappropriate theory or paradigm it is akin to a worker wearing the 304
Qualitative, naturalistic and ethnographic research learning for biology students?’, we can ask a specific, Stage 3: addressing ethical issues focused, bounded and answerable question such as ‘how has the introduction of an online teacher–student chat Deyle et al. (1992, p. 623) and Hammersley and Tra- room improved Form 5 students’ interest in learning ianou (2012) identify several critical ethical issues that biology?’. Here the word ‘should’ (as an open question) need to be addressed in the research: how does one has been replaced with ‘has’, the general terminology of present oneself in the field? As whom does one present ‘online learning’ has been replaced with ‘an online oneself? How ethically defensible is it to pretend to be teacher–student chat room’ and the open-endedness of somebody that you are not in order to gain knowledge the first question has been replaced with the closed that you would otherwise not be able to acquire or to nature of the second (cf. Maxwell, 2005, p. 21). obtain and preserve access to places which otherwise Whereas in quantitative research, a typical research you would be unable to secure or sustain. question asks ‘what’ and ‘how much’ (e.g. ‘how much The issues here are several. First, there is the matter do male secondary students prefer female teachers of of informed consent (to participate and for disclosure), mathematics, and what is the relative weighting of the whether and how to gain participant assent (see also factors that account for their preferences?’), a qualita- LeCompte and Preissle, 1993, p. 66). Hammersley and tive research question often asks more probing, Traianou (2012) comment that the researcher must process-d riven research questions (e.g. ‘how do sec- respect the autonomy of the participants and this ondary school students in school X decide their prefer- means gaining informed consent and, where appropri- ences for male or female teachers of mathematics?’). ate, regarding participants as equals in the research Maxwell (2005, p. 75) suggests that qualitative project (they also note that researchers studying certain research questions are suitable for answering questions groups, e.g. paedophiles, rapists, elite power groups about: (a) the meanings attributed by participants to (p. 82) may not wish to regard them as equals). They situations, events, behaviours and activities; (b) the note that consideration has to be given to who gives influence of context (e.g. physical, social, temporal, consent, and on behalf of whom, and for what, and interpersonal) on participants’ views, actions and what ‘fully informed’ means (see Chapter 7 of the behaviours; and (c) the processes by which actions, present volume). behaviours, situations and outcomes emerge. Gaining consent also uncovers another considera- Whilst in quantitative research, the research ques- tion, namely covert or overt research. On the one hand tions (or hypotheses to be tested) typically drive the there is a powerful argument for informed consent. research and are determined at the outset, in qualitative However, the more participants know about the research a more iterative process occurs (Light et al., research the less naturally they might behave (Ham- 1990, p. 19). Here the researcher may have an initial set mersley and Traianou, 2012, p. 108), and naturalism is of research purposes, or even questions, but these may a key criterion of the naturalistic paradigm. change over the course of the research, as the researcher Mitchell (1993) catches the dilemma for researchers finds out more about the research setting, participants, in deciding whether to undertake overt or covert context and phenomena under investigation, i.e. decid- research. The issue of informed consent, he argues, can ing research questions is not a once-a nd-for-a ll affair. lead to the selection of particular forms of research – This is not to say that qualitative research is an unprin- those where researchers can control the phenomena cipled, aimless activity; rather it is to say that, whilst under investigation – thereby excluding other kinds of the researcher may have clear purposes, she is sensitive research where subjects behave in less controllable, to the emergent situation in which she finds herself, and predictable, prescribed ways, indeed where subjects this steers the research questions. Research questions may come in and out of the research over time. are the consequence, not the driver, of the situation and Mitchell argues that in the real social world access the interactions that take place within it. It is important to important areas of research is prohibited if informed for the qualitative researcher to ask the right questions consent has to be sought, for example, in researching rather than to ask about what turn out to be irrelevan- those on the margins of society or the disadvantaged. It cies to the participants. As Tukey (1962, p. 13) is to the participants’ own advantage that secrecy is remarked, it is better to have approximate, inexact or maintained as, if it is not, important work may not be imprecise answers to the right question than to have done and ‘weightier secrets’ (1993, p. 54) may be kept precise answers to the wrong question. The qualitative that are of legitimate public concern. Mitchell makes a researcher has to be sensitized to the emergent key fea- powerful case for secrecy, arguing that informed tures of a situation before firming up the research consent may excuse social scientists from the risk of questions. confronting powerful, privileged and cohesive groups 305
Methodologies for educational research who wish to protect themselves from public scrutiny. undertaking the research, and Blix and Wettergren Secrecy and informed consent are moot points. (2015) note that this can particularly feature when Patrick (1973) indicates this point sharply when, as gaining and maintaining access to the field and in build- an ethnographer of a Glasgow gang, he was witness to ing trust. Whilst the emotions of the researcher may, a murder; the dilemma was clear – to report the matter indeed, become part of the research data, this does not (and thereby to act legally but ‘blow his cover’, conse- obviate the ethical concern of ensuring that the research quently endangering his own life) or to stay as a covert does not harm the researcher. Emotional self- researcher, thereby breaking the law. As Ary et al. management is an issue (cf. Hochschild (2012) on (2002) remark, researchers may obtain knowledge of ‘emotion work’). unforeseen illicit activities, or even be part of those, A standard protection for participants is often the and this raises ethical dilemmas for them (p. 437). The guarantee of confidentiality and privacy, withholding researcher, then, has to consider her loyalties and participants’ real names and other identifying charac- responsibilities (LeCompte and Preissle, 1993, p. 106), teristics. The authors contrast this with anonymity, for example, what is the public’s right to know and where identity is withheld because it is genuinely what is the individual’s right to privacy? Researchers unknown (p. 106). Issues of identifiability and trace must decide ‘whose side are we on’ (Becker, 1967). ability are raised, and participants might be able to In addition to the issue of overt or covert research, identify themselves in the research report, though LeCompte and Preissle (1993) indicate that the prob- others may not be able to identify them. A related lems of risk to, and vulnerability of, subjects must be factor here is the ownership of the data and the results, addressed; steps must be taken to prevent risk or harm the control of the release of data (and to whom, and to participants (non-m aleficence – the principle of when) and what rights respondents have to veto the primum non nocere) (cf. Hammersley and Traianou, research results. 2012; see Chapter 7 of this volume). Bogdan and Positionality addresses relationships; it is an ethical Biklen (1992, p. 54) extend this to include issues of matter. Relationships between researcher and the embarrassment as well as harm to those taking part researched are rarely symmetrical in terms of power; it (e.g. harm from physical or psychological pain, mate- is often the case that those with more power, informa- rial damage, damage to a project in which people are tion and resources research those with less (Ham involved, damage to reputation or status) (Hammersley mersley and Traianou, 2012, p. 12). Bettez (2015) notes and Traianou, 2012, p. 62). Bettez (2015) asks what to that research knowledge, how it is produced, under- do with information from one participant that could be stood, evaluated and used, is affected by, or refracted emotionally painful for another. through, ‘positionalities’ – how we see ourselves and The question of vulnerability is present when partic- others – which are influenced by cultural values, ipants in the research have their freedom to choose beliefs, ascribed and achieved social position, status, limited, for example, by dint of their age, health, social gender, race, sexuality, insider and outsider status etc. constraints, by dint of their lifestyle, social acceptabil- She argues for ‘communion’ in qualitative research: ity, experience of being victims (e.g. of abuse, of meaningful connection between all participants (includ- violent crime) (Bogdan and Biklen, 1992, p. 107). As ing the researcher) in a spirit of mutual and shared the authors comment, participants rarely initiate equality, inclusion, respect, humanity and dignity. research, so it is the responsibility of the researcher to Bogdan and Biklen (1992, p. 54) add to this discus- protect them. sion the need to respect participants as subjects, not Ethical issues concern both those being researched simply as research objects to be used and then dis- and the researcher. As we mention in Chapter 13, carded. It is important for researchers to consider the research can also take its toll on the researcher, not only parties, bodies, practices that might be interested in, or in terms of the sensitivity or nature of the topic but in affected by, the research and the implications of the terms of the process of undertaking the enquiry itself, answers to these questions for the conduct, reporting which may be stressful, emotional and disturbing and dissemination of the inquiry (Mason, 2002, p. 41). (Dickson-S wift et al., 2008, 2009; Blix and Wettergren, This extends to exiting the research (Ary et al., 2002), 2015; Emerald and Carpenter, 2015). Emerald and Car- as the researcher may have built up strong relationships penter (2015) note that researchers often downplay the with the participants over the course of the research, emotional and personal risk of the research, in which and indeed is likely to have built up friendships which they may feel vulnerable and exposed (p. 744). They cannot be severed simply because the research has fin- must be aware of, and reflexive about, the emotional ished. The researcher performs a balancing act, as such signals they may be obtaining about themselves in friendships may develop during the research, and this 306
Qualitative, naturalistic and ethnographic research raises issues of mutual trust in reporting the results. The OO some members of a subset may not be drawn from issue also concerns reciprocity and respect: how do the the population from which the sampling is intended participants benefit from the research? to be drawn. We address ethics in Chapters 7 and 8 and we advise readers to refer to these. Hence other types of sampling are required. A criterion- b ased selection requires the researcher to specify in Stage 4: deciding the sampling advance a set of attributes, factors, characteristics or criteria that the study must address. The task is to In an ideal world the researcher would be able to study ensure that these appear in the sample selected (the a group in its entirety: a population. This was the case equivalent of a stratified sample). There are other forms in Goffman’s (1968) work on ‘total institutions’, for of sampling (see Chapter 12) that are useful in ethno- example, hospitals, prisons and police forces (see also graphic research (Patton, 1980; Guba and Lincoln, Chapter 35). It was also the practice of anthropologists 1989; Bogdan and Biklen, 1992, p. 70; LeCompte and who were able to explore specific isolated communities Preissle, 1993, pp. 69–83; Ezzy, 2002), such as: or tribes. That is rarely possible nowadays because such groups are no longer isolated or insular. Hence the OO convenience sampling (opportunistic sampling, researcher is faced with the issue of sampling, that is, selecting from whomever happens to be available); deciding which people it will be possible to select to represent the wider group (however defined). The OO critical-c ase sampling (e.g. people who display the researcher has to decide the groups for which the issue or set of characteristics in their entirety or in a research questions are appropriate, the contexts which way that is highly significant for their behaviour). are important for the research, the time periods needed This is done in order to permit maximum applicabil- and the possible issues and artefacts of interest to the ity to others: if the information holds true for critical investigator. This takes the discussion beyond conven- cases (e.g. cases where all of the factors sought are tional notions of sampling. present), then it is likely to hold true for others; In several forms of research, sampling is fixed at the start of the study, though there may be attrition of the OO extreme-/deviant-c ase sampling (the norm of a char- sample through ‘mortality’ (e.g. people leaving the acteristic is identified, then the extremes of that char- study), and this is problematic. Ethnographic research acteristic are located and, finally, the bearers of that regards this as natural rather than irksome. People come extreme characteristic are selected). This is done in into and go from the study. This impacts on the deci- order to gain information about unusual cases that sion whether to have a synchronic investigation at a may be particularly troublesome or enlightening; single point in time, or a diachronic study where events and behaviour are monitored over time to allow for OO typical-c ase sampling (where a profile of attributes change, development and evolving situations. In ethno- or characteristics that are possessed by an ‘average’, graphic inquiry sampling is recursive and ad hoc rather typical person or case is identified, and the sample is than fixed at the outset; it changes and develops over selected from these conventional people or cases). time. Let us consider how this might happen. This is done in order to avoid rejecting information LeCompte and Preissle (1993, pp. 82–3) point out on the grounds that it has been gained from special that ethnographic methods rule out statistical sampling, or deviant cases; for a variety of reasons: OO unique-c ase sampling, where cases that are rare, OO the characteristics of the wider population are unique or unusual on one or more criteria are identi- unknown; fied, and sampling takes places within these. Here whatever other characteristics or attributes a person OO there are no straightforward boundary markers (cat- might share with others, a particular attribute or egories or strata) in the group; characteristic sets that person apart; OO generalizability, a goal of statistical methods, is not OO reputational-c ase sampling, a variant of extreme- necessarily a goal of ethnography; case and unique-c ase sampling, where a researcher chooses a sample on the recommendation of experts OO characteristics of a sample may not be evenly dis- in the field; tributed across the sample; OO snowball sampling: using the first interviewee to OO only one or two subsets of a characteristic of a total suggest or recommend other interviewees, and so on; sample may be important; OO maximum variation sampling. This is done in OO researchers may not have access to the whole order to document the range of unique changes that population; have emerged, often in response to the different conditions to which participants have had to adapt. 307
Methodologies for educational research It is useful if the aim of the research is to investigate OO to examine critical cases or extreme cases that the variations, range and patterns in a particular phe- provide a ‘crucial test’ of theories or that can illumi- nomenon or phenomena; nate a situation in ways which representative cases OO intensity sampling: according to the intensity with may not be able to do; which the features of interest are displayed or occur; OO sampling politically important or sensitive cases. OO to identify reasons for similarities and differences This can be done to draw attention to the case; between individuals or settings (comparative OO convenience sampling. This saves time and money research). and spares the researcher the effort of finding less amenable participants. He notes that methods of data collection and sampling are not a logical corollary of, nor an analytically neces- One can add to this list, from Miles and Huberman sary consequence of, the research questions (p. 91). (1994, p. 28): Research questions and data collection are two con ceptually separate activities, though, as we have men- OO homogeneous sampling (which focuses on groups tioned earlier in this book, the researcher needs to with similar characteristics); ensure that they are mutually informing, in order to demonstrate cohesion and fitness for purpose. OO theoretical sampling (in grounded theory, discussed Methods and sampling cannot simply be cranked out, below, where participants are selected for their ability mechanistically, from research questions; rather the to contribute to the developing/emergent theory); methods of data collection and the research questions are strongly influenced by the setting, the participants, OO confirming and disconfirming cases (akin to the relationships and the research design as they unfold extreme- and deviant-c ase sampling), in order to over time. look for exceptions to the rule, which may lead Lincoln and Guba (1985, pp. 201–2) suggest an to the modification of the rule; important difference between ‘conventional’ and natu- ralistic research designs. In the former the intention is OO random purposeful sampling (when the potential to focus on similarities and to be able to make generali- sample is too large, a smaller sub-sample can be zations, whereas in the latter the objective is informa- used which still maintains some generalizability); tional, to provide such a wealth of detail that the uniqueness and individuality of each case can be repre- OO stratified purposeful sampling (to identify sub- sented. To the charge that naturalistic inquiry, thereby, groups and strata); cannot yield generalizations because of sampling flaws, the writers argue that this may be necessarily though OO criterion sampling (all those who meet some stated trivially true, i.e. unimportant. criteria for membership of the group or class under Patton (1980, p. 184) suggests that ‘there are no study); rules for sample size in qualitative inquiry’, with the size of the sample depending on what one wishes to OO opportunistic sampling (to take advantage of unan- know, the purposes of the research, what will be useful ticipated events, leads, ideas, issues). and credible and what can be done within the resources available, for example, time, money, people, support – Miles and Huberman make the point that these strate- important considerations for the novice researcher. gies can be used in combination as well as in isolation, In much qualitative research, it may not be possible, and that using them in combination contributes to or, indeed, desirable, to know in advance whom to triangulation. sample or whom to include. One of the features of Patton (1980, p. 181) and Miles and Huberman qualitative research is its emergent nature. Hence the (1994, pp. 27–9) also note the dangers of convenience researcher may only know which people to approach or sampling, arguing that, being ‘neither purposeful nor include as the research progresses and unfolds (Flick, strategic’ (Patton, 1980, p. 88), it cannot demonstrate 2009, p. 125). In this case the nature of sampling is representativeness even to the wider group being determined by the emergent issues in the study; this is studied, let alone to a wider population. ‘theoretical sampling’ (Glaser and Strauss, 1967, p. 45): Maxwell (2005, pp. 89–90) indicates four possible once data have been collected, the researcher decides purposes of ‘purposeful selection’: where to go next, in light of the analysis of the data, in order to gather more data in order to develop his or her OO to achieve representativeness of the activities, behav- theory (Flick, 2009, p. 118). iours, events, settings and individuals involved; OO to catch the breadth and heterogeneity of the popu- lation under investigation (i.e. the range of the pos- sible variation: the ‘maximum variation’ sampling discussed above); 308
Qualitative, naturalistic and ethnographic research Ezzy (2002, p. 74) underlines the importance of related to disruptive behaviour. In this case the sam- ‘theoretical sampling’ in his comment that, unlike other pling emerges as the research proceeds and the theory forms of research, qualitative inquiries may not always emerges; this is theoretical sampling, the ‘royal way for commence with the full knowledge of whom to sample, qualitative studies’ (Flick, 2004b, p. 151). Schatzman but the sample is determined on an ongoing, emergent and Strauss (1973, pp. 38ff.) suggest that theoretical basis. Theoretical sampling starts with data and then, sampling may change sampling according to time, having reviewed these, the researcher decides where to place, individuals and events. go next to collect data for the emerging theory (Glaser The above procedure accords with Glaser’s and and Strauss, 1967, p. 45). Strauss’s (1967) view that sampling involves continu- In theoretical sampling, individuals and groups are ously gathering data until practical factors (bounda- selected for their potential – or hoped-for – ability to ries) put an end to data collection, or until no offer new insights into the emerging theory, i.e. they amendments have to be made to the theory in light of are chosen on the basis of their significant contribution further data – their stage of ‘theoretical saturation’ – to theory generation and development. As the theory where the theory fits the data even when new data are develops, so the researcher decides whom to approach gathered. Theoretical saturation is described by Glaser to request their participation. Theoretical sampling and Strauss (1967, p. 61) as being reached when, even does not claim to know the population characteristics when further data are used, the properties of the cate- or to represent known populations in advance, and gory in question are not developed any further. That sample size is not defined in advance; sampling is said, the researcher has to be cautious to avoid prema- only concluded when theoretical saturation (discussed ture cessation of data collection; it would be too easy below) is reached. We discuss this more fully in to close off research with limited data, when, in fact, Chapter 37. further sampling and data collection might lead to a In the educational field one could imagine theoreti- reformulation of the theory. cal sampling in the following example: interviewing An extension of theoretical sampling is ‘analytic teachers about their morale might give rise to a theory induction’, a process advanced by Znaniecki (1934). that teacher morale is negatively affected by disruptive Here the researcher starts with a theory (that may have student behaviour in schools. This might suggest the emerged from the data as in grounded theory) and then need to sample teachers working with many disruptive deliberately proceeds to look for deviant or discrepant students in difficult schools, as ‘critical-c ase sampling’. cases, to provide a robust defence of the theory. This However, the study finds that some of the teachers accords with Popper’s notion of a rigorous scientific working in these circumstances have high morale, not theory having to stand up to falsifiability tests. In ana- least because they have come to expect disruptive lytic induction, the researcher deliberately seeks data behaviour from students with so many problems and so which potentially could falsify the theory, thereby are not surprised or threatened by it, and because the giving strength to the final theory. staff in these schools provide tremendous support for We are suggesting here that, in qualitative research, each other in difficult circumstances – they all know sampling cannot always be decided in advance on a what it is like to have to work with challenging ‘once-a nd-for-all’ basis. It may change through the students. stages of data collection, analysis and reporting. Data So the study decides to focus on teachers working in collection, analysis, interpretation and reporting and schools with far fewer disruptive students. The sampling do not necessarily proceed in a linear researcher discovers that it is these teachers who expe- fashion; the process is recursive and iterative. Sam- rience far lower morale, and she hypothesizes that this pling is not decided a priori – in advance – but may be is because this latter group of teachers has higher decided, amended, added to, increased and extended expectations of student behaviour, such that having as the research progresses. Indeed, whilst sampling only one or two students who do not conform to these often refers to people, in qualitative research it expectations deflates staff morale significantly, and also refers to events, places, times, behaviours, activi- because disruptive behaviour is regarded in these ties, settings and processes (cf. Miles and Huberman, schools as teacher weakness, and there is little or no 1984, p. 36). mutual support. Her theory, then, is refined, to suggest Many researchers will conduct short-term, small- that teacher morale is affected more by teacher expec- scale qualitative research (e.g. qualitative interviews) tations than by disruptive behaviour, so she adopts a rather than extended or large-s cale ethnographic ‘maximum variation sampling’ of teachers in a range of research. A fundamental question for the researcher is schools, to investigate how expectations and morale are to decide how long to stay in a situation. Too short, and 309
Methodologies for educational research she may miss an important outcome; too long, and key Wolff (2004, pp. 195–6) suggests that there are two features may become a blur. fundamental questions to be addressed in considering For example, let us imagine a situation of two teach- access and entry into the field: ers in the same school. Teacher A introduces collabora- tive group work to a class, in order to improve their 1 How can the researcher succeed in making contact motivation for, say, learning a foreign language. She and securing cooperation from informants? gives them a pre-test on motivation, and finds that it is low; she conducts the intervention and then, at the end 2 How can the researcher position herself/himself in of two months, gives them another test of motivation, the field so as to secure the necessary time, space, and finds no change. She concludes that the interven- social relations to be able to carry out the research? tion has failed. However, months later, after the inter- vention has finished, the students tell her that, in fact, Flick (1998, p. 57), summarizing Wolff ’s work, identi- their overall motivation to learn that foreign language fies several issues concerning entering institutions for had improved, but it took time for them to realize it conducting research: after the intervention. Teacher B tries the same inter- vention, but decides to administer the post-test one year 1 Research is always an intrusion and intervention after the intervention has ended; she finds no change to into a social system, and, so, disrupts the system to motivation levels of the students, but had she conducted be studied, such that the system reacts, often the post-test sooner, she would have found a difference. defensively. Timing and sampling of timing are important. 2 There is a ‘mutual opacity’ between the social Stage 5: finding a role and managing entry system under study and the research project, which into the context is not reduced by information exchange between the system under study and the researcher; rather this This involves matters of access and permission, estab- increases the complexity of the situation and, hence, lishing a reason for being there, developing a role and a ‘immune reactions’. persona, identifying the ‘gatekeepers’ who facilitate entry and access to the group being investigated (see 3 Rather than striving for mutual understanding at the LeCompte and Preissle, 1993, pp. 100, 111). This is point of entry, it is more advisable to recognize complex, as the researcher will be both a member of agreement as a process. the group and yet studying that group, so it is a deli- cate matter to negotiate a role that will enable the inves- 4 Whilst it is necessary to agree storage rights for tigator to be both participant and observer. The most data, this may contribute to increasing the complex- important elements in securing access are the willingness ity of the agreement to be reached. of researchers to be flexible and their sensitivity to nuances of behaviour and response in the participants 5 The field under study only becomes clear when one (p. 112). has entered it. De Laine (2000, p. 41) remarks that an ability to get on with people in the situation in question, and a will- 6 The research project usually has nothing to offer the ingness to join in with, and share experiences in, the social system; hence no great promises for benefit or activities in question, are important criteria for gaining services can be made by the researcher, yet there and maintaining access and entry into the field. Barley may be no real reason why the social system should and Bath (2014) note that this is a particular challenge reject the researcher. when conducting research with young children, and they suggest that a period of ‘familiarisation’ is impor- As Flick (1998, p. 57) remarks, the research will disturb tant before the research ‘officially’ commences, partic- the system and disrupt routines without being able to ularly as so much advice is given to children about offer any real benefit for the institution. ‘stranger-danger’ (p. 184). Such familiarization can The issue of managing relations is critical for the help the researcher to understand the norms, rules and qualitative researcher. We discuss issues of access, rituals of the field location, developing early mutual gatekeepers and informants in Chapter 12. The relationships of trust, establishing positionality (dis- researcher is seen as coming ‘without history’ (Wolff, cussed earlier), unobtrusively collecting data, ‘mapping 2004, p. 198), a ‘professional stranger’ (Flick, 1998, the setting’ (p. 185) and preparing for informed consent p. 59), one who has to be accepted, become familiar or assent. and yet remain distant from those being studied. Indeed Flick (p. 60) suggests four roles of the researcher: stranger, visitor, insider and initiate. The first two essentially maintain the outsider role, whilst the latter two attempt to reach into the institution from an insid- er’s perspective. These latter two become difficult to 310
Qualitative, naturalistic and ethnographic research manage if one is dealing with sensitive issues (see when researchers have to conceal information, take on Chapter 13). This typology resonates with the four roles different roles in order to gain access, retain neutrality, typically cited for observers: compromise personal beliefs and values, and handle situations where they are seeking information from OUTSIDER INSIDER others but not divulging information about themselves. Walford suggests that researchers may have little Detached as Observer Observer as Participant Complete opportunity to negotiate roles and manoeuvre roles, as they are restricted by the expectations of those being observer participant participant researched. A related issue is the timing of the point of entry, so Swain (2006), discussing ethnography, suggests that that researchers can commence the research at an researchers may have to switch roles, from being com- appropriate time (e.g. before the start of a programme, pletely passive observers to being completely active at the start of a programme, during a programme, at the participants, as the situation demands, i.e. to draw on end of a programme, after the end of a programme). the complete continuum of observations and roles. Par- Further, the ethnographer seeks acceptance into the ticipant observation is not without its debates. Mills group, which engages matters of dress, demeanour, and Morton (2013, pp. 52–3) note that, whilst some persona, age, colour, religion, ethnicity, empathy and researchers advocate participant observation as ena- identification with the group, language, accent, argot and bling the researcher to get inside the workings of the jargon, willingness to become involved and to take on the institution and its members, others are more hesitant group’s values and behaviour etc. (see Patrick’s (1973) about whether a researcher should be a participant, as study of a Glasgow gang). The researcher, then, must be this might threaten the objectivity of the researcher and, sensitive to the significance of ‘impression management’ anyway, being a participant takes valuable time away (Hammersley and Atkinson, 1983, pp. 78ff.). In covert from the research work of the researcher. research these factors take on added significance, as one Role negotiation, balance and trust are significant slip could ‘blow one’s cover’ (Patrick, 1973). and difficult. For example, if one were to research a Lofland (1971) suggests that the field researcher school, what role should one adopt: a teacher, a should attempt to adopt the role of the ‘acceptable researcher, an inspector, a friend, a manager, a provider incompetent’, balancing intrusion with knowing when of a particular service (e.g. extra-curricular activities), a to remain apart. Such balancing is an ongoing process. counsellor, a social worker, a resource provider, a Hammersley and Atkinson (1983, pp. 97–9) suggest librarian, a cleaner, a server in the school shop or that researchers also have to handle the management of canteen, and so on? One has to try to select a role that ‘marginality’: they are in the organization but not of it. will provide access to as wide a range of people as pos- They comment that ‘the ethnographer must be intellec- sible, preserve neutrality (not being seen as on any- tually poised between “familiarity” and “strangeness”, body’s side) and enable confidences to be secured. while socially he or she is poised between “stranger” Role conflict, role strain and role ambiguity are to and “friend” ’, and that this management of several be expected in qualitative research. For example, De roles, not least the management of marginality, can Laine (2000, p. 29) comments on the potential conflicts engender ‘a continual sense of insecurity’ (p. 100). between the researcher qua researcher, therapist and Gaining access and entry is a process that unfolds friend; she indicates that diverse roles are rarely possi- over time rather than a once-a nd-for-all matter ble to plan in advance, and are an inevitable part of (Walford, 2001, p. 31), as setbacks, delays and modifi- fieldwork, giving rise to ethical and moral problems for cations can occur and have to be expected in gaining the researcher, and, in turn, require ongoing negotiation entry to qualitative research sites. and resolution. Roles change over time. Walford (2001, p. 62) Stage 6: finding informants reports a staged process wherein the researcher’s role moved through five phases: newcomer, provisional This involves identifying those people who have the acceptance, categorical acceptance, personal accept- knowledge about the group, issue or institution being ance and imminent migrant. He also reports (p. 71) that studied. This places the researcher in a difficult posi- it is almost to be expected that managing different roles tion, for she has to be able to evaluate key informants, not only throws the researcher into questioning his/her to decide: ability to handle the situation, but brings considerable emotional and psychological stress, anxiety and feel- OO whose accounts are more important than others; ings of inadequacy. This is thrown into sharp relief OO which informants are competent to pass comments; 311
Methodologies for educational research OO which are reliable; OO handling people and issues with which the OO what the statuses of the informants are; researcher disagrees or finds objectionable or OO how representative are the key informants (of the repulsive; range of people, of issues, of situations, of views, of OO being attentive and empathizing; status, of roles, of the group); OO being discreet; OO how to see the informants in different settings; OO deciding how long to stay. Spindler and Spindler OO how knowledgeable informants actually are – do they have intimate and expert understanding of the (1992, p. 65) suggest that ethnographic validity is situation; attained by having the researcher in situ long enough OO how central to the organization or situation the to see things happening repeatedly rather than just informant is (e.g. marginal or central); once, that is to say, observing regularities. OO how to meet and select informants; OO how critical the informants are as gatekeepers to LeCompte and Preissle (1993, p. 89) suggest that field- other informants, opening up or restricting entry to work, particularly because it is conducted face-to-face, people; raises challenges and questions that are less significant OO the relationship between the informant and others in in research that is conducted at a distance, for example: the group or situation being studied. (a) how to communicate meaningfully with partici- pants; (b) how they and the researcher might be Selecting informants and engaging with them is chal- affected by the emotions evoked in one another, and lenging; LeCompte and Preissle (1993, p. 95), for how to handle these; (c) differences and similarities example, suggest that the first informants that an eth- between the researcher and the participants (e.g. per- nographer meets might be self-selected people who are sonal characteristics, power, resources), and how these marginal to the group, who have a low status and who, might affect relationships between parties and the therefore, might be seeking to enhance their own pres- course of the research; (d) the researcher’s responsibili- tige by being involved with the research. Lincoln and ties to the participants (qua researcher and member of Guba (1985, p. 252) argue that the researcher must be their community), even if the period of residence in the careful to use informants rather than informers, the community is short; (e) how to balance responsibilities latter possibly having ‘an axe to grind’. Researchers to the community with responsibilities to other inter- who are working with gatekeepers, they argue, will be ested parties. engaged in a constant process of bargaining and negotiation. Rapport A ‘good’ informant, Morse (1994, p. 228) declares, is one who has the necessary knowledge, information Critically important in this area is the maintenance of and experience of the issue being researched, is capable trust and rapport (De Laine, 2000, p. 41), showing of reflecting on that knowledge and experience, has interest, assuring confidentiality (where appropriate) time to be involved in the project, is willing to be and avoiding being judgemental. De Laine adds to involved in the project and, indeed, can provide access these (p. 97) the ability to tolerate ambiguity, to keep to other informants. An informant who fulfils all of self-d oubt in check, to withstand insecurity and to be these criteria he termed a ‘primary informant’. Morse flexible and accommodating. Such features cannot be also cautions that not all these features may be present encapsulated in formal agreements, but they are the in the informants, but that they may still be useful for lifeblood of effective qualitative enquiry. They are the research, though the researcher would have to process matters. decide how much time to spend with these ‘secondary’ Qualitative research recognizes that relationships informants (those who meet some but not all of the emerge over time, they are not a one-o ff affair or in selection criteria). which access is negotiated and achieved on a once-a nd- for-all basis; rather, relationships, trust, intimacy, reci- Stage 7: developing and maintaining procity, intrusion, consideration and access have to be relations in the field constantly negotiated, renegotiated and agreed as time, relationships and events move on, as in real life (De This involves addressing interpersonal and practical Laine, 2000, pp. 83–5). In this context Maxwell (2005, issues, for example: p. 83) suggests that ‘rapport’ is problematic in discussing relationships, as it is not a unitary concept concerning its OO building participants’ confidence in the researcher; amount or degree (indeed one may have too much or too OO developing rapport, trust, sensitivity and discretion; little of it) (Seidman, 1998, pp. 80–2), but its nature and kind changes over time, as people and events evolve. 312
Qualitative, naturalistic and ethnographic research Rapport and relationships influence data collection, to be made – similarities and differences (e.g. Miles’s sampling and research design (Maxwell, 2005, p. 83). and Huberman’s (1984) cross-site analysis of several Indeed, in longitudinal qualitative research, Thomson schools). and Holland (2003, p. 235) report that maintaining and Less structured approaches to qualitative research sustaining positive relationships over time can contrib- enable specific, unique and idiographic accounts to be ute significantly to lower attrition rates of participants given, in which the research is highly sensitive to the and researchers (and attrition is a problem in longitudi- specific situation, the specific participants, the relation- nal research as people move out of the area, leave as ships between the researcher and the participants they grow older, lose contact, become too busy and so (Maxwell, 2005, p. 82), and the emergent, most suita- on; p. 241). Similarly, Gordon and Lahelma (2003, ble ways of conducting the data analysis. p. 246), researching the transition of participants from For data collection the researcher can use field being secondary school students into becoming adults, notes, participant observation, journal notes, interviews, comment that maintaining rapport is a critical factor in diaries, life histories, artefacts, documents, video longitudinal ethnographic research. Rapport, they aver recordings, audio recordings etc. Several of these are (p. 248), is signified in attention to non-v erbal commu- discussed elsewhere in this book. Lincoln and Guba nication as well as in the sensitive handling of verbal (1985, p. 199) distinguish between ‘obtrusive’ methods communication. (e.g. interviews, observation, non-v erbal language) and Rapport is not easy to maintain: for example, Bettez ‘unobtrusive’ methods (e.g. documents and records), on (2015) records the dilemma when maintaining rapport the basis of whether another human typically is present with one participant might negatively affect rapport at the point of data collection. with another or with readers, and another situation Field notes can be written both in situ and away where a participant in a powerful, oppressive position from the situation. They contain the results of observa- may not want to be reported as such, or where a family tions, analysis, researchers’ comments and self-memos may not wish to be portrayed in a particular way as it (cf. Mills and Morton, 2013, chapter 4). The nature of would affect their standing in the community, i.e. observation in ethnographic research is discussed fully where the researchers and the participants do not agree in Chapter 26 of the present volume. Accompanying about the reporting. observation techniques are interviews, documentary Rapport is often overlaid with power relations. For analysis and life histories (discussed in Chapters 25 and example, Swain (2006, p. 205) comments that, as an 16). A popularly used interview technique employed in adult conducting an ethnography with junior school qualitative research is the semi-s tructured interview, children, he felt obliged, at times, to take the ‘adult’, where an interview schedule (list of items, questions, controlling position in the research, and that he could prompts and probes) is prepared that is sufficiently not act as a young child, indeed that the children would open-e nded to enable the contents to be re-o rdered, find it odd if he did (p. 207). He was not a child – he digressions and expansions made, new avenues to be was older, taller, had a deeper voice and dressed differ- included and further probing to be undertaken. ently, but he gave the children freedom to respond to Carspecken (1996, pp. 159–60) describes how such his questions as they wished. That said, he commented interviews can range from the interrogator giving bland that he tried to adopt a role that made it clear to the encouragements, ‘non-leading’ leads, active listening children that he was not a teacher. and low-inference paraphrasing to medium- and high- The issue here is that the data-collection process is inference paraphrasing. In interviews, the researcher itself socially situated; it is neither a clean, antiseptic might wish to further explore some matters arising activity nor always a straightforward negotiation. from observations. In naturalistic research, validity in interviews include honesty, depth of response, richness Stage 8: data collection in situ of response and commitment of the interviewee (Oppenheim, 1992). The qualitative researcher can use a variety of tech- Lincoln and Guba (1985, pp. 268–70) propose niques for gathering information. There is no single several purposes for interviewing, including: present prescription for which data-collection instruments to constructions of events, feelings, persons, organiza- use; rather the issue here is of ‘fitness for purpose’ tions, activities, motivations, concerns, claims, etc.; because, as mentioned earlier, the ethnographer is a reconstructions of past experiences; projections into methodological omnivore. Some qualitative research the future; verifying, amending and extending data. can be highly structured, with the structure being deter- Silverman (1993, pp. 92–3) adds that interviews in mined in advance of the research (pre‑ordinate qualitative research are useful for: (a) gathering facts; research), for example in order to enable comparisons 313
Methodologies for educational research (b) accessing beliefs about facts; (c) identifying feel- In addition to interviews, Lincoln and Guba (1985) ings and motives; (d) commenting on the standards of discuss data collection from non‑human sources, actions (what could be done about situations); (e) including: exploring present or previous behaviour; (f ) eliciting reasons and explanations. OO documents and records (e.g. archival records, private Lincoln and Guba (1985) emphasize that the plan- records). These have the attraction of being always ning of the conduct of the interview is important, available, often at low cost, and being factual. On including the background preparation, the opening of the other hand, they may be unrepresentative or the interview, its pacing and timing, keeping the con- selective, they may lack objectivity, may be of versation going and eliciting knowledge, and rounding unknown validity and may possibly be deliberately off and ending the interview. It is important for careful deceptive (see Finnegan, 1996; see also Chapter 16); consideration to be given to the several stages of the interview. For example, at the planning stage, attention OO unobtrusive informational residues. These include will need to be given to the number of interviews per artefacts, physical traces and a variety of other interviewer, duration, timing, frequency, setting/location, records. Whilst they frequently have face validity, number of people in a single interview situation (e.g. and whilst they may be simple and direct, gained by individual or group interviews) and respondent styles non-interventional means (hence reducing the prob- (LeCompte and Preissle, 1993, p. 177). At the imple- lems of reactivity), they may also be very heavily mentation stage the conduct of the interview will be inferential, difficult to interpret and may contain ele- important, for example, responding to interviewees, ments whose relevance is questionable. prompting, probing, supporting, empathizing, clarify- ing, crystallizing, exemplifying, summarizing, avoiding Qualitative data collection is not hidebound to a few censure, accepting. At the analysis stage there are named strategies; it is marked by eclecticism and fitness several considerations, for example: the ease and clarity for purpose. It is not to say that ‘anything goes’ but that of communication of meaning; the interest levels of the ‘use what is appropriate’ is sound advice. Mason (2002, participants; the clarity of the question and the pp. 33–4) advocates integrating methods, for several response; the precision (and communication of this) of reasons: the interviewer; how the interviewer handles question able responses (e.g. fabrications, untruths, claims OO to explore different elements or parts of a phenome- made). non, ensuring that the researcher knows how they The qualitative interview tends to move away from interrelate; a pre-s tructured, standardized format and towards an open-e nded or semi-structured format (see Chapter 25), OO to answer different research questions; which enables respondents to project their own ways of OO to answer the same research question but in differ- defining the world. It permits flexibility rather than fixity of sequence of discussions, allowing participants ent ways and from different perspectives; to raise and pursue issues and matters that might not OO to give greater or lesser depth and breadth to have been included in a pre-devised schedule (Denzin, 1970; Silverman, 1993). analysis; The use of interviews is not automatic for qualita- OO to triangulate – corroborate – by seeking different tive research. Some participants may find it alien to their culture; they may feel uncomfortable with inter- data about the same phenomenon. views, or indeed with any such formal verbal commu- nication (Maxwell, 2005, p. 93). The qualitative She argues that integration can take many forms, and researcher has to find a culturally appropriate and cul- she suggests that researchers should consider whether turally sensitive way of gathering data. Maxwell the data are to complement each other, to be combined, (echoing Whyte, 1993, p. 303, discussed in Chapters 12 grouped and aggregated, and to contribute to an overall and 13) cites sensitive research (heroin users) which picture. She also argues that it is important for the data indicates that it is unwise or inappropriate to ask too to complement each other ontologically, to be ontologi- many questions, and that conducting formal interviews cally consistent (p. 35). Added to this, integration must is an alienating activity, better replaced by informal be in an epistemological sense, i.e. where the data conversations and field notes. emanate from the same, or at least complementary, epistemologies, and whether they are based on ‘similar, complementary or comparable assumptions’ (p. 36) about what researchers can legitimately constitute as evidential knowledge. Finally, she argues that integra- tion must occur at the level of explanation. By this she means that the data from different sources and methods 314
Qualitative, naturalistic and ethnographic research must be able to be combined into a coherent, convinc- the data-collection process. There are several reasons ing and relevant explanation and argument (p. 36). for this, discussed below. Data collection also relates to sampling. For At a practical level, qualitative research rapidly example, in qualitative or ethnographic interviews, amasses huge amounts of data, and early analysis though the researcher may wish to include a range of reduces the problem of data overload by selecting sig- participants, in fact some of those participants may be nificant features for future focus. Miles and Huberman shy, inarticulate, marginalized, dominated, introverted, (1984) suggest that careful data display is an important overwhelmed or fearful in the presence of others or of element of data reduction and selection. ‘Progressive being censured, or uninterested in participating (Swain, focussing’, according to Parlett and Hamilton (1976), 2006, p. 202). In these circumstances the researcher starts with the researcher taking a wide-a ngle lens to may have to use alternative methods of gathering data, gather data, and then, by sifting, sorting, reviewing and such as observation. Miller and Dingwall (1997) point reflecting on them, the salient features of the situation out that an interview may be very unsettling for some emerge. These are then used as the agenda for subse- participants, being too formal or unnatural; it is not the quent focusing. The process is like funnelling from the same as a conversation, and some participants may not wide to the narrow. ‘open up’ in a non-conversational situation. We discuss Maxwell (2005, p. 95) argues for data analysis not interviews and interviewing in Chapter 25. only to be built into the design of qualitative research, but to start as soon as each stage or round of data col- Stage 9: data collection outside the field lection happens, or as soon as any data have been col- lected, i.e. without waiting for the next stage, round or In order to make comparisons and to suggest explana- piece of data to have taken place. He cites the analogy tions for phenomena, researchers might find it useful to of the fox having to keep close to the hare: keeping the go beyond the confines of the groups in which they collection and the analysis close together ensures that occur. That this is a thorny issue is indicated in the fol- the researchers can keep close to changes and their lowing example. Two students are arguing violently effects. He suggests that data analysis commences with and physically in a school. At one level it is simply a careful reading and re-reading of the data, then con- fight between two people. However, this is a common structing memos, categorizations (e.g. coding into occurrence between these two students as they are organizational, substantive – descriptive – and theoreti- neighbours outside school and they don’t enjoy posi- cal categories (e.g. related to prior theory, ‘etic’ catego- tive, amicable relations as their families are frequently ries, grounded theory)) and thematic analysis, and feuding. The two households have been placed next ‘connecting strategies’ such as narrative analysis (p. 96) door to each other by the local authority because it has and vignettes, discourse analysis and profiles (p. 98) taken a decision to keep together families who are very that set the data in context and indicate relationships poor at paying for local housing rent (i.e. a ‘sink’ between different parts of the data such that the integ- estate). The local authority has taken this decision rity – the wholeness – of the original context is pre- because of a government policy to keep together disad- served (p. 98), rather than the fracturing and regrouping vantaged groups so that targeted action and interven- of the data that can occur in a coding exercise. tions can be more effective, thus meeting the needs of Analytical memos, including striking observations whole communities as well as individuals. and comments, enable researchers to make connections The issue here is: how far out of (or indeed inside) a between observations, analysis and literature (Mills and micro-s ituation does the researcher need to go to under- Morton, 2013, p. 122). They act as a record, a reminder, stand that micro-situation (Morrison, 2009, p. 7), for a focus, a conjecture, a tentative explanation and a sug- example, the individual, familial, neighbourhood, local gestion for future steps to take in the research. government or national government level? At a theoretical level, a major feature of qualitative research is that analysis commences early on in the Stage 10: data analysis data-c ollection process so that theory generation can happen (LeCompte and Preissle, 1993, p. 238). Though we devote six chapters specifically to qualita- LeCompte and Preissle (1993, pp. 237–53) advise tive data analysis later in this book (Part 5), there are researchers to: (a) set out the main outlines of the phe- some preliminary remarks that we make here. Data nomena that are under investigation; then (b) assemble analysis involves organizing, accounting for and chunks or groups of data, putting them together to explaining the data; in short, making sense of data in make a coherent whole (e.g. through writing summaries terms of participants’ definitions of the situation, noting of what has been found); then (c) painstakingly take patterns, themes, categories and regularities. Typically in qualitative research, data analysis commences during 315
Methodologies for educational research apart their field notes, matching, contrasting, aggregat- it may not be possible to include everyone’s voice, ing, comparing and ordering notes made. The intention even though the canons of validity in qualitative is to move from description to explanation and theory research might call for multiple voices to be heard. generation. Eisenhart (2001, p. 19) points out that researchers all Thomson and Holland (2003, p. 236) suggest that, too easily can privilege some voices at the expense of in longitudinal qualitative research, data analysis others and that the express, beneficent intention of pro- should be both cross-sectional (in order to discover the tecting some participants can have the effect of silenc- discourses and themes at work in the construction of ing them. How will the researcher present different, identities and interpretations at a particular point in even conflicting voices, accounts or interpretations? time) and longitudinal (in order to chart the develop- What are the politics surrounding inclusion and exclu- ment of narrative(s) over time). However, they also sion of voices? We return to this issue in Part 5 on recognize that cross-s ectional approaches and longitu- qualitative data analysis. dinal approaches may sit together uncomfortably, as the For clarity, the process of data analysis can be por- former chops up and reassembles text from different trayed in a sequence of seven steps which are set out participants in order to present themes at one moment here and addressed in subsequent pages (Figure 15.4). in time, whilst the latter seeks individual narratives that require the continuity that only emerges over time and Step 1: establish units of analysis of the data, within individuals (p. 239). indicating how these units are similar to and Longitudinal research that uses ethnographic tech- different from each other niques (e.g. life histories) can also be used to chart tran- The criterion here is that each unit of analysis (category sitions in participants, for example, from primary to – conceptual, actual, classification element, cluster, secondary school, from secondary school to university, issue) should be as discrete as possible whilst retain- from school to work, from childhood to adulthood etc. ing fidelity to the integrity of the whole, i.e. that Gordon and Lahelma (2003) comment that in such each unit must be a fair rather than a distorted represen- research, the reflexivity of the participants can increase tation of the context and other data. The creation over time, and that sensitivity and rapport (discussed of units of analysis can be done by ascribing codes earlier) are key elements for success. Indeed the authors to the data (Miles and Huberman, 1984). This is akin go further, to argue that as the research develops over to the process of ‘unitizing’ (Lincoln and Guba, time, so does the obligation to demonstrate reciprocity 1985, p. 203). in the relationships between researcher(s) and partici- pants, so that, just as the participants give information, Step 2: create a ‘domain analysis’ so the researcher has an ethical obligation to ensure that A domain analysis involves grouping together items the research offers something positive, in return, to the and units into related clusters, themes and patterns, a participants. This need not necessarily mean a material domain being a category which contains several other incentive or reward; it could mean an opportunity for categories. the participants to reflect on their own situation, to learn more about themselves and to support their devel- Step 3: establish relationships and linkages opment (p. 249). In this case reflexivity is not confined between the domains to the researcher, but extends to the participants as This process ensures that the data, their richness and well (p. 252). ‘context-g roundedness’ are retained. Linkages can be We discuss cross-s ectional and longitudinal studies found by identifying confirming cases, by seeking (surveys) in Chapter 17. ‘underlying associations’ (LeCompte and Preissle, Thomson and Holland (2003) indicate the frustra- 1993, p. 246) and connections between data subsets. tion and intimidation that early analysis in longitudinal This helps to establish core themes, i.e. those themes research can cause for researchers, as there is never which seem to underpin or to have reference made to complete closure on data analysis, as ‘the next round of them most frequently or most significantly in the data, data’ can challenge earlier interpretations made by or which explain a lot (Gonzales et al., 2008, pp. 5–6). researchers. Indeed they question when the right time is to commence writing up or make interpretations. Step 4: make speculative inferences In addition to the challenge of continual openness to This stage moves the research from description to infer- interpretation as qualitative research unfolds is the ence. It requires the researcher, on the basis of the evi- related issue of whose views/voices one includes in the dence, to posit some explanations for the situation, data analysis, given that, in the interests of practicality, some key elements and possibly even their causes. It is 316
Qualitative, naturalistic and ethnographic research Step 1 Establish units of analysis of the data, indicating how these units are similar to and different from each other Step 2 Create a ‘domain analysis’ Step 3 Establish relationships and linkages between the domains Step 4 Make speculative inferences Step 5 Summarize Step 6 Seek negative and discrepant cases Step 7 Generate theory FIGURE 15.4 Seven steps in qualitative data analysis the process of hypothesis generation or the setting of LeCompte and Preissle (1993, pp. 250–1) define a neg- working hypotheses that feeds into theory generation. ative case as an exemplar which disconfirms or refutes the working hypothesis, rule or explanation so far. The Step 5: summarize theory that is being developed becomes more robust if Here the researcher writes a preliminary summary of it addresses and can embrace or explain negative cases, the main features, key issues and key concepts, con- for it sets the boundaries to the theory, modifies the structs and ideas encountered so far in the research. We theory and sets parameters to the applicability of the address summarizing in more detail in Chapter 33. theory. Discrepant cases are not so much exceptions to the Step 6: seek negative and discrepant cases rule (as in negative cases) as variants of the rule In theory generation it is important to seek not only (LeCompte and Preissle, 1993, p. 251). The discrepant confirming cases but to weigh the significance of dis- case leads to the modification or elaboration of the con- confirming cases. LeCompte and Preissle (1993, p. 270) struct, rule or emerging hypothesis. Discrepant case suggest that because interpretations of the data are analysis requires the researcher to seek out cases grounded in the data themselves, results that fail to for which the rule, construct or explanation cannot support an original hypothesis are neither discarded account or with which they will not fit, i.e. they are nor discredited; rather, it is the hypotheses themselves neither exceptions nor contradictions, they are simply that must be modified to accommodate these data. different! 317
Methodologies for educational research Step 7: generate theory as it disturbs the natural setting, even though its intention might be in the interests of serving the Here the theory derives from the data; it is grounded in ethical issue of ‘beneficence’; see Chapter 7); the data and emerges from it (see Chapter 37). As OO searching for discrepant evidence and negative Lincoln and Guba (1985, p. 205) argue, grounded cases, in order to constitute a strong test of the theory must fit the situation that is being researched. theory or conclusions drawn; Grounded theory is an iterative process, moving back- OO triangulation, in order to give reliability to the find- wards and forwards between data and theory until the ings and data (see Chapter 14); theory fits the data. This breaks the linearity of much OO quasi-statistics, where quasi-q uantitative statements conventional research (Flick, 1998, pp. 41, 43) in which are interrogated, for example, claims that a finding hypotheses are formulated, sampling is decided, data is rare, extreme, unusual, typical, frequent, domi- are collected and then analysed and hypotheses are sup- nant, prevalent and so on; ported or not supported. In grounded theory, a circular OO comparison, between groups, sub-groups, sites and and recursive process is adopted, wherein modifications settings, events and activities, times, contexts, are made to the theory in light of data, more data are behaviours and actions etc., to look for consistency sought to investigate emergent issues (theoretical sam- or inconsistency, similarity or difference across pling), and hypotheses and theories emerge from these. the data. Lincoln and Guba (1985, pp. 354–5) urge the Swain (2006, p. 202) comments that, in writing up an researcher to be mindful of several issues in analysing ethnography or qualitative research, the researcher must and interpreting the data, including: (a) data overload; exercise discipline, in that a faithful account has to be (b) the problem of acting on first impressions only; written, yet, for manageability, the level of detail on the (c) the availability of people and information (e.g. how context, emerging situation and events has to be representative these are and how to know if missing reduced. Indeed he argues that less than 1 per cent of people and data might be important); (d) the dangers of the collected data may feature in the final report, and seeking only confirming rather than disconfirming that, even if all the data that were collected were instances; (e) the reliability and consistency of the data included, these would constitute less than 1 per cent of and confidence that can be placed in the results. everything that took place or that was experienced by Maxwell (2005, p. 108) draws attention to some the researcher. Fidelity to the detail may stand in a rela- important issues of validity for the qualitative data tion of tension to the final, necessarily selective, use of analyst, including researcher bias and reactivity. The data, and care has to be given to issues of reliability and former concerns the projection of the researcher’s own validity in such a situation. values and judgements onto the situation, whilst the These are significant issues in addressing reliability, latter concerns the effect of the research(er) on the par- trustworthiness and validity in the research (see Chapter ticipants, giving rise to unreliable behaviours or changes 14). Further, the essence of this approach, that theory to the natural setting (a particular problem, for example, emerges from and is grounded in data, is not without its in interviewing or observing children). Maxwell sets out critics. For example, Silverman (1993, p. 47) suggests a useful checklist of ways in which attention can be that it fails to acknowledge the implicit theories which given to validity in qualitative research: guide research in its early stages (i.e. data are not theory-n eutral but theory saturated) and that the theory OO intensive, long-term involvement, enabling the might be strong on providing categorizations without researcher to probe beneath immediate behaviours, necessarily explanatory potential. These caveats should for reducing reactivity and for revealing causal feed into the process of reflexivity in qualitative processes; research. Maxwell (2005, pp. 115–16) also indicates that the OO ‘rich’ data, sufficient to provide a sufficiently, process of data analysis, and the conclusions drawn revealing, varied and full picture of the phenome- from the data, should address generalizability, i.e. to non, participants and settings; whom the results are generalizable. Internal generaliza- bility will indicate that the results and conclusions are OO respondent validation, to solicit feedback from par- generalizable to the group in question, whilst external ticipants on the interpretations made of, and conclu- generalizability will indicate that the results and con- sions from, the data; clusions are generalizable to the wider population beyond the group under study. He suggests that, whilst OO intervention, where the researcher intervenes for- mally or informally, in a small or a large way, in the natural setting in order to contribute positively to a situation (whether this is legitimate is a moot point, 318
Qualitative, naturalistic and ethnographic research the former may be applicable to qualitative research, OO the writer should make clear the data that give rise the latter often may not. However, he also indicates that to the report, so the readers have a means of check- this by no means rules out the external generalizability ing back for reliability and validity and inferences; of qualitative studies, as respondents themselves might have commented on the generalizability of their situa- OO a fixed completion date should be specified. tion, or the researcher or readers might see similarities to other, comparable situations, constraints or dynam- Spradley (1979) suggests a sequence of nine practical ics, or the research might be corroborated by, or cor- steps in writing an ethnography: roborate, other studies. He indicates, however, that external generalizability is not a strong feature, indeed 1 Select the audience. a concern, of qualitative research. 2 Select the thesis. 3 Make a list of topics and create an outline of the Stage 11: leaving the field ethnography. The issue here is how to conclude the research, how to 4 Write a rough draft of each section of the terminate the roles adopted, how (and whether) to bring to an end the relationships that have built up over the ethnography. course of the research, and how to disengage from the 5 Revise the outline and create subheadings. field in ways that bring as little disruption to the group 6 Edit the draft. or situation as possible (LeCompte and Preissle, 1993, 7 Write an introduction and a conclusion. p. 101). De Laine (2000, p. 142) remarks that some par- 8 Re-read the data and report to identify examples. ticipants may want to maintain contact after the 9 Write the final version. research is over, and not to do this might create, for them, a sense of disappointment, exploitation or even Clearly there are several other aspects of case study betrayal. reporting that need to be addressed. These are set out in The researcher has to consider the after-e ffects of Chapter 19. leaving and take care to ensure that nobody comes to The writing of a qualitative report can also consider harm or is worse off from the research, even if it is the issue of the generalizability of the research. Whilst impossible to ensure that they have benefited from it. much qualitative research strives to embrace the uniqueness and individual idiographic features of the Stage 12: writing the report phenomenon and/or participants, rendering generaliza- tion irrelevant (though the study would still need to Often the main vehicle for writing naturalistic research ensure that it contributes something that is worthwhile is the case study (see Chapter 19), whose ‘trustworthi- and significant for the research community), this need ness’ (Lincoln and Guba, 1985, p. 189) is defined in not preclude attention to generalization where applica- terms of credibility, transferability, dependability and ble in qualitative research. Indeed one can question the confirmability (see Chapter 14). Case studies are useful value or contribution of idiographic research that does in that they can provide the thick descriptions that not have any generalizable function or utility (Wolcott, typify ethnographic research, and can catch and portray 1994, p. 113). to the reader what it is like to be involved in the situ Generalization takes many forms; it is not a unitary ation (p. 214). As Lincoln and Guba comment (p. 359), or singular concept, and it connotes far more than the the case study is the ideal instrument for ‘emic’ inquiry. familiar terms ‘transferability’ (Denzin and Lincoln, They provide several guidelines for writing case studies 1994) or ‘external validity’ (Cook and Campbell, 1979). (pp. 65–6): Larsson (2009, p. 27) comments that defining generali- zation as that which is derived by strict sampling from a OO the writing should strive to be informal and to defined population is often irrelevant in qualitative capture informality; research. He also suggests that those single studies that seek to undermine ‘universal’ truths similarly do not OO as far as possible, the writing should report facts need to aspire to be generalizable, as the single instance except in those sections where interpretation, evalu- of falsification (‘negative cases’; p. 30) may be suffi- ation and inference are made explicit; cient to bring down the theory (though the case would need to be made that the ‘truths’ claimed to be universal OO in drafting the report it is more advisable to opt for in the first place as social actions may not be susceptible over-inclusion rather than under-inclusion; to universal laws of behaviour). However, he suggests three kinds of reasoning on which generalization in OO the ethical conventions of report writing must be qualitative research might be useful: honoured, for example, anonymity, non-traceability; 319
Methodologies for educational research 1 Enhancing the potential for generalization by maxi- Whether a pattern is indeed a pattern, or whether a mizing the range of a sample’s characteristics in construction is an acceptable construction, is a exploring a particular issue (e.g. in theoretical sam- matter of debate and interpretation. Researchers pling) or phenomenon, i.e. to ensure that as many have to be sure that the patterns between both different cases or categories of an issue as possible research and the wider context are, indeed, tenable. are included in the research. Here uncommon cases Interpretation is an inescapable feature of qualitative have as equal a weight as the typical cases, and the research, and it is this precise matter that renders variation that exists within the study should be difficult the applicability of research from one expected to exist in the wider population, context or context to another, because it is not the context but situation to which one wishes to generalize (p. 31). the interpretation of the context that has to be This, in turn, may require a larger sample than may similar to that to which it is being applied. Further, be normal in qualitative research, in order to have as one is faced with the added problem of identifying broad a variation and range of characteristics as pos- whose interpretation should stand (not only the issue sible included, and this may not be possible in some of ‘emic’ and ‘etic’ research, but also whose ‘etic’ qualitative research, for example, case studies. It and ‘emic’ interpretations, given that there will be also assumes that the researcher will know what the multiple variants of each type). maximum variation will look like, so that he or she knows when it is reached, and this, too, may not be Larsson (2009, p. 36) is arguing powerfully that respon- realistic (p. 32). sibility for generalization from qualitative research resides with the audience rather than the researcher. 2 Generalization by ensuring the similarity of contexts However, to suggest this may be to invite the view that between that of the qualitative research and the the researcher has no special expertise to offer here; if wider contexts to which it is wished to be applied so, then how is the research justified? Perhaps the solu- (akin to the ‘transferability’ criterion of Guba and tion to this is to regard the research, as with other kinds Lincoln (1994)). Here Strauss and Corbin (1990, of research, as raising working hypotheses rather than p. 267) argue that generalizability might also be conclusions, i.e. as ‘work in progress’ rather than unas- replaced by ‘explanatory power’ in the context of sailable truths. the research and the wider contexts. This view of Whilst it appears that writing comes late in the stage generalizability assumes that the characteristics of the research, in fact it should be a continuous, ongoing of the wider contexts are known, and this may not activity, from the start to the finish of the research. be for the researcher to judge, but, rather, for the Indeed Mills and Morton (2013) place the ongoing outsider readers, audiences or users of the research writing of an ethnography as a key, central feature of to make such judgements (cf. Wolcott, 1994, doing ethnographic work. Writing on an ongoing basis p. 113). Hence, Larsson (2009, p. 32) argues, the clarifies thoughts, observations, steps to take, reflections, task of the researcher is to provide sufficient details analysis and so on. We strongly advise ethnographers to and ‘thick descriptions’ for the audiences to come to start writing from day one of their research. an informed judgement about generalizability here. A problem is raised in this kind of generalizability, 15.11 Some challenges in in deciding when and on what – and how many – qualitative, ethnographic and criteria the contexts of the research and the wider naturalistic approaches contexts are similar and when sufficient similarity of contexts has been reached for the research to be gen- There are several challenges in qualitative, ethno- eralizable to those wider contexts (p. 33), as the graphic and natural approaches. These might affect the same people or kinds of people may act differently reliability and validity of the research, and include: in different – or even the same – contexts. 1 The definition of the situation: participants are 3 Generalization by recognizing similar patterns asked for their definition of the situation, yet they between the research and other contexts (Larsson, have no monopoly on wisdom. They may be 2009, pp. 33–5) in terms of, for example: theoretical ‘falsely conscious’ (unaware of the ‘real’ situ constructions; themes; concepts; behaviours; ation), deliberately distorting or falsifying informa- assumptions made; processes; interpretations of tion, or being highly selective. Issues of reliability actions, events or descriptions. Here the issue of and validity here are addressed in Chapter 14 (see interpretation is raised, as interpretations of one the discussions of triangulation). context may be very different from the interpreta- tions made of another – however similar – context. 320
Qualitative, naturalistic and ethnographic research 2 Reactivity – the Hawthorne effect – the presence of contexts and situations might overemphasize dif- the researcher, or the fact that it is ‘research’ can ferences between contexts and situations rather alter the situation as participants may wish to than their gross similarity and routine features. avoid, impress, direct, deny or influence the Researchers should be as aware of regularities as research(er). Again, this is discussed in Chapter 14. of differences. Reactivity can be addressed by careful negotiation 7 The neglect of wider social contexts and constraints. in the field, remaining in the field for a consider Studying situations that emphasize how highly able time and ensuring a careful presentation of the context-b ound they are might neglect broader researcher’s self. currents and contexts – micro-level research risks putting boundaries that exclude important macro- 3 The halo effect – where existing or given informa- level factors. Wider macro-contexts cannot be ruled tion about the situation or participants might be out of individual situations. used in judging subsequent data or people, or may 8 The issue of generalizability. If situations are unique bring about a particular reading of a subsequent and non-generalizable, as many naturalistic princi- situation (the research equivalent of the self- ples would suggest, how is the issue of generaliza- fulfilling prophecy). This is an issue of reliability, bility to be addressed? To which contexts will the and can be addressed by having a wide, triangu- findings apply, and what is the role and nature of lated database and the assistance of an external replication studies (and are they necessary)? observer. The halo effect commonly refers to the 9 How to write up multiple realities and explana- researcher’s belief in the goodness of participants tions? How will a representative view be reached? (the participants have haloes around their heads!), What if the researcher sees things that are not seen such that the more negative aspects of their behav- by the participants? iour or personality are neglected or overlooked. By 10 Who owns the data and the report, and who has contrast, the horns effect refers to the researcher’s control over the release of the data? belief in the badness of the participants (the partici- pants have devil’s horns on their heads!), such that Naturalistic and ethnographic research raises important, the more positive aspects of their behaviour or per- if challenging, questions for research in education. sonality are neglected or overlooked. To interview or not to interview? 4 The implicit conservatism of the interpretive meth- odology. The kind of research described in this Should the qualitative researcher, seeking to research a chapter, with the possible exception of critical eth- natural setting in as undisturbed a way as possible, nography, accepts the perspective of the participants intervene by interviewing, as interviewing is a non‑ and corroborates the status quo. It is focused on the natural activity, a disturbance of the natural setting? On past and the present rather than on the future. the one hand, open-ended interviewing can find out participants’ views on a situation, event, experience or 5 There is the difficulty of focusing on the familiar, phenomenon: it provides ‘witness information’ (Ham- as participants (and, maybe, researchers too) may mersley, 2013, p. 68) and involves participants in the be so close to the situation that they neglect certain, situation. On the other hand, an interview is a contrived often tacit, aspects of it. The task, therefore, is to activity that is not part of the normal run of events for make the familiar strange. Delamont (1981) sug- the participants but, rather, is a non-n ormal activity ini- gests that this can be done by: tiated by the researcher and his/her agenda, i.e. framing and shaping the situation through the researcher’s eyes OO studying unusual examples of the same issue and asking for second-h and information in the sense of (e.g. atypical classrooms, timetabling or organi- asking participants to comment on others’ views in zations of schools); addition to their own. As we mention in Chapter 25, interviews are speech acts in their own right, not simply OO studying examples in other cultures; vehicles for collecting proxy data (cf. Atkinson and OO studying other situations that might have a Delamont, 2006, p. 752). Further, participants and interviews may not be bearing on the situation in hand (e.g. if study- genuine. Participants may withhold information ing schools it might be useful to look at other (deliberately or not), distort the truth, promote their own similar-b ut-different organizations, for instance agenda (e.g. ‘position’ themselves) and overlook the non- hospitals or prisons); v erbal interactions involved in interviews and their OO taking a significant issue and focusing on it deliberately, for example, gendered behaviour. 6 The open-e ndedness and diversity of the situations studied. The drive towards focusing on specific 321
Search
Read the Text Version
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
- 18
- 19
- 20
- 21
- 22
- 23
- 24
- 25
- 26
- 27
- 28
- 29
- 30
- 31
- 32
- 33
- 34
- 35
- 36
- 37
- 38
- 39
- 40
- 41
- 42
- 43
- 44
- 45
- 46
- 47
- 48
- 49
- 50
- 51
- 52
- 53
- 54
- 55
- 56
- 57
- 58
- 59
- 60
- 61
- 62
- 63
- 64
- 65
- 66
- 67
- 68
- 69
- 70
- 71
- 72
- 73
- 74
- 75
- 76
- 77
- 78
- 79
- 80
- 81
- 82
- 83
- 84
- 85
- 86
- 87
- 88
- 89
- 90
- 91
- 92
- 93
- 94
- 95
- 96
- 97
- 98
- 99
- 100
- 101
- 102
- 103
- 104
- 105
- 106
- 107
- 108
- 109
- 110
- 111
- 112
- 113
- 114
- 115
- 116
- 117
- 118
- 119
- 120
- 121
- 122
- 123
- 124
- 125
- 126
- 127
- 128
- 129
- 130
- 131
- 132
- 133
- 134
- 135
- 136
- 137
- 138
- 139
- 140
- 141
- 142
- 143
- 144
- 145
- 146
- 147
- 148
- 149
- 150
- 151
- 152
- 153
- 154
- 155
- 156
- 157
- 158
- 159
- 160
- 161
- 162
- 163
- 164
- 165
- 166
- 167
- 168
- 169
- 170
- 171
- 172
- 173
- 174
- 175
- 176
- 177
- 178
- 179
- 180
- 181
- 182
- 183
- 184
- 185
- 186
- 187
- 188
- 189
- 190
- 191
- 192
- 193
- 194
- 195
- 196
- 197
- 198
- 199
- 200
- 201
- 202
- 203
- 204
- 205
- 206
- 207
- 208
- 209
- 210
- 211
- 212
- 213
- 214
- 215
- 216
- 217
- 218
- 219
- 220
- 221
- 222
- 223
- 224
- 225
- 226
- 227
- 228
- 229
- 230
- 231
- 232
- 233
- 234
- 235
- 236
- 237
- 238
- 239
- 240
- 241
- 242
- 243
- 244
- 245
- 246
- 247
- 248
- 249
- 250
- 251
- 252
- 253
- 254
- 255
- 256
- 257
- 258
- 259
- 260
- 261
- 262
- 263
- 264
- 265
- 266
- 267
- 268
- 269
- 270
- 271
- 272
- 273
- 274
- 275
- 276
- 277
- 278
- 279
- 280
- 281
- 282
- 283
- 284
- 285
- 286
- 287
- 288
- 289
- 290
- 291
- 292
- 293
- 294
- 295
- 296
- 297
- 298
- 299
- 300
- 301
- 302
- 303
- 304
- 305
- 306
- 307
- 308
- 309
- 310
- 311
- 312
- 313
- 314
- 315
- 316
- 317
- 318
- 319
- 320
- 321
- 322
- 323
- 324
- 325
- 326
- 327
- 328
- 329
- 330
- 331
- 332
- 333
- 334
- 335
- 336
- 337
- 338
- 339
- 340
- 341
- 342
- 343
- 344
- 345
- 346
- 347
- 348
- 349
- 350
- 351
- 352
- 353
- 354
- 355
- 356
- 357
- 358
- 359
- 360
- 361
- 362
- 363
- 364
- 365
- 366
- 367
- 368
- 369
- 370
- 371
- 372
- 373
- 374
- 375
- 376
- 377
- 378
- 379
- 380
- 381
- 382
- 383
- 384
- 385
- 386
- 387
- 388
- 389
- 390
- 391
- 392
- 393
- 394
- 395
- 396
- 397
- 398
- 399
- 400
- 401
- 402
- 403
- 404
- 405
- 406
- 407
- 408
- 409
- 410
- 411
- 412
- 413
- 414
- 415
- 416
- 417
- 418
- 419
- 420
- 421
- 422
- 423
- 424
- 425
- 426
- 427
- 428
- 429
- 430
- 431
- 432
- 433
- 434
- 435
- 436
- 437
- 438
- 439
- 440
- 441
- 442
- 443
- 444
- 445
- 446
- 447
- 448
- 449
- 450
- 451
- 452
- 453
- 454
- 455
- 456
- 457
- 458
- 459
- 460
- 461
- 462
- 463
- 464
- 465
- 466
- 467
- 468
- 469
- 470
- 471
- 472
- 473
- 474
- 475
- 476
- 477
- 478
- 479
- 480
- 481
- 482
- 483
- 484
- 485
- 486
- 487
- 488
- 489
- 490
- 491
- 492
- 493
- 494
- 495
- 496
- 497
- 1 - 50
- 51 - 100
- 101 - 150
- 151 - 200
- 201 - 250
- 251 - 300
- 301 - 350
- 351 - 400
- 401 - 450
- 451 - 497
Pages:
