the context of educational research 6 Function: What appears to be something that are only at the early stages of formulation and are thus works poorly in achieving an aim is actually the characterized by unevenness. opposite, and vice versa (pp. 319–20). Empirical theories 7 Evaluation: What appears to be a bad matter or phenomenon is actually the opposite, and vice Many definitions of ‘theory’, which add to the features versa (p. 321). of ‘theory’ set out in the bullet points earlier, include a requirement that locates it in an empirical context, 8 Co-relation: What appear to be unrelated items or relating to observational evidence (widely defined), of phenomena are actually the opposite, i.e. are inter- which scientific theories are prime examples: dependent, and vice versa (p. 322). OO A theory is based on, and guides, empirical research, 9 Coexistence: What appear to be matters or phe- typically in hypothesis testing or testing by making nomena that can coexist actually cannot, and vice systematic, objective predictions and observations; versa (pp. 23–4). OO A theory is testable and, therefore, falsifiable and 10 Co-v ariation: What appear to be positive co- provisional, such as a scientific theory; variations between matters or phenomena actually are negative co-v ariations, and vice versa (p. 324). OO A theory is conjectural, suggesting relations (e.g. correlational or causal) between two or more items 11 Opposition: What appear to be identical or very (however defined, for example, variables, factors, similar matters or phenomena are actually the observations). opposite, and vice versa (p. 325). Kettley (2012) suggests that an empirical theory is a 12 Causation: What appears to be an independent ‘coherent description and explanation of observed phe- causal variable is actually a dependent variable, nomena which provides a testable, verifiable or falsifi and vice versa (p. 326). able, representation of social relationships’ which ‘enables the researcher to speculate about future social Alvesson’s and Sandberg’s (2013) methodology for activity and, perhaps, to predict behaviour drawing on generating ‘interesting’ research requires researchers the inferences of the explanation’ (p. 9). Gorard (2013), to expose and evaluate assumptions (e.g. in the litera- too, suggests that a theory ‘is a tentative explanation’ ture, in ‘theories’), and, from there, to develop and (p. 31). He holds that a reasonable theory is one that evaluate an alternative ground of assumptions (p. 56). offers a reasonable explanation based on the research Some ‘interesting’ theories act as a bridge between evidence, that this is surpassed by a theory which not two or more different theories (Bacharach, 1989, only explains the observations but holds true when p. 511) or identify redundant or incorrect earlier theo- tested further, and, in turn, this is surpassed by a theory ries, i.e. help us to re‑evaluate existing theories. As which not only explains and survives further testing, Kaplan (1964) remarks: a ‘new theory requires its but which has predictive value for something which is own terms and generates its own laws: the concepts completely unexpected (p. 31). are not merely reorganized, but reconstituted, the old Here a theory has a tentative, impermanent and con- laws are not just connected but given a new meaning’ jectural quality; it can be tested, upheld, disproved or (p. 297). modified, and its strength resides in its surviving such How we conduct research is informed by the types ‘severe tests’, i.e. those tests which are deliberately of theory in which we are working or which underpin designed to try to falsify the theory (Popper, 1968, the research, and these are introduced below. 1980). Popper’s (1968) view of a scientific theory takes the form of a universal law applying to a particular type 4.4 Types of theory of phenomenon. Such a law should demonstrate preci- sion and universality, ‘it should set the criteria for its There are several different types of theory, and each own falsification’ (p. 92) and possess explanatory and type of theory defines its own kinds of ‘proof ’: for predictive power. Indeed Popper (1968) comments that example, empirical theory, ‘grand’ theory, normative the ‘best theory’ is that which is testable, survives theory and grounded theory (discussed below), and being tested, has greater explanatory power than com- ‘critical’ theory (introduced below as an instance of peting theories and has the greatest content and sim- normative theory and which has the entire Chapter 3 plicity (p. 419). devoted to it). The status of theory varies quite consid- Popper (1968), Lakatos (1970), Mouly (1978), Laudan erably according to the discipline or area of knowledge (1990) and Rasmussen (1990) identify the following in question. Some theories, as in the natural sciences, are characterized by a high degree of elegance and sophistication; others, perhaps like educational theory, 72
Theory in educational research characteristics of a sound empirical theory (cf. Morrison, theory is best which explains the most in the sim- 1995a). It should: plest way. A theory must explain the data adequately and yet must not be so comprehensive as to be OO be operationalizable precisely; unwieldy. On the other hand, it must not overlook OO be testable, and against evidence which is different variables simply because they are difficult to explain; from that which gave rise to the theory, i.e. moving OO be corrigible in light of further evidence; beyond simply corroboration and induction and OO be a spur to empirical research, spawning research towards ‘testing’, identifying the type of evidence and new ideas (fertility); which is required to confirm or refute the theory; OO lead to new ideas that would otherwise not have OO permit deductions to be made; emerged. OO have explanatory and causal power; OO be compatible with both observation and previously Empirical theories, by their very nature, are provisional. validated theories. It must be grounded in empirical A theory can never be complete in the sense that it data that have been verified and must rest on sound encompasses all that can be known or understood or postulates and hypotheses. The better the theory, the certain about the given phenomenon. As Mouly (1978) more adequately it can explain the phenomena under argues, one (scientific) theory is replaced by a superior, consideration, and the more facts it can incorporate more sophisticated theory, as new knowledge is into a meaningful structure of ever-greater general- acquired (echoing Kuhn’s (1962) discussion of para- izability. There should be internal consistency digms and paradigm shifts). An empirical theory between these facts; gathers together all the isolated bits of empirical data OO clarify the precise terms in which it seeks to explain, into a coherent conceptual framework of wider applica- predict and generalize about empirical phenomena; bility. More than this, however, empirical theory is OO be tentative, conjectural, provisional and falsifiable, itself a potential source of further information and dis- stating the grounds, criteria and circumstances for coveries. In this way it is a source of new hypotheses its own empirical verification, proof, falsification or and hitherto unasked questions; it identifies critical rejection; areas for further investigation; it discloses gaps in our OO demonstrate precision and universality, identifying the knowledge; and enables a researcher to postulate the nature and operation of a ‘severe test’ (Popper, 1968), existence of previously unknown phenomena. permit deductions that can be tested empirically, i.e. it must provide the means for its confirmation or rejec- Grand theory tion. One can test the validity of a theory through the validity of the propositions (hypotheses) that can be Not all theories are testable. Some theories are artefacts derived from it. If repeated attempts to disconfirm its comprising abstract concepts, for example, theories of various hypotheses fail, then greater confidence can be modernism and postmodernism, Freudian theory, placed in its validity. This can go on indefinitely; Talcott Parsons’s theory of social formations and OO clarify its methodologies (e.g. hypothetico- systems, Habermas’s theory of communicative action, deductive, inductive); Bourdieu’s theory of habitus. Such theories are very OO be faithful to the subject matter and evidence/data different from testable theories or hypotheses, and often from which it has been derived; include large-s cale conceptual frameworks which are OO be useful: describe and explain all the relevant data used to comment on or understand phenomena or and observations; explain them in broad terms (‘grand theory’), and are OO be clear: clarify the conceptual framework and the not bounded by space and time (Bacharach, 1989, paradigm in which it works; p. 500). OO demonstrate internal coherence of its component Such ‘grand theory’ (from Mills’s The Sociological elements, consistency and internal logic; Imagination, 1959) is typically abstract and removed OO have great explanatory, predictive, retrodictive and from a specific situation, standing back to set out a generalizable potential; view of the world through a conceptual framework. It OO be replicable; can be formal, conceptual, speculative, overarching and OO have logical and empirical adequacy and internal non-empirical, and it defines areas of study, clarifies coherence; and refines their conceptual frameworks and enlarges OO be able to respond to observed anomalies; the way we consider the social and educational world. OO be parsimonious, excluding any unnecessary ideas Grand theory sets out some fundamental ontological and explanations and stated in simple terms; that and epistemological frameworks and concepts which 73
the context of educational research define an area of study or domain of enquiry (cf. is just so much over-e laboration of concepts almost to Layder, 1994, pp. 28–30). Grand theory is untestable the wilful exclusion of any empirical import’ (p. 45) and unable to be ‘proved’ or ‘disproved’, being a way and that it involves elaborating distinctions arbitrarily of considering the world, and, in that respect, is more which do little or nothing to increase understanding or like an orientation, a rationalization, a belief or an make greater sense of experience (Mills, 1959, p. 33). article of faith (cf. Gorard, 2013, p. 32) than a testable, This echoes Merton’s (1957) view that, though they scientific theory. It is a discourse. may have the architectonic splendour of ‘large philo- Grand theory is a metanarrative, defining an area of sophical systems’, they are also marked by ‘scientific study, being speculative, clarifying conceptual struc- sterility’ (p. 10) and, whilst being admirable for their tures and frameworks and enlarging the way we con- logical consistency, are largely of no relevance to the sider phenomena (Layder, 1994). It defines a field of everyday world (Coser and Rosenberg, 1969, p. 14). To enquiry (Hughes, 1976) and uses empirical material by add to this is the familiar critique of grand theory as way of illustration rather than ‘proof ’ (p. 44). This is being totalizing metanarratives in a world in which no the stuff of some sociological theories, for example single metanarrative is operable. Marxism, rational choice theory, structuralism and For the educational researcher, grand theories can functionalism. inform an understanding of the world and articulate a Whilst sociologists may be excited by the totalizing way of looking at phenomena or explain the context of and all-e ncompassing nature of such grand theories, a study, and, in this respect, might prompt the develop- they have been subject to considerable critique. For ment of research questions (cf. White, 2009, p. 26). example, Merton (1957), Coser and Rosenberg (1969), Whereas empirical theory looks to ‘proof ’ as a criterion Doll (1993) and Layder (1994) contend that whilst they of its validity, grand theory looks to logical coherence, might possess the attraction of large philosophical explanatory potential and articulation of key concepts systems of considerable – Byzantine – architectonic in understanding a phenomenon. splendour and logical consistency, nevertheless they are scientifically sterile, irrelevant and out of touch with Middle-range theory a world that is characterized by openness, fluidity, change, heterogeneity and fragmentation. Between ‘grand’ theory and small-scale theories or However, Murphy (2013) makes a strong case for minor working hypotheses lie what Merton (1967) the applicability of such social theories to everyday termed ‘middle-range theories’: subsets of overarching practices and lives, as they can explain: social change theories which focus on specific topics and seek to and development; how and why people behave as they explain them. These typify much educational research. do; the operations of power, culture and social struc- Middle-range theory focuses on a particular phenome- tures; issues of gender, race, class, ability and identity; non or case in context, and seeks to explain it in terms modernity and postmodernity; institutions and their of underlying mechanisms, factors or principles that operations, and so on. He argues powerfully for the role give rise to the phenomenon or case in point. It uses a of social theory in educational research, indeed his limited set of assumptions to derive hypotheses/ques- website for Social Theory Applied (http://socialtheo- tions logically which can be tested empirically ryapplied.com) provides many links between theory, (Merton, 1957). It starts with a specific empirical phe- practice and research. For example, in the field of edu- nomenon (in contrast to a broad abstract area such as cation he argues that social theory can inform topics ‘capitalism’ or ‘social structure’, which is the stuff of such as inequality and inclusion, educational selves and ‘grand theory’) and abstracts and creates from the phe- subjectivities, curricular and pedagogic practice, and nomenon general statements which can be verified governance and management (pp. 8–9), and he draws by data. on the social theories of Habermas, Foucault, Bourdieu Merton (1967), in defining middle-range theories, and Derrida. notes that this intermediate position uses abstractions Grand theories have been criticized for their aridity and concepts ‘close enough to observed data to be and inability to stand empirical scrutiny or testing incorporated in propositions that permit empirical (Merton, 1957; Mills, 1959; Layder, 1994). This testing’ (p. 39) in studying delimited aspects of social charge, however, might appear unfair, attempting to life. Here ‘theory’ comprises propositions which are judge a theory by criteria which it does not strive to logically interconnected and from which ‘empirical meet. There remains the problem that too easily grand uniformities’ can be derived to explain all the observed theory can become empty rationalization; for example, uniformities of social behaviour, social organization Hughes (1976) comments that ‘this form of theorizing and social change (p. 39). He is against theories that are so abstract that they cannot be tested, but he also argues 74
Theory in educational research for middle-range theories which can apply abstract repressive factors, illegitimate in the sense that they do concepts to different spheres of social structure and not operate in the general interest – one person’s or social behaviour, i.e. which bridge micro- and macro- group’s freedom and power is bought at the price of social problems, looking at here-a nd-now matters. another’s freedom and power. Pawson (2008) provides an example of middle- For the educational researcher, a normative theory, range theory in education. In improving higher educa- of which critical theory is an example, should be clear tion, recourse may be made to a variant of ‘naming and in its methodology, set criteria for its validation (both shaming’, based on the theory that publishing compara- empirical and non‑empirical) and denote the type(s) of tive data on higher education institutions (e.g. rankings) evidence that could substantiate the theory. As with will stimulate competition between them, and thereby other views of ‘theory’, normative theory must have drive up standards (p. 18). substantive concepts which are internally coherent and Middle-range theory draws on empirical evidence, logically tenable. It might be a middle-range theory, and embraces, but does not confine itself to, hypothesis ‘grand theory’ or empirical theory. It should demon- testing, prediction, isolation and control of variables. It strate appropriate precision and universality, possess can be explanatory and interpretive, seeking to under- explanatory power and predictive validity, and, as for stand a situation in its context, a hermeneutic, practical other theories, have greater validity claims and war- exercise, following the verstehen approaches of Weber, rants than rival theories. Criteria for judging its worth looking at meaning in social contexts. Much educa- include its ability to achieve the norms and values tional research appeals to middle-range theory, focus- explicit in the theory, for example, for critical theory ing on a specific case or phenomenon and seeking to this means its potential for, and achievement of, practi- explain it using concepts and hypotheses which, whilst cal empowerment, freedom, equality, social justice, being somewhat abstracted from the specific case in democracy and emancipation. question, are not part of the parlance of grand, totaliz- A contentious issue raised by normative theory is ing theoretical edifices. whether it is the task of educational research to have an explicit political or ideological agenda, to engage in Normative theory political activism and/or policy making, or whether educational research should simply stick to providing Normative theories explain how people, groups, institu- factual knowledge that is used by others for political, tions etc. ought to operate within a specific system of normative agendas and policy making (Hammersley, social values (norms). They are prescriptive, for 2014). Should educational researchers be concerned example: ‘all students should be taught ideology cri- only with the neutral, disinterested pursuit and provi- tique’; ‘schools should promote democracy’. sion of knowledge (recognizing that this is itself a value A clear example of a normative theory is critical position) or seek to press home a particular ideology or theory, which is such a large field that we devote an set of values? Should educational researchers directly entire chapter to it (Chapter 3). Critical theory seeks to answer questions of values, of desirability, of right and uncover the interests at work in particular situations, to wrong, or should they just stick to facts, not values? interrogate the legitimacy of those interests and to iden- We discuss this in Chapter 3. tify the extent to which they are legitimate in serving equality and democracy. It has a deliberate intention of Grounded theory being transformative and emancipatory, promoting democracy and individual freedoms. It is practical and Grounded theory (addressed fully in Chapter 37) is not political, to bring about a more just, egalitarian society predetermined, but, rather, emerges from, and is conse- in which individual and collective freedoms operate, quent to, data (Glaser and Strauss, 1967), i.e. it is and it seeks to eradicate the exercise and effects of ille- grounded in data and rises up from the ground of data: gitimate power. a ‘bottom-u p’ process. It seeks to generate rather than Critical theory, operating through ideology critique, simply to test an existing theory. Grounded theory com- identifies unequal power relations in society, interro- mences with data on a topic or phenomenon of concern, gates their legitimacy, identifies what has brought an and then, using tools such as theoretical sampling, individual or social group to relative powerlessness or, coding, constant comparison, identification of the core indeed, to power, and questions the legitimacy of variable, and saturation, the theory – the explanation repression, voice, ideology, power, participation, repre- and explanatory framework – emerges from the analy- sentation, inclusion and interests. It argues that much sis and study of, and reflection on, the phenomena and behaviour (including research behaviour) is the data under scrutiny (cf. Strauss and Corbin, 1990, outcome of particular illegitimate, dominatory and p. 23). Grounded theory identifies key features and 75
the context of educational research relationships emerging from data and categories and Stage 5: Set out the assumptions underlying the causal then proposes a plausible explanation of the phenome- explanation (which concerns income differentials, class non under study by drawing on the data generated, differentials, risk aversion, anticipated costs and ensuring that the theory explains all the data without benefits). exception. Stage 6: Test the causal hypotheses empirically, in A concern of researchers working with grounded which data are collected on differences in aspirations theory is how far one can generalize from it: is the and decisions by social class which are caused by: (a) grounded theory limited to the specific study; can it be perceptions of costs; (b) relative risk aversion; and (c) applied to other similar situations; does it have wider perceptions of relative benefit. generalizability like a ‘grand’ theory? We explore this Stage 7: Draw conclusions based on the test. Here, in Chapter 37, noting that grounded theory researchers there are class differences in terms of relative ambition, often refer to ‘transferability’ of the findings from one risk aversion, perceived costs and benefits, amounts of situation to another, which is based on the judgement effort required, assurances of success (and the signifi- of the researcher or reader, and that it is problematic to cance of this), fear of downward social mobility, rely too heavily on reader judgement in determining the income, occupational choices, and the need for qualifi- status of the grounded theory. Grounded theory has cations. These, in turn, based on empirical data, support some affinity to ‘middle-range’ theories. his explanation of the factors of relative risk aversion Empirical, grand, middle-range, normative and and fear of downward social mobility exerting causal grounded theories have practical value. They under- power on educational decision making which, in turn, pin research design, data analysis and, indeed, the lead to class differentials in educational attainment generation of new theories. For example, they suggest being maintained (p. 99). the relationships between variables and concepts; they can be used to set up the research, what it seeks In this example, grand theory (sociological theory of to do and what key concepts are included in the rational choice), empirical theory (e.g. hypothesis gen- research; they can predict and explain findings, sug- eration and testing) and middle-range theory (the expla- gesting at a high level of abstraction why such and nation of a particular phenomenon) all have their place such occurs. at different stages of the research. We advise readers to Social theories and psychological theories can be look at the example in Chapter 6 for a fuller account usefully brought into educational research. We give a of this. fully worked example in Chapter 6, and we refer readers to this. By way of summary of that example 4.5 Where does theory come from? here, Goldthorpe (2007) plans, conducts and reports research which explains the causes of ‘persistent differ- Where does a theory come from? How does one estab- entials in educational attainment’ despite increased lish a theory? Clearly there are many starting points. educational expansion, provision and uptake across the One starting point may be through observation and class structure (p. 21). In his research, theory plays a analysis, for example, of observed regularities or rela- part at each of seven stages: tionships, of an association of events, of data; another is through reflection and creativity. Another may be Stage 1: Establish exactly what has to be explained, through asking a ‘what if ’ question, for example, ‘if examining regularities and patterns of relevant assessment were to become more authentic, would it phenomena. increase student motivation?’, or ‘does repeating a year Stage 2: Set out possible theoretical foundations for the at school improve student performance?’ Another start- investigation, which utilize high-level sociological and ing point might be less concerned with regularities than economic theory (Marxist theory, liberal theory, cul- a single instance: ‘why did such-a nd-such happen?’ or tural theory and rational choice theory). ‘why is such-and-such happening?’ Another starting Stage 3: Examine, evaluate and eliminate rival theoreti- point may be from literature which gives rise to a cal foundations, selecting the most fitting and justifying theory, or previous research. the selection (in his research it is rational choice Echoing C. Wright Mills’s (1959) view of The Socio- theory). logical Imagination, theory generation is a human act. It Stage 4: Hypothesize a causal explanation on the basis of is the creative imagination in the minds of humans which the best theoretical foundation. This operates at a sophis- links concepts and sets the grounds for logical coherence ticated theoretical level, arguing that different classes and theory validation. As Bacharach (1989) argues, view the costs, risks and benefits differently (p. 34). theories derive from people’s ‘creative imagination and 76
Theory in educational research ideological orientation or life experience’ (p. 498). generalize, i.e. what is it a theory of, and why is this Theory comes from humans creating and connecting relevant for your research? ideas and concepts into an explanatory framework OO What makes your theory interesting? within articulated boundaries, and testing them. OO How important is theory in your research? (Why) do Similarly, it is humans who decide whether a theory you need to make it explicit? holds water; who proffer explanations, predictions and OO What theories are you using, and why these: how generalizations; who select what is relevant to include relevant are they to your study? in a theory; and who determine how to test, validate OO What is the purpose of your research with regard to and falsify the theory. Whilst observations and data theory, for example, to test, apply, explain, under- may provide the fuel for theory generation and testing, stand, generate, critique, validate, extend, refine, they are not the theory itself. refute a theory? Theory typically precedes research questions; OO What is the relationship between your theory, your research does not start with research questions. We research and your research question(s)? assemble observations, ideas, concepts, reflections, OO What type of theory are you using (e.g. empirical, consider what they mean, and then formulate our theo- grand, normative, middle-range, critical, grounded)? ries, our frameworks of related concepts and proposi- OO What methodologies are you using to work with tions. Then we construct our tests of the theories, which your theory (e.g. empirical testing, hermeneutic may utilise a hypothetico-deductive empirical method interpretation and explanation, ideology critique)? (e.g. correlational, causal analysis, difference testing, OO What criteria are you using to validate your theory regressions, or other kinds of analysis and interven- (e.g. compatibility with empirical data, logical tion), a hermeneutic method, an emancipatory, trans- coherence and adequacy, explanatory potential, formative method through ideology critique, a achievement of transformative and emancipatory grounded theory approach, or others. potential etc.)? How will you validate your theory? Depending on the type of research and research OO How does your theory give rise to testable proposi- question, we often commence with a theory and then tions/hypotheses, or inform a hermeneutic exercise, test it, moving from theory to hypothesis generation to or bring about its espoused values or normative hypothesis testing and observation to prediction to con- intentions? clusion to generalization. Alternatively, as in a OO What are the boundaries of, and assumptions in, grounded theory approach, we may conduct post hoc your theory? theory generation, i.e. starting with data and, through the tools of grounded theory, end up with an emergent 4.7 Conclusion theory which subsequently we may wish to test in other contexts and conditions. Researchers frequently pose the question ‘do we need theory?’ This is an inappropriate question, for, one the 4.6 Questions about theory for one hand, like it or not, we cannot escape theory: it is researchers there, it underpins what we do, whether or not we are conscious of it. One of the contributions of the post- In considering the role of theory in educational positivists is in drawing attention to the point that no research, researchers can address the following observation is theory-free. A more useful question is questions: what we need theory for, as, by itself, it may underde- termine or unnecessarily constrict the full gamut of the OO What definition of theory are you using? research enterprise. OO What is your theory (state it clearly)? Is it a hypoth- Theories help us to think. They articulate and organ- ize ways of approaching a problem or phenomenon. esis, a set of related concepts, a value system, a They assemble and clarify key concepts and their rela- political/ideological agenda, an explanatory frame- tionships, principles and abstractions, explanations and work, a possible explanation, an opinion, an propositions. They can stimulate research questions and approach etc.? hypotheses. Theories connect concepts into a logical OO What is the theory/theoretical framework in which and coherent whole or framework. you are working? What are its key components, Theories help us to learn: they can render ideas test- constructs, concepts and elements, and how do they able, define ways of working, tell us which ideas, state- relate to each other logically and coherently? ments, conclusions, lines of reasoning stand fast when OO What is your theory seeking to describe (‘what’), tested rigorously and which appear to be valid, reliable, explain (‘how’, ‘why’, ‘when’), predict (‘what if ’), 77
the context of educational research credible, legitimate, sound, reasonable and useful. Some Like it or not, use them or not, one or more theories theories (e.g. descriptive, analytical, explanatory) help us lie behind an educational research study. Whether these to understand a phenomenon; others define an approach, drive the research, are incidental or unimportant to it conceptual frame and reference system (e.g. grand are matters for each researcher. Perhaps middle-range theory); yet others (e.g. normative, critical) seek to theory is both a useful compromise and, more posi- change the world or to promote an agenda. Theories are tively, a useful way forward. tools for thinking, describing, understanding, predicting, The companion website to the book provides explaining, proving, organizing, connecting ideas and PowerPoint slides for this chapter, which list the struc- concepts, generalizing, generating research enterprises ture of the chapter and then provide a summary of the and suggesting research questions and answers. It is for key points in each of its sections. This resource can be each researcher to decide which meaning(s) of ‘theory’ found online at: www.routledge.com/cw/cohen. is/are being used in a specific research project. Companion Website The companion website to the book includes PowerPoint slides for this chapter, which list the structure of the chapter and then provide a summary of the key points in each of its sections. These resources can be found online at www.routledge.com/cw/cohen. 78
Evaluation and research CHAPTER 5 This brief chapter indicates some key similarities and a formulating operational questions, differences between research and evaluation. The b deciding appropriate methodologies, chapter sets out: c deciding which instruments to use for data OO similarities between research and evaluation collection, OO differences between research and evaluation d deciding on the sample for the investigation, OO connections between evaluation, research, politics e addressing reliability and validity in the investi- and policy making gation and instrumentation, f addressing ethical issues in conducting the 5.1 Similarities and differences between research and evaluation investigation, g deciding on data-analysis techniques, There are many similarities between research and h deciding on reporting and interpreting results. evaluation. Both share commonalities in terms of methodologies, ethical issues, sampling, reliability The features outlined above embrace many elements of and validity, instrumentation and data analysis, i.e. the scientific method (see Chapter 1). their operational practices (Arthur and Cox, 2014, Researchers and evaluators pose questions and p. 139; Garcia et al., 2014). However, there are also hypotheses, select samples, manipulate and measure differences; the problem of trying to identify differ- variables, compute statistics and data, and state conclu- ences between evaluation and research is compounded sions (cf. Garcia et al., 2014). Nevertheless there are because not only do they share several of the same important differences between evaluation and research methodological characteristics, but one branch of that are not always obvious simply by looking at publi- research is called evaluative research or applied cations. Publications do not always make clear the back- research. This is often kept separate from ‘blue-skies’ ground events that gave rise to the investigation, nor do research in that the latter is open‑ended, exploratory, they always make clear the uses of the material that they contributes something original to the substantive field report, nor do they always make clear what the dissemi- and extends the frontiers of knowledge and theory, nation rights are (Sanday, 1993) and who holds them. whereas in the former the theory is given rather than Several commentators set out some of the differences interrogated or tested. Plewis and Mason (2005, between evaluation and research. For example, Smith p. 192) suggest that evaluation research is, at heart, and Glass (1987) offer eight main differences: applied research that uses the tools of research in the social sciences to provide answers to the effective- 1 The intents and purposes of the investigation: the ness and effects of programmes. One can detect researcher wants to advance the frontiers of know many similarities between the two in that they both ledge of phenomena, to contribute to theory and to use methodologies and methods of social science be able to make generalizations; the evaluator is less research generally (Norris, 1990), covering, for interested in contributing to theory or the general example: body of knowledge. Evaluation is more parochial than universal (pp. 33–4). OO the need to clarify the purposes of the investigation; OO the need to operationalize purposes and areas of 2 The scope of the investigation: evaluation studies tend to be more comprehensive than research in the investigation; number and variety of aspects of a programme that OO the need to address principles of research design that are being studied (p. 34). include: 3 Values in the investigation: much research aspires to value-n eutrality. Evaluations must represent multi- ple sets of values and include data on these values. 79
the context of educational research 4 The origins of the study: research has its origins and 8 Salience of the value question: In evaluation, value motivation in the researcher’s curiosity and desire to questions are central and usually determine what know (p. 34). The researcher is autonomous and information is sought. answerable to colleagues and scientists (i.e. the research community), whereas the evaluator is 9 Investigative techniques: While there may be legit- answerable to clients and stakeholders. The imate differences between research and evaluation researcher is motivated by a search for knowledge; methods, there are far more similarities than differ- the evaluator is motivated by the need to solve prob- ences with regard to techniques and procedures for lems, allocate resources and make decisions. judging validity. Research studies are public; evaluations are for a restricted audience. 10 Criteria for assessing the activity: The two most important criteria for judging the adequacy of 5 The uses of the study: the research is used to further research are internal and external validity; for eval- knowledge; evaluations are used to inform decisions. uation they are utility and credibility. 6 The timeliness of the study: evaluations must be 11 Disciplinary base: The researcher can afford to timely; research need not be. Evaluators’ timescales pursue inquiry within one discipline; the evaluator are given; researchers’ timescales need not be given. cannot. 7 Criteria for judging the study: evaluations are However, we include below a more comprehensive set judged by the criteria of utility and credibility; of distinguishing features, but it must be emphasized research is judged methodologically and by the con- that they are not at all as rigidly separate as the bullet tribution that it makes to the field (i.e. internal and points below might suggest: external validity). OO O rigins: Research questions originate from scholars 8 The agendas of the study: an evaluator’s agenda is working in a field; evaluation questions issue from given; a researcher’s agenda is her own. stakeholders. Norris (1990) reports work by Glass and Worthen in OO A udiences: Evaluations are often commissioned and which they identified eleven main differences between they become the property of the sponsors and are evaluation and research: not for the public domain; research is disseminated widely and publicly. 1 The motivation of the enquirer: Research is pursued largely to satisfy curiosity; evaluation is OO P urposes: Research contributes to knowledge in the undertaken to contribute to the solution of a field, regardless of its practical application, and pro- problem. vides empirical information, i.e. ‘what is’; evalu ation is designed to use that information and those 2 The objectives of the search: Research and evalu facts to judge the worth, merit, value, efficacy, ation seek different ends. Research seeks conclu- impact and effectiveness of something (Scriven, sions; evaluation leads to decisions. 2004; Arthur and Cox, 2014), i.e. ‘what is valuable’ (Mathison, 2007, p. 189). Research is conducted to 3 Laws versus description: Research is the quest for gain, expand and extend knowledge; evaluation is laws (nomothetic); evaluation merely seeks to conducted to assess performance and to provide describe a particular thing (idiographic). feedback (Levin-R ozalis, 2003). Research is to gen- erate theory; evaluation is to inform policy making 4 The role of explanation: Proper and useful evalu (Patton, 2002). Research is to discover; evaluation ation can be conducted without producing an is to uncover (Arthur and Cox, 2014). Research explanation of why the product or project is good seeks to predict what will happen; evaluation con- or bad or of how it operates to produce its effects. cerns what has happened or what is happening (ibid.). 5 The autonomy of the enquiry: Evaluation is under- taken at the behest of a client; researchers set their OO S tance: The evaluator is reactive (e.g. to a pro- own problems. gramme); the researcher is active and proactive (Levin-R ozalis, 2003, p. 15). 6 Properties of the phenomena that are assessed: Evaluation seeks to assess social utility directly; OO S tatus: Evaluation is a means to an end; research is research may yield evidence of social utility but an end in itself (Levin-R ozalis, 2003). often only indirectly. OO F ocus: Evaluation is concerned with how well 7 Universality of the phenomena studied: Research- something works; research is concerned with how ers work with constructs having a currency and something works (Mathison, 2007). scope of application that make the objects of evalu- ation seem parochial by comparison. 80
Evaluation and research OO O utcome focus: Evaluation is concerned with the OO P olitics of the situation: The evaluator may be achievement of intended outcomes; research may unable to stand outside the politics of the purposes not prescribe or know its intended outcomes in and uses of, or participants in, an evaluation; the advance (science concerns the unknown). researcher provides information for others to use. OO P articipants: Evaluation focuses almost exclusively OO U se of theory: Researchers base their studies in social on stakeholders; research has no such focus (Math- science theory; this is not a necessary component of ison, 2007). evaluation (Scriven, 1991). Research is theory- dependent; evaluation is ‘field-d ependent’, i.e. not OO S cope: Evaluations are concerned with the particular, theory-d riven but derived from the participants, the for example, a focus only on specific programmes. project and stakeholders. Researchers create the They seek to ensure internal validity and often have a research findings; evaluators may (or may not) use more limited scope than research. Research often research findings (Levin-R ozalis, 2003, pp. 10–11). seeks to generalize (external validity) and, indeed, may not include evaluation (Priest, 2001). OO R eporting: Evaluators report to stakeholders/com- missioners of research; researchers may include OO S etting of the agenda: The evaluator works within a these and may also report more widely, for example, given brief; the researcher has greater control over in publications (Beney, 2011). what will be researched (though often constrained by funding providers). Evaluators work within a set OO S tandards for judging quality: Judgements of of ‘givens’, for example, programme, field, partici- research quality are made by peers; judgements of pants, terms of reference and agenda, variables; evaluation are made by stakeholders (Patton, 2014). researchers create and construct the field. For researchers, standards for judging quality include validity, reliability, accuracy, causality, generaliza- OO R elevance: Relevance to the programme or what is bility, rigour; for evaluators, to these are added being evaluated is a prime feature of evaluations; utility, feasibility, involvement of stakeholders, side relevance for researchers has wider boundaries (e.g. effects, efficacy, fitness for purpose (though, increas- in order to generalize to a wider community). ingly, utility value and impact are seen as elements Research may be prompted by interest rather than for judging research) (Patton, 1998). relevance. For the evaluator, relevance has to take account of timeliness and particularity (Levin- Mathison (2007) comments that the two major dimen- Rozalis, 2003, pp. 20–1). sions for distinguishing between research and evaluation are on the particularization/generalization continuum and OO T ime frames: Evaluation begins at the start of the the decision-o riented/conclusion-o riented continuum. project and finishes at its end; research is ongoing The statements above are set out in an either/or and less time-b ound (Levin-Rozalis, 2003) (though manner for conceptual clarity. However, the reality of this may not be the case with funded research). the situation is nowhere near as clear as this; research and evaluation are not mutually exclusive binary oppo- OO U ses of results: Evaluation is designed to improve; sitions, nor, in reality, are there differences between research is designed to demonstrate or prove (Stuf- them. Their boundaries are permeable, similarities are flebeam, 2001). Evaluation ‘provides the basis for often greater than differences and there is often overlap; decision making; research provides the basis for indeed, evaluative research and applied research often drawing conclusions’ (Mathison, 2007, p. 189). bring the two together (Levin-R ozalis, 2003, p. 3); Evaluations might be used to increase or withhold indeed Arthur and Cox (2014) note how easily the two resources or to change practice; research provides have elided in research and assessment exercises. For information on which others might or might not act, each of the above there are many exceptions. For i.e. it does not prescribe. example, both evaluation and research might be con- cerned with generalization, or, indeed with the particu- OO D ecision making: Evaluation is used for micro deci- lar; evaluation may not be for decision making whereas sion making; research is used for macro decision research may be precisely for this purpose (Mathison, making (Mathison, 2007, p. 191). 2007). Both research and evaluation are concerned to produce information and to promote explanation and OO D ata sources and types: Evaluation has a wide field understanding, both of which can contribute to decision of coverage (e.g. costs, benefits, feasibility, justifi– making and policy formation, i.e. both are intimately ability, needs, value for money), so evaluators concerned with politics and both involve political pro employ a wider and more eclectic range of evidence cesses, with differences between them being more from an array of disciplines and sources than researchers. OO O wnership of data: The evaluator often cedes own- ership to the sponsor, upon completion; the researcher holds onto the intellectual property. 81
the context of educational research matters of degree, and both can operate at different Take, for example, the ‘healthy attachments’ pro- levels: individual, local and institutional to national and gramme in a range of UK schools, which Pawson international (Arthur and Cox, 2014). Elliott (1991) (2013) reports. This project was designed to improve notes that evaluation is an essential ingredient of action students’ ‘well-b eing, sense of security and positive research. Patton (1998) argues that differences between regard’ (p. 73), which, in turn, sought to reduce risks to research and evaluation are often arbitrary, and cases health associated with tobacco, alcohol and illegal can be made for saying that they are the same or, drugs (p. 73). Pawson, carefully dissecting out the indeed, are different. claims made for the intervention, shows that it is typical Mathison (2007) reports that some people put of ‘black-b ox analysis’ (p. 74) and that, actually, the research as a subset of evaluation whilst others put eval- positive results overlooked important elements and uation as a subset of research (p. 189). MacDonald were far less clear than the claims made for the (1987) argues that ‘[t]he danger therefore of conceptual- project’s success. Why? Because a range of events and izing evaluation as a branch of research is that evaluators factors occurred, and people’s perspectives and situa- become trapped in the restrictive tentacles of research tions were neglected in the evaluation which, if taken respectability.… How much more productive it would be more seriously, would have led to much more cautious to define research as a branch of evalua tion’ (p. 43). claims. As he says, in response to the question ‘is this A clue to some of the differences between evalu the correct interpretation?’ an accurate answer should ation and research can be seen in the definition of be ‘[p]ossibly. Possibly not’ (p. 75). Pawson demon- evaluation. Most definitions of evaluation include ref- strates clearly the dangers of making unequivocal erence to several key features: (1) answering specific, claims on the basis of under-researched, over- given questions; (2) gathering information; (3) making interpreted, neglectful data and narrow enquiry, and he judgements; (4) taking decisions; (5) addressing the counsels caution in making simplistic claims. As he politics of a situation (Morrison, 1993, p. 2). Morrison says, there are reasons why we may not believe the provides one definition of evaluation as: ‘the provision claims made (p. 16), and evaluators worth their salt of information about specified issues upon which would do well to keep this to the fore in understanding judgements are based and from which decisions for how educational interventions unfold. This may come action are taken’ (p. 2). as unwelcome news to policy makers who seek unequivocal results. 5.2 Evaluation research and policy Pawson notes that the search for ‘what it is about an making intervention that works for whom, in what circumstances, in what respects, over which duration’ (2013, p. 167), i.e. In an era in which educational innovations come thick to focus on the contexts, mechanism and outcomes of the and fast, often on the crest of one wave after another of intervention, is a complex, pragmatic and ongoing government policies and interventions, evaluation has endeavour. In this he notes for evaluators the significance to take to task any notion that quick fixes from inter- of context (pp. 36–8) and of attention to the personal and ventions ‘work’ straightforwardly, or at all. This runs interpersonal dimensions of interventions (pp. 127–31, counter to claims that slogans of evidence-b ased prac- 139–46). Epistemological and substantive support for his tice and ‘what works’ are unproblematic. Rather, iden- approach finds voice in: (a) Popperian doubt, with its tifying ‘what works’ contains myriad complexities and emphasis on conjectures, refutations and the tentative, challenges. Understanding ‘what works’ in education conditional, falsifiable nature of scientific ‘truths’ (p. 99); recognizes that an intervention often unfolds in unex- and (b) Rossi’s disconcerting ‘iron law’ (p. 12) which pected ways because of the complex interplay of par- states that the value of any assessment of impact of a ticipants and contingencies. Often as not, this may large-s cale programme is typically zero (p. 12). This may frustrate the clean, antiseptic world of policy makers. be uncomfortable for both policy makers and evaluators, Innovations rarely ‘work’ entirely in the ways in which but that is the reality of many interventions. they are intended. A range of factors operate on the sit- In putting forward his case for a realist approach to uation, and impact, effectiveness and efficacy are evaluation, Pawson uses acronyms to identify key ele- viewed differently by different participants. Hence key ments and foci of evaluation: questions for evaluation concern, for example, which part(s) of an intervention or programme are working OO CMOs: Contexts, Mechanisms and Outcomes well/less well, why, for whom, under what contingen- (pp. 21–6) (redolent of Stufflebeam’s (1967) cies and conditions, in what circumstances and over Context, Input, Process and Product model of edu- what time period (cf. Pawson, 2013, p. 167). cational evaluation); 82
Evaluation and research OO VICTORE: Volitions, Implementation, Contexts, obtained.… The resolution of these issues commits Time, Outcomes, Rivalry and Emergence the evaluator to a political stance, an attitude to the (pp. 33–46); government of education. No such commitment is required of the researcher. He stands outside the OO TARMATO: Theory, Abstraction, Reusable Con- political process, and values his detachment from it. ceptual Platforms, Model Building, Adjudication, For him the production of new knowledge and its Trust and Organized Scepticism (pp. 85–111). social use are separated. The evaluator is embroiled in the action, built into a political process which Even though it may not be good news for policy concerns the distribution of power, i.e. the allocation makers, Pawson notes that many interventions typically of resources and the determination of goals, roles work and then don’t work. This, he avers, is due in part and tasks.… When evaluation data influences power to context, situations, perspectives, participants, cir- relationships the evaluator is compelled to weight cumstances and affinity to decision makers’ agendas. carefully the consequences of his task specification. He notes that an evaluation is prone to distortion and … The researcher is free to select his questions, and misrepresentation of the multiple factors in, and inter- to seek answers to them. The evaluator, on the other pretations of, an intervention phenomenon if: (a) it hand, must never fall into the error of answering neglects understanding how the perceptions, actions, questions which no one but he is asking. agency and circumstances of humans unfold in, and affect, an intervention; and (b) it is selective on whose (MacDonald, 1987, p. 42) interpretation of a project is adopted. This is salutatory advice for both evaluators and policy makers. Whether that holds as true at the present moment as The effectiveness of interventions vary from school when it was written is a moot point, as funded research to school and from individual to individual (see also often has a strong political motive, and institutional Cartwright and Hardie, 2012). Policy makers are politics (e.g. internal funding decisions) may have a attracted by ‘what works’. However, Pawson (2013) bearing on research. MacDonald argues that evaluation shows that what works for some is unlikely to work for is an inherently political enterprise. His much-u sed all, that it is massive over-d etermination rather than the threefold typology of evaluations as autocratic, bureau- intervention itself which frequently contributes to cratic and democratic is premised on a political reading policy uptake, regardless of impact, and that, even of evaluation (see also Chelinsky and Mulhauser though some policies are doomed to be implemented, (1993), who refer to ‘the inescapability of politics’ to expect them to provide only gains rather than losses (p. 54) in the world of evaluation). is simply naive. In other words, ‘what works’ is prob- The reality of politics often blurs distinctions lematic. This being the case, evaluation seeks to iden- between research and evaluation. Two principal causes tify the contingencies and conditions surrounding a of this blurring lie in the funding and the politics of decision or intervention, for example, if we want to do both evaluation and research. For example, the view of such-and-such then it would be better to adopt approach research as uncontaminated by everyday life is naive A and B, targeted at M and N, and to be aware of and simplistic; Norris (1990, p. 99) argues that such an dangers of X and Y (cf. Pawson, 2013, p. 190). antiseptic view of research ignores the social context of educational research, some of which is located in the 5.3 Research, evaluation, politics hierarchies of universities and research communities and policy making and the funding support provided for some but not all research projects by governments. His point has a pedi- Evaluation and research are beset with issues of poli- gree that reaches back to Kuhn (1962), and is a tics; they take place within a political environment, comment on the politics of research funding and which might be a micro-environment (e.g. a single research utilization. For decades one can detect a huge school) or a larger environment (e.g. a funding body, a rise in ‘categorical’ funding of projects, i.e. defined, research institute). MacDonald (1987) comments that given projects (often by government or research spon- the evaluator: sors) for which bids have to be placed. This may seem unsurprising if one is discussing research grants from is faced with competing interest groups, with diver- government bodies, which are typically deliberately gent definitions of the situation and conflicting policy‑oriented, though one can also detect in projects informational needs.… He has to decide which which have been granted by non‑governmental organi- decision-m akers he will serve, what information will zations a move towards sponsoring policy-o riented be of most use, when it is needed and how it can be projects rather than the ‘blue-s kies’ research mentioned 83
the context of educational research earlier, and the rise of the evidence-based movement in The classic definition of the role of evaluation as research and policy making draws this link ever tighter. providing information for decision makers … is a Indeed Burgess (1993b) argues that ‘researchers are fiction if this is taken to mean that policy-m akers little more than contract workers.… [R]esearch in edu- who commission evaluations are expected to make cation must become policy relevant.… [R]esearch must rational decisions based on the best (valid and come closer to the requirement of practitioners’ (p. 1), reliable) information available to them. echoing Stenhouse’s (1975) advocacy of the teacher- as-researcher and more recently in Pring’s (2015) com- (James, 1993, p. 119) ments on the centrality of practitioner research in educational research. Burgess’s view also points to the Where evaluations are commissioned and have heavily constraints under which research is undertaken; if it is political implications, Stronach and Morris (1994) not concerned with policy issues then research may not argue that the response to this is that evaluations be funded, and research must have some impact on become more ‘conformative’, possessing several policy making. characteristics: The view of the tension between research, evalua- tion and politics is reinforced by several articles in the 1 short-term, taking project goals as given and sup- collection edited by Anderson and Biddle (1991) which porting their realization; show that research and politics go together uncomfort ably because researchers have different agendas and 2 ignoring the evaluation of longer-term learning out- longer timescales than politicians and try to address the comes, or anticipated economic/social consequences complexity of situations, whereas politicians, anxious of the programme; for short-term survival, want telescoped timescales, simple remedies and research that will be consonant 3 giving undue weight to the perceptions of pro- with their political agendas. As James (1993) notes: gramme participants who are responsible for the successful development and implementation of the the power of research-b ased evaluation to provide programme; as a result, tending to ‘over-report’ evidence on which rational decisions can be change; expected to be made is quite limited. Policy-makers will always find reasons to ignore, or be highly 4 neglecting and ‘under-reporting’ the views of class- selective of, evaluation findings if the information room practitioners and programme critics; does not support the particular political agenda oper- ating at the time when decisions have to be made. 5 adopting an atheoretical approach, and generally regarding the aggregation of opinion as the determi- (James, 1993, p. 135) nation of overall significance; Her comments demonstrate a remarkable prescience as, 6 involving a tight contractual relationship with the if anything, the situation has become even more acute programme sponsors that either disbars public than when it was written. reporting, or encourages self-censorship in order to Not only is research a political issue, but this protect future funding prospects; extends to the use being made of evaluation studies. Whilst evaluations can provide useful data to inform 7 undertaking various forms of implicit advocacy for decision making, as evaluation has become more politi- the programme in its reporting style; cized so its uses (or non-u ses) have become more polit- icized. Indeed Norris (1990) provides examples of how 8 creating and reinforcing a professional schizophre- politics frequently overrides evaluation or research evi- nia in the research and evaluation community, dence, and, despite the evidence-based movement, it is whereby individuals come to hold divergent public common to read how politicians introduce interventions and private opinions, or offer criticisms in general in education on the basis of poor, scant or, indeed, no rather than in particular, or quietly develop ‘aca- evidence of their efficacy. Gorard (2005, 2014), for demic’ critiques which are at variance with their example, demonstrates that ‘academies’ in the UK were contractual evaluation activities, alternating between doomed to succeed as there was no evidence to suggest ‘critical’ and ‘conformative’ selves. that they were any better than the school which they replaced. This echoes James’s earlier comment (1993) The points raised so far can apply to large-s cale and where she writes: small-s cale projects. Hoyle (1986), for example, notes that evaluation data are used to bring resources into, or take resources out of, a department or faculty. In this respect the evaluator may have to choose carefully his or her affinities and allegiances (Barton, 2002), as the outcomes and consequences of the evaluation may call these into question. Barton writes that, although the evaluator may wish to remain passive and apolitical, in 84
Evaluation and research reality this view is not shared by those who commis- particularly true in the case of ‘categorically funded’ sion the evaluation or the reality of the situation, not and commissioned research – research which is funded least when the evaluation data are used in ways that by policy makers (e.g. governments, fund-a warding distort the data or use them selectively to justify differ- bodies) under any number of different headings that ent options (p. 377). those policy makers devise. On the one hand this is The issue relates to both evaluations and research, laudable, for it targets research directly towards policy as school-based research is often concerned more with (e.g. the ‘what works’ initiatives); on the other hand it finding out the most successful ways of organization, is dangerous in that it enables others to set the research planning, teaching and assessment of a given agenda agenda. Research ceases to become open‑ended, pure rather than setting agendas and following one’s own research and, instead, becomes the evaluation of given research agendas. This is problem solving rather than initiatives. problem setting. That evaluation and research are being Evaluators may have the power to control the opera- drawn together by politics at both a macro- and micro- tion of the evaluation project and may influence the level is evidence of a continuing interventionism by brief given, whilst the sponsor can only support but not politics into education, reinforcing the hegemony of the control the independence of the evaluator. The issue of government in power. sponsoring research reaches beyond simply commis- Several points have been made so far: sioning research towards the dissemination (or not) of research: who will receive or have access to the find- OO There is considerable overlap between evaluation ings and how the findings will be used and reported. and research; This, in turn, raises the fundamental issue of who owns and controls data, and who controls the release of OO There are some conceptual differences between research findings. Unfavourable reports might be with- evaluation and research, though, in practice, there is held for a time, suppressed or selectively released. In considerable blurring of the edges of the differences other words, research can be brought into the service of between the two; wider educational purposes, for example, the politics of a local education authority, or indeed the politics of OO The funding and control of research and research government agencies. agendas often reflect the persuasions of political Though research and politics intertwine, the rela- decision makers; tionships between educational research, politics and policy making are complex because research designs OO Evaluative research has increased in response to cat- strive to address a complex social reality (Anderson egorical funding of research projects; and Biddle, 1991). A piece of research does not feed simplistically or directly into a specific piece of policy OO The attention being given to, and utilization of, making. Rather, research generates a range of different evalua tion varies according to the consonance types of knowledge: concepts, propositions, explana- between the findings and their political attractive- tions, theories, strategies, evidence and methodologies. ness to political decision makers. These feed subtly and often indirectly into the decision- m aking process, providing, for example, direct inputs, There is very considerable blurring of the edges general guidance, a scientific gloss, orienting perspec- between evaluation and research because of the politi- tives, generalizations and new insights. cal intrusion into, and use of, these two types of study. The degree of influence exerted by research depends One response to this can be seen in Burgess’s (1993a) on careful dissemination; too little and its message is view that a researcher needs to be able to meet the ignored, too much and data overload confounds deci- sponsor’s requirements for evaluation whilst also gen- sion makers and makes them cynical – the syndrome of erating research data (engaging the issues of the need the boy who cried wolf (Knott and Wildavsky, 1991). to negotiate ownership of the data and intellectual prop- Hence researchers must give care to utilization by erty rights); for an example of this, see Garcia et al. policy makers (Weiss, 1991a), reduce jargon, provide (2014). summaries and improve links between the two cultures Research and politics are inextricably bound of researchers and policy makers (Cook, 1991) and, together. Researchers in education are advised to give further, to the educational community. Researchers serious consideration to the politics of their research must cultivate ways of influencing policy, particularly enterprise and the ways in which politics can steer when policy makers can simply ignore research find- research (Hammersley, 2014). For example, one can ings, commission their own research (Cohen and Garet, detect a trend in educational research towards more evaluative research, where, for example, a researcher’s task is to evaluate the effectiveness (often of the imple- mentation) of given policies and projects. This is 85
the context of educational research 1991) or underfund research into social problems find much research too uncertain in its effects (Kerlin- (Coleman, 1991; Thomas, 1991). Researchers must rec- ger, 1991; Cohen and Garet, 1991), too unspecific and ognize their links with the power groups who decide too complex in its designs, and of limited applicability policy. Research utilization takes many forms depend- (Finn, 1991). This, reply the researchers, misrepresents ing on its location in the process of policy making, for the nature of their work (Shavelson and Berliner, 1991) example, in research and development, problem and belies the complex reality which they are trying to solving, interactive and tactical models (Weiss, 1991b). investigate (Blalock, 1991). Capturing social complex- The impact of research on policy making depends ity and serving political utility can run counter to each on its degree of consonance with the political agendas other. As Radford (2008, p. 506) remarks, the work of of governments (Thomas, 1991) and policy makers researchers is often driven by objectivity and independ- anxious for their own political survival (Cook, 1991) ence from, or disinterestedness in, ideology, whereas and the promotion of their social programmes. Research policy makers are driven by interests, ideologies and is used if it is politically acceptable. That the impact of values. research on policy is intensely and inescapably political The issue of the connection between research and is a truism. Research too easily becomes simply an politics – power and decision making – is complex. On ‘affirmatory text’ which ‘exonerates the system’ another dimension, the notion that research is inherently (Wineburg, 1991) and is used by those who seek to a political act because it is part of the political processes hear in it only echoes of their own voices and wishes of society has not been lost on researchers (cf. Hammers- (Kogan and Atkin, 1991). ley, 2011, 2014), and this harks back to Chapter 3 in its There is a significant tension between researchers discussion of value-neutrality and partisan research. and policy makers. The two parties have different, and Researchers cannot be blind to politics, just as politicians often conflicting, interests, agendas, audiences, times- and decision makers should not be blind to research evi- cales, terminology and concern for topicality (Levin, dence and evaluation. As evaluation and research draw 1991). These have huge implications for research ever closer together it is prudent for researchers to reflect styles. Policy makers anxious for the quick fix of super- on Becker’s (1967) rallying call for researchers to con- ficial facts seek unequivocal data, short-term solutions sider carefully ‘whose side are we on’. and simple, clear remedies for complex and generalized The companion website to the book provides Pow- social problems (Cartwright, 1991; Cook, 1991; erPoint slides for this chapter, which list the structure Radford, 2008, p. 506; Cartwright and Hardie, 2012): of the chapter and then provide a summary of the key the Simple Impact model (Biddle and Anderson, 1991; points in each of its sections. This resource can be Weiss, 1991a, 1991b). Moreover, policy makers often found online at: www.routledge.com/cw/cohen. Companion Website The companion website to the book includes PowerPoint slides for this chapter, which list the structure of the chapter and then provide a summary of the key points in each of its sections. These resources can be found online at www.routledge.com/cw/cohen. 86
The search for CHAPTER 6 causation This chapter introduces key issues in understanding and ‘under what conditions and circumstances’. They causation in educational research. These include: want to predict what will happen if such-and-such an intervention is introduced, and how and why it will OO causes and conditions produce a particular effect. This points us to an impor- OO causal inference and probabilistic causation tant feature of educational research, which is to look for OO causation, explanation, prediction and correlation causation: what are the effects of causes and what are OO causal over-d etermination the causes of effects? This is not a straightforward OO the timing and scope of the cause and the effect enterprise, not least because causation is often not OO causal direction, directness and indirectness observable but can only be inferred, and it is highly OO establishing causation unlikely that indisputable causality is ever completely OO the role of action narratives in causation discoverable in the social sciences. At best probabilistic OO researching causes and effects causation offers a more fitting characterization of cau- OO researching the effects of causes sation in educational research. Causation is often con- OO researching the causes of effects sidered to be the ‘holy grail’ of educational research, and this chapter introduces some key considerations in 6.1 Introduction investigating causation. Our brains seemed to be hard-wired to think causally, 6.2 Causes and conditions but, for educational researchers, tracing and using cau- sality is challenging. Working with cause and effect, Novice researchers are faced with many questions con- researchers must address the importance of being able cerning causation in their research, for example: to use educational research findings in the ‘real world’, that is, the world in which we have not isolated and OO whether the research is seeking to establish causa- controlled out a swathe of pertinent factors, or made so tion, and if so, why; many assumptions and ceteris paribus get-out clauses (Kincaid, 2009) as to render the research of little or no OO deciding when causation is demonstrated, recogniz- value, or found effects which are wonderful in the sani- ing that causation is never 100 per cent certain; tized, artificial world of the laboratory but useless in the ‘real world’ outside it (e.g. Cartwright and Hardie, OO deciding what constitutes a cause and what consti- 2012). tutes an effect; There is a wide vocabulary of causality in research, with many relevant words and phrases: for example, OO deciding what constitutes evidence of the cause and ‘are caused by’, ‘influence’ (verb and noun), ‘attributed evidence of the effect; to’, ‘depend on’, ‘impact’ (verb and noun), ‘effect’ (verb and noun), ‘direction of the relationship’, ‘posi- OO deciding the kind of research and the methodology tive influence’, ‘positive impact’, ‘result’ (verb and of research needed if causation is to be investigated; noun), ‘mediation effect’, ‘effect of ’, ‘due to’, ‘condi- tion’ (verb), ‘leading to’, ‘consequences of ’, ‘because OO deciding whether the research is investigating the of ’, ‘affect’, ‘reason for’, ‘to force’, ‘driven’, ‘lead to’ cause of an effect, the effect of a cause, or both. etc. Though the vocabulary is varied, the issue of dem- onstrating causality remains a challenge to researchers. To infer simple, deterministic or regular causation may Educationists and social scientists are concerned not be to misread many situations, excepting, perhaps, only for ‘what works’ but ‘why’, ‘how’, ‘for whom’ those where massive single causation is clear. It may be more useful for the researcher to consider causal proc- esses rather than single events (Salmon, 1998), not least because there is often more than a single cause at work in any effect and there may be more than one effect from a single cause. Indeed, the researcher has to 87
the context of educational research distinguish between causes, reasons, motives, determi- situation, i.e. the several conditions that, themselves, nation and entailment, and whilst these might all exert contribute to the accident. The presence of those causes causal force in some circumstances or enable us to that are included affects their relative strengths in a spe- make causal explanations or predictions, in other cir- cific context, and the absence of some of these causes in cumstances they do not. the same context may raise or lower the relative strengths What, then, makes a cause a cause, and an effect an of others. effect? How do we know? Though we can say that cau- The example of falling on the ice also indicates an sation takes places in a temporal sequence – the cause important feature of causation: causes cannot be taken precedes the effect, and with temporal succession in isolation, they may need to be taken together (com- (Hume’s criterion of ‘priority’; Hume, 1955; Norton pound causes, i.e. they only exert causative force when and Norton, 2000), this does not help the researcher acting in concert), and there may be interaction effects very much. between them. On its own, the patch of ice might not One distinguishing indication that causation is have caused my fall and broken arm; it was perhaps taking place or has taken place is the presence of coun- neither sufficient nor necessary, as I could have fallen terfactuals (Mackie, 1993), i.e. the determination that and broken my arm anyway because of my slippery the absence of X (the supposed cause) would have led shoes and poor balance. On its own, my poor balance to the absence of Y (the effect); ‘if X had not happened did not cause me to fall and break my arm. On its own, then Y would not have happened’. If we are seeking to the darkness did not cause me to fall and break my arm. establish that such-and-such is a contributing cause (X) On their own, my slippery soles did not cause me to fall of an effect (Y) we ask ourselves whether, if that sup- and break my arm. On their own, my brittle bones did posed cause had not been present, then would the effect not cause me to fall and break my arm. But put all these have occurred or been what it actually was; if the together and we have sufficient conditions to cause the answer is ‘no’ then we can suppose that X is a true accident. For the researcher, looking for individual cause. For example, if there had been no ice on a path causes in a contextualized situation may be futile. then I would not have fallen over and broken my arm. In understanding the causes of effects, one has to So the presence of ice must have been a contributing understand the circumstances and conditions in which cause of the effect, one of many causes (e.g. my poor the two independent factors – the cause and the effect – sense of balance, my poor eyesight in not seeing the are located and linked (the link is contingent rather than ice, the ambient darkness, wearing slippery-s oled analytic). Discovering the circumstances – conditions – shoes, brittle bones because of age etc.). in which one variable causes an effect on another is The counterfactual argument is persuasive, but prob- vital in understanding causation, for it is the specific lematical: how do we know, for example, what the combination of necessary and/or sufficient conditions outcome would have been if there had been no patch of that may produce an effect. Causes of effects work in ice on the path where I was walking? Can we predict specific circumstances and situations, and account has with sufficient certainty to attribute counterfactual cau- to be taken of these circumstances and conditions. sality here? How can we prove that the effect would not For the researcher, the difficulty in unravelling the have happened if a particular cause had not been present? effects of causes and the causes of effects is heightened How do we know that I would or would not have slipped by the fact that causes may be indirect rather than direct and fallen if the ice had not been present? In true experi- (cause A causes effect B, and effect B causes effect C) ments this is addressed by having a control group: the or that they may only become a cause in the presence control group is supposed to indicate what would have of other factors (I may fall over on ice and not break happened if the intervention had not occurred. The my arm if I am young and land well, but, as an older problem is that much research is not experimental. person with more brittle bones, I may land awkwardly If it were only the presence of ice that caused me to and break my arm – the fall is not a sufficient condition fall and break my arm, then this would be a very simple or cause of my broken arm). indication of causality; the problem is that the presence of ice in this instance is perhaps not a sufficient cause – 6.3 Causal inference and had my balance been good, my eyesight good, the probabilistic causation ambient light good, and if my shoes had had good grips on their soles and my bones were less brittle, then I Identifying and understanding causation may be prob- would not have fallen and broken my arm. lematic for researchers, as effects may not be direct The difficulty here also is to establish the role (if any) linear functions of causes, and because there may be and relative strength of the causes in a multi-c ausal few, many, increasing, reducing, unpredictable, i.e. 88
The search for causation non-linear effects of causes (see Chapter 1 on complex- (2006) sets out five main approaches to establishing ity theory). A small cause can bring about a large or causality, and these are outlined below. irregular effect (or, indeed, no effect, in the presence of Mill’s method of agreement: Let us say that in dif- other factors); a large cause may bring about a small or ferent regions of a country there are several combined irregular effect (or, again, no effect, in the presence of educational reforms taking place, designed to increase other factors). Causation is often an inductive and student mobility (Table 6.1): (a) increased educational empirical matter rather than a logical, deductive matter, financing; (b) curriculum reforms; (c) providing more and, indeed, it is often unclear what constitutes a cause places in vocational training; and (d) introducing and what constitutes an effect as these are often National Qualifications Frameworks (NQFs). We wish umbrella terms, under which are sub‑causes and sub- to see which of these factors is causing increased effects, causal processes, causal chains, causal webs student mobility. and causal links bringing several factors together both Mill’s method of agreement here states that in all the at a particular point in time (the moment of falling, in cases where the effect occurs (‘increased student mobil- the example above) and in a temporal sequence. ity’), if there is only one factor common to all the cases Further, there is an asymmetry at work in causation then that factor is the cause. Here only one column effect – a cause can produce an effect but not vice (‘Introducing the NQF ’) has all four factors present. Is versa: being young, good-looking and female may help it safe to conclude that, ceteris paribus, the NQF is the me to pass my driving test if I am in the presence of a cause of the increased student mobility? Perhaps. If leering male examiner, but passing my driving test does these were the only relevant factors in the situation then not cause me to be young, good-looking and female. the conclusion might be safe, but, of course, it is not; It is often dangerous to say that such-and-such is the real situation includes far more factors. Mill’s definitely the cause of something, or that such-and-such method is an over-s implification in the empirical world. is definitely the effect of something. Causation in the Mill’s method of difference: Let us say that different human sciences is much more tentative, and may be regions of a country have several reforms taking place, probabilistic rather than deterministic. Hume’s (2000) designed to increase student mobility (Table 6.2). One own rules for causation are: region did not have an NQF, and this is the only factor where there is no increased student mobility. Is it safe OO contiguity (of space and time) (the cause is contigu- to conclude that the NQF is the cause of the increased ous with the effect); student mobility? Perhaps. As before, if these were the only relevant factors in the situation then the conclu- OO priority/succession (the cause precedes the effect); sion might be safe, but in the ‘real’ world of multiple OO constant conjunction (the coupling of one event and factors, it is not. Mill’s method of agreement and difference: This its successor are found to recur repeatedly); applies both the preceding methods (Table 6.3) OO necessary connection (which is learned from experi- together. Here the method suggests that the NQF may be the cause of increased student mobility: ence, habit and custom rather than from deductive, logical, necessary proof ). 1 It could not be ‘increased educational financing’, as it is present where there is no ‘increased student One can detect correlation in Hume’s ideas rather than mobility’ in region 1. actual causation. He argues that causation is inferred, inductively, by humans rather than being an objective matter. To try to gain some purchase on causality, Mill TABLE 6.1 MILL’S METHOD OF AGREEMENT Region Increased Curriculum More places for Introducing the Increased student educational reforms vocational NQF mobility financing training Region 1 Region 2 No No Region 3 No No Region 4 No No 89
the context of educational research TABLE 6.2 MILL’S METHOD OF DIFFERENCE Region Increased Curriculum More places for Introducing the Increased student educational reforms vocational NQF mobility financing training No No Region 1 Region 2 Region 3 Region 4 TABLE 6.3 MILL’S METHOD OF AGREEMENT AND DIFFERENCE Region Increased Curriculum More places for Introducing the Increased student educational reforms vocational NQF mobility financing training No Region 1 No No Region 2 Region 3 No Region 4 No 2 It could not be ‘curriculum reforms’, as it is absent the effect. Is it safe to conclude that the NQF is the where there is ‘increased student mobility’. cause of the increased student mobility? Perhaps. Again, if these were the only relevant factors in the sit- 3 It could not be ‘more places for vocational training’, uation then the conclusion might be safe, but in the as it is absent where there is ‘increased student ‘real’ world, it is not. mobility’. Mill’s method of residues: If one is able to remove (e.g. control out) all the factors but one that may be 4 This leaves NQF, which is the only item which is causing all the effects but one, then the remaining remaining when (1) to (3) above are taken into factor is the cause of the remaining effect (Table 6.5). account. Here, we have a range of possible causes and effects; we see that all the factors except ‘introducing As with the two previous cases, is it safe to conclude the NQF ’ are causing all the effects (those in the top that the NQF is the cause of the increased student row of Table 6.5) except one (‘increased student mobil- mobility? Perhaps. Again, if these were the only rele- ity’). Hence we hold that it is ‘introducing the NQF ’ vant factors in the situation then the conclusion might which is the cause of ‘increased student mobility’. Is it be safe, but in the ‘real’ world, it is not. safe to conclude that the NQF is the cause of the Mill’s method of concomitant variation: If, across increased student mobility? Perhaps. As with all the several factors, one finds that the property (e.g. the previous four methods from Mill, if these were the only amount) of variation in Factor (A) is similar to, or the relevant factors in the situation then the conclusion same as, the amount of variation in the effect (Factor might be safe, but in the ‘real’ world, it is not. (B)), whilst such common variation is not demonstrated Mill’s methods are only as powerful as the factors in other independent variables, then it may be reason– included, and, in their search for the single cause, may able to infer that Factor (A) is the cause of Factor (B) overlook the interplay of myriad causes in producing (Table 6.4, matching the row entry for each cause with the effect and, indeed, assume a deterministic rather the effects on the same row). Table 6.4 indicates that than probabilistic view of causation (Kincaid, 2009). the only concomitant variation between the independ- Further, Mill’s approach to establishing causality in this ent and dependent variable (‘increased student mobil- instance operates at a single country level. Imagine, ity’) is for the variable ‘introducing the NQF ’, suggesting that it is the NQF which might be causing 90
The search for causation TABLE 6.4 MILL’S METHOD OF CONCOMITANT VARIATION Causes Effect Increased educational financing Increasing student mobility A little Moderate amount A lot A little Moderate amount A lot Curriculum reforms Increasing student mobility A little Moderate amount A lot A little Moderate amount A lot More places for vocational training Increasing student mobility A little Moderate amount A lot A little Moderate amount A lot Introducing the NQF Increasing student mobility A little Moderate amount A lot A little Moderate amount A lot TABLE 6.5 MILL’S METHOD OF RESIDUES Causes Effects More students in Greater vocational Clearer Increased student higher education mobility relevance progression in qualifications Increased educational financing Curriculum reforms More places for vocational training Introducing the NQF then, the additional complexity where his approach is Whilst statistical modellers may argue that it is pos- applied to more than one country at a time; the problem sible to operate controls and to utilize structural equa- expands exponentially in trying to detect the causal tion modelling, these necessarily simplify the complex, links between one factor and its putative effects. Nev- dynamic, changing scenario of the relationship between ertheless Mill’s view has substantially informed more the NQF, its instigator, its context and the multiple out- recent enterprises in working with causation, for comes operating in a situation. It is impossible to isolate example in the work of Ragin (1987, 2008). and control variables in a situation (cf. Cartwright 91
the context of educational research and Hardie, 2012); indeed the situation may rely on the 1997, p. 106; Salmon, 1998, pp. 5–8) (e.g. an explana- dynamic interplay of these variables. This frustrates tion may be wrong, or it may be giving the meaning of any easy attempts to utilize Mill’s (2006) five main something, or it may be indicating how to do some- approaches to establishing causality. thing). Nor is causation the same as giving a reason. Methods for establishing whether an effect is truly For example, I might take a day off from work, giving the result of a cause are beset with problems. Even if the reason that I am sick, but the real reason may be we observe outcomes, we cannot conclude with any simply that I am lazy or want to go shopping. certainty that these are unequivocally caused by the Nor is causation the same as prediction. Just because NQF in the example here. Even though probabilistic I observe something happening once does not mean I causality may replace deterministic causality, this does can predict that it will happen again (the problem of not attenuate the problem of deciding what is legitimate induction, see Chapter 1), as the conditions could be and illegitimate inference. different, or, indeed, even if the conditions were very The inferential, conjectural and probabilistic nature similar (as chaos theory tells us). I might be able to of much causation in educational research (rather than predict something even though my prediction is based being absolute, deductive and deterministic), coupled on the wrong identification of causes, for example, I with the fact that causation is frequently unobservable, can predict that there will be a storm because I have renders the study of causation challenging for educa- observed the barometric reading falling, but the fall in tional researchers. Indeed there is a danger in isolating the barometric reading does not cause the storm. For- and focusing on singular causes separately from other mally put, the two variables – the barometric reading contributing causes, contexts and conditions, and it is and the storm – are ‘screened off ’, separated and kept perhaps more fitting to regard causes as processes over apart from each other (Reichenbach, 1956; Salmon, time rather than single events. Further, in an intercon- 1998). They have correlational but no causal relation- nected world of multiple causes and causal nets, condi- ships to each other, and are both caused by a third tions and interactions may provide better accounts of factor – the drop in air pressure (see Figure 6.1). causation than linear determinism (Morrison, 2012). I might predict that a person’s hands might be large In unravelling causes and effects, the researcher is if she has large feet, but having large hands does not faced with the task of identifying what actually consti- cause her to have large feet – the cause might lie in a tutes a cause and what constitutes an effect. The contexts genetic predisposition to both. It is one thing to say that and conditions of an event are as important as the trigger a change in one variable (A) is associated with a change of an effect, and may be contributing causes. In the in another variable (B); it is an entirely different thing example earlier, my falling and breaking my arm was to say that a change in one variable (A) brings about a precipitated – triggered – by the ice on the path, but, change in another variable (B); and it is an entirely dif- without the presence of other contributing factors I might ferent thing again to say that a change in one variable not have fallen and I might not have broken my arm. The (B) is brought about by a change in another variable trigger of the effect may not be its sole cause but only (A), i.e. that it is caused by that change in variable A. the last cause in a causal chain, sequence of events, In attributing genuine causation, it is useful to series or network of conditions before the effect occurs, ‘screen off ’ unrelated dependent variables from the even though causes often raise the likelihood of their variables that are directly relevant to the situation being effects rather than guaranteeing them (Mellor, 1995, researched, i.e. to ensure that the effect of one variable pp. 69–70). Indeed, whilst probability often concerns is removed from the equation, to discount that variable identifying likelihood, the strongest probability is not or to control for the effects of other variables, for always the same as the strongest causation. I might think that putting pressure on a child to succeed has the strong- Atmospheric conditions Low barometric reading est possibility of causing her success, but the actual Storm cause might lie elsewhere, for example, the teacher might be very effective, the students might be highly FIGURE 6.1 Two unrelated factors caused by a third motivated or the examination might be very easy. factor 6.4 Causation, explanation, prediction and correlation The demonstration of causation is difficult. Causation is not the same as explanation (Clogg and Haritou, 92
The search for causation example, the presence of a third variable or several var- (motivation) in producing the effect, I might find that C iables, by partial correlations. It is also important in (motivation) is the strongest of the three causes. screening off to ensure that one variable is not deemed However, if I were to add a new variable (D) (an out- to have an influence on another when, in fact, this is not standing teacher helping the student), then it may be the case. This presumes that it is actually possible to that D is the overriding cause and that A, B and C are identify which factors to screen off from which. Pearl of equally low strength, or that A becomes the second (2009, pp. 423–7) indicates how this can be strongest factor. approached; in the case of multiple causality (or in the Statistical tools such as crosstabulations, correlation cases of over-d etermination, discussed below), this may and partial correlation, regression and multiple regres- not be possible. sion, and structural equation models (see Chapters 40–43) Screening off requires the ability to separate out can be used to assist here in the analysis of causation, causes, and this may be difficult to the point of impos- though it is often difficult to control direct, indirect, ante- sibility. However, in seeking to establish genuine cau- cedent, intervening and combined influences of variables sation, the researcher must consider controlling for the on outcomes (though statistical tools and graphical effects of additional variables, be they prior/exogenous methods can assist here; Pearl, 2009, pp. 423–7). variables or intervening/endogenous variables, as these In considering the control of variables, let us might exert a non-causal influence on the dependent examine, for example, the subject choices of secondary variables. school male and female students (Table 6.6). In conducting research that seeks to attribute causa- Here we can see overwhelmingly that males choose tion, it is important to control for the effects of vari physics far more than females, and females choose ables, i.e. to hold them constant so that fair attribution biology far more than males. The researcher wishes to of causality and the weight of causal variables can be know if the allocation of certain teachers to teach the assessed (though relative weights of causes are, strictly secondary school science subjects affects the students’ speaking, superfluous in discussing causation; they are choice (i.e. whether it is the subject or the teacher, or questionable indicators of causation). Further, identify- some combination of these, that is causing the students ing the relative strengths of causes depends on the pres- to choose the subjects that they choose). The researcher ence or absence of other causes. For example, in introduces the third variable of the ‘teacher’ as a control looking at examination success (the effect), if my variable, with two values: Teacher A and Teacher B, and research confines itself to looking at the relative then partitions the data for males and females according strength of causes A (hours of study), B (IQ) and C to either Teacher A or Teacher B (see Table 6.7). TABLE 6.6 SCIENCE CHOICES OF SECONDARY SCHOOL MALES AND FEMALES Male Female Total Preference for physics 175 (55.1%) 87 (27.9%) 262 (41.6% of total) Preference for biology 143 (44.9%) 225 (72.1%) 368 (58.4% of total) Column total 318 (100%) 312 (100%) 630 Percentage of total 100% 50.5% 49.5% TABLE 6.7 SCIENCE CHOICES OF MALE AND FEMALE SECONDARY STUDENTS WITH TEACHER A OR B Males with Females with Males with Females with Total Teacher A Teacher A Teacher B Teacher B Preference for physics 86 (55.8%) 44 (28.4%) 89 (54.3%) 43 (27.4%) 262 (62.5% of total) Preference for biology 68 (44.2%) 111 (71.6%) 75 (45.7%) 114 (72.6%) 368 (37.5% of total) Column total 154 (100%) 155 (100%) 164 (100%) 157 (100%) 630 Percentage of total 24.4% 24.6% 26.1% 24.9% 100% 93
the context of educational research When the data are partitioned by teacher (Teacher A explains matters more clearly than Teacher B, and stu- and Teacher B) the researcher notes that the percent- dents prefer clear explanations, and so on. The point ages in each of the partial tables (one part of the table here is that, though one can deduce certain points from for Teacher A and the other part of the table for contingency tables and partial tables, they may not Teacher B) in Table 6.7 are very similar to the original actually indicate causality. The same principle for percentages of the root table (Table 6.6). She concludes holding variables constant, this time in correlational that whether Teacher A or Teacher B is teaching the research, is discussed in Chapter 40. class makes no appreciable difference to the choices In establishing causation, it is important to separate made by the students. The percentages in the new table covariance and correlation between two unrelated and (Table 6.7) replicate very closely those in the original non-interacting dependent variables due to a common (Table 6.6). The researcher concludes that the teacher cause from the interaction of dependent variables due involved is exerting no causal influence on the choice to the presence of a common cause (the examples of the of subjects by the secondary school males and females. barometer and the storm earlier). However, let us imagine that the partial tables had yielded different data (Table 6.8). This time the results 6.5 Causal over-d etermination of the choices made by males and females who are with Teacher A and Teacher B are very different. The per- It is rare to find a single cause of a single effect. It more centages in the new table (Table 6.8) are very different often the case that there are several causes at work in a from those in the original (Table 6.6). This suggests to single situation and that these produce a multiplicity of the researcher that, in this instance, the teacher of the effects (see the discussion of evaluation in Chapter 5, class in question is making a causal difference to the and the work of Pawson (2013) in that chapter). For choices of science subject made by the students. example, why are so many young children well- However, this only tells us the ‘what’ of causation, behaved at school, when nobody has explicitly taught or, to be more precise, it only gives us an indication of them the hidden curriculum (Jackson, 1968) of rules, association and possible causation: it appears that the regulations, taking turns, sharing, being quiet, knowing teacher makes no difference in Table 6.7 but does make that the teacher is in charge and has all the power, a difference in Table 6.8. How this becomes a causal putting up with delay, denial and only being one out of matter is another question altogether: how does the many children who has to gain the teacher’s attention? teacher actually affect the males’ or females’ choices One answer is over-determination: many events, both of which science subject to follow. For example is it separately and in combination, lead to the same that: (a) Teacher A is male and Teacher B is female, outcome – the young child must do as she is told, and and students tend to prefer to be with teachers of their having a nice time at school depends on how effec- own gender; (b) Teacher A has a better reputation than tively she learns these rules and abides by them. Many Teacher B for helping students to pass public examina- causes; same effect: good behaviour. tions with high grades, and students are anxious to do Causal over-d etermination is ‘where a particular well; (c) Teacher A is more sympathetic than Teacher effect is the outcome of more than one cause, each of B, so that students can relate more easily to Teacher A, which, in itself, would have been sufficient to have pro- and so they choose Teacher A; (d) Teacher A has a duced the effect’ (Morrison, 2009, p. 51). A familiar better sense of humour than Teacher B, and students example is the issue of which bullet can be said to have prefer a good-h umoured teacher; (e) Teacher A killed a man, which causes his death (Horwich, 1993), TABLE 6.8 FURTHER SCIENCE CHOICES OF MALE AND FEMALE SECONDARY STUDENTS WITH TEACHER A OR B Boys with Girls with Boys with Girls with Total Teacher A Teacher A Teacher B Teacher B Preference for physics 133 (56.8%) 55 (23.5%) 39 (46.4%) 35 (44.8%) 262 (62.5% of total) Preference for biology 101 (43.2%) 179 (76.5%) 45 (53.6%) 43 (55.1%) 368 (37.5% of total) Column total 234 (100%) 234 (100%) 84 (100%) 78 (100%) 630 Percentage of total 37.1% 37.1% 13.3% 12.5% 100% 94
The search for causation if two bullets simultaneously strike a man’s head. onset of cancer, or when did smoking first bring about Either one bullet or the other caused the death (cf. the onset of cancer) or ends, and when an effect begins Mellor, 1995, p. 102). Let us say that, in a study of (e.g. I may continue smoking even after the early onset homework and its effect on mathematics performance, of lung cancer). I might hate studying mathematics at a rise in homework might produce a rise in students’ school but find it very attractive twenty years later; had mathematics performance. However, this is not all: my interest in mathematics been post-tested immedi- there may have been tremendous parental pressure on ately I left school, the result would have been lower the child to do well in mathematics, or the student than if I had been tested twenty years later. might have been promised a vast sum of money if her Where a cause begins and ends, where an effect mathematics performance increased, or the school begins and ends, when and how causes and effects might have exerted huge pressure on the student to should be measured, evaluated, ascertained and succeed, or the offer of a university place was contin- assessed, are often open questions, requiring educa- gent on a high mathematics score. The rise in mathe- tional researchers to clarify and justify their decisions matics performance may not have required all of the on timings in isolating and investigating causes and factors to have been present in order to bring about the effects. Quantitative data may be useful for identifying effect; any one of them could have produced the effect. the ‘what’ of causation – what causes an effect – but The effect is ‘over-d etermined’. One effect may have qualitative data are pre-e minently useful for identifying one or several causes. Whilst this is commonplace, it is the ‘how’ of causation – how causation actually works, important to note this in order to refute claims fre- the causal processes at work. quently made by protagonists of such-and-such an Consider, too, the reason for the ice patch being intervention in education that it alone improves per- present on the path in the earlier example, and my being formance; if only it were that simple! on the ice on the day in question. Maybe the local gov- ernment services had not properly cleared the path of ice 6.6 The timing and scope of the on that day, or maybe, as an ailing pensioner, I would cause and the effect normally be accompanied by a carer or an assistant whenever I went out, but on that day the person failed to Turn back to the earlier example of my falling on the turn up, so I was forced to go out on my own. Again, the ice and breaking my arm. Maybe I had a weakness in issue is not idle for researchers, for it requires them to my arm from an injury many years before, and maybe consider how widely or narrowly to cast their net in when I injured my arm years before I could not have terms of looking for causes (how far out and how far in). predicted that, many years later, I would have fallen on In determining what are relevant causes, the researcher ice and broken my arm. The issue is not idle for has to decide what to include and exclude from studies researchers, for it requires them to consider, in terms of of causation, for example, from the psychological to the temporality, what are relevant causes and what to social, from the micro to the macro, and to decide the include and exclude from studies of causation, how far direction and combination of such causes. back in time to go in establishing causes and how far The determination of a cause involves decisions on forward in time to go in establishing effects. how far back to go in a temporal causal chain or Just as the timing of causes may be unclear, so the network of events, and how wide or narrow to go in the timing of the effects of a cause may be unclear. Effects causal space (how many conditions and circumstances may be short-term only, delayed, instantaneous, imme- contribute to the causation at work in a given situation). diate, cumulative and long-term; indeed the full effects It may be difficult, if not impossible, to identify and of a cause may not be revealed in a single instance, as include all the causal antecedents in a piece of research. an effect may be a covering term for many effects that Here the concepts of necessary and sufficient condi- emerge over time (e.g. the onset and presenting of tions are raised, as is the importance of identifying the cancer has several stages; cancer is not a single event at causal trigger in a situation (the last cause in a causal one point in time). Temporality and causation are inti- chain or a linkage of several conditions). The striking mately connected but separate. of a match might cause it to flare, but that is not the The examples above also indicate that terms such as only factor to be taken into account. Whether it flares ‘cause’ and ‘effect’ are, in many cases, shorthand for depends on the abrasiveness of the striking surface, the many sub-c auses, sub-processes and sub-effects. materials used in the match, the dryness of the materi- Further, causes and effects may only reveal themselves als, the strength of the strike, the duration of the strike, over time, and, indeed, it may be difficult to indicate the presence of sufficient oxygen in the atmosphere, when a cause begins (which cigarette brought about the and so on. 95
the context of educational research There may be an infinite number of causes and seldom the case, as between A and B might be a huge effects, depending on how far back one goes in time number of intervening, prior or additional variables and and how wide one goes in terms of contexts. This processes operating, both exogenous and endogenous. presents a problem of where to establish the ‘cut-o ff ’ An exogenous variable is one whose values are point in identifying causes of an effect. Whilst this may determined outside the model (e.g. a structural equation be addressed through the identification of necessary model or a causal model) in which that variable is and sufficient conditions (Mackie, 1993) or the screen- being used, or which is considered not to be caused by ing off of some ‘ancestors’ (antecedents) (Pearl, 2009), another variable in the model, or which is extraneous to in fact this does little to attenuate the problem in social the model. sciences, as not only is it problematic to identify what An endogenous variable is one whose values or var- qualify as necessary or sufficient conditions, but these iations are explained by other variables within the will vary from context to context, and even though model, or which is caused by one or more variables there may be regularities of cause and effect from within the model. It is important to identify which context to context, there are also differences from causes mediate, and are mediated by, other causes. One context to context. also has to consider the role of moderators and media- The issue to be faced by researchers here is one of tor variables (the former influences the strength of a ‘boundary conditions’ and ‘circumscription’ (Pearl, relationship between two variables, and the latter 2009, p. 420): which factors we include or exclude can explains the relationship between two variables). How affect our judgements of causality. If, in a study of the researcher does this takes many forms, from theo- student performance, I only look at teacher behaviour retical modelling and testing of the model with data, to and its influence on student performance then I might eliciting from participants what are the causes. be led to believe that teacher behaviour is the cause of Whilst causation is not straightforward to demon- student performance, whereas if I only look at student strate, this is not to suggest that establishing causation motivation and its influence on student performance should not be attempted. There are regularities, likeli- then I might be led to believe that student motivation is hoods (probabilities) based on experience and previous the cause of student performance. Researchers rarely, if research, and similarities between situations and ever, include the universe of conditions, but rather only people. Indeed the similarities may be stronger than the a selection from that universe, and this might distort the differences. This suggests that establishing probabilis- judgements made about causation or where to look for tic causation or inferring causation, whilst complex and causation. Whilst it may not be possible to identify the daunting, may be possible for the researcher. universe of conditions, the researcher has to be aware The problem for the researcher is to decide which of the dangers of circularity, i.e. I am only interested in variables to include, as the identification and inclusion/ effect Y, so I only look at possible cause X, and then I exclusion of relevant variables in determining causation find, unsurprisingly, that X is the cause of Y, simply is a major difficulty in research. Causes, like effects, because I have not considered alternatives. It is impor- might often be better regarded in conjunction with tant to identify and justify the inclusion and exclusion other causes, circumstances and conditions rather than of variables in researching causation, to select the field in isolation (Morrison, 2009, 2012). Contextuality – the of focus sufficiently widely and to consider possible conditions in which the cause and effect take place – alternative explanations of cause and effect. and the careful identification and inclusion of all rele- vant causes, are key factors in identifying causation. 6.7 Causal direction, directness and indirectness 6.8 Establishing causation The problem of identifying causes and effects is further It is not easy to establish causation. For example, cau- compounded by consideration of direct and indirect sation may be present but unobserved and indeed unob- causes and effects and causal directions. Many models servable, particularly in the presence of stronger causes of causality often make too great a claim for unidirec- or impeding factors. I might take medication for a tionality rather than, for example, multi‑directionality headache but the headache becomes worse; this is not and mutual-d irectionality, or overlook clusters of to say that the medication has not worked, as the head- causes that act together in multiple directions (Morri- ache might have become even stronger without the son, 2012). medication. The effects of some causes may be masked Whilst the research may wish to identify the cause by the presence or strength of others, but nonetheless A that brings about the effect B, in practice this is causation may be occurring. 96
The search for causation Morrison (2009, p. 45) gives an example where, in A(Exam Increases 35% B the case of the causal relationship between smoking (A), heart disease (B) and exercise (C), smoking (A) is pressure) Increases 20% (Hard work) highly correlated with exercise (C): smokers exercise much more than non-smokers. Though smoking causes Reduces 75% heart disease, exercise actually is an even stronger pre- ventative measure that one can take against heart C disease. The corollary of this is that smoking prevents heart disease (cf. Hitchcock, 2002, p. 9). (Lack of The way in which the cause operates may also be self-confidence) unclear. There are many examples one can give here. Morrison (2009, p. 45), for instance, gives the example FIGURE 6.2 P ositive and negative causes on an of small class teaching. In one class operating with effect (1) small class teaching, the teacher in that class uses highly didactic, formal teaching with marked social regular relationship, e.g. a small cause has a regular distance between the teacher and the student (Factor small effect and a large cause has a regular large effect, A), and this is deemed to be an inhibitor of the benefi- or a small cause has a regular large effect and a large cial effects of small class teaching on students’ attain- cause has a regular small effect). However, seeking ment in mathematics: didactic teaching reduces linear relations between cause and effect might be mis- mathematics performance. However, the same highly guided, as the effects of causes might be non-linear didactic, formal class teaching (Factor A) significantly (e.g. a small or large cause may produce a large, small, raises the amount of pressure placed on the students to irregular or no effect), and it might be to deal with sin- achieve highly (Factor B), and this (Factor B) is known gular or a few causes and singular or a few effects, to be the overriding cause of any rise in students’ per- overlooking the interrelatedness and interactions of formance in mathematics, for example, in small classes multiple causes with each other, with multiple effects the teacher can monitor very closely the work of each and indeed with the multiple interactions of multiple child: high pressure raises mathematics performance. effects. Relationships and their analysis may be proba- Now, it could be argued that Factor A – an ostensibly bilistic, conditional and subjunctive rather than linear. inhibiting factor for the benefits of small class teaching Indeed nets and conditions of causation might be more – actually causes improvements in mathematics per- fitting descriptions of causation than causal lines or formance in the small class teaching situation. chains of events or factors (Morrison, 2012). Another example is where greater examination pres- One way of focusing on a causal explanation is to sure (A) on students increases their lack of self- examine regularities and then to consider rival explana- confidence (C), but it also increases the student’s hard tions of causes and rival hypotheses of these regulari- work (B), and hard work reduces the student’s lack of ties. The observation of regularities, however, is not self-c onfidence (C). In other words, the likelihood of essential to an understanding of causation, as all cases the effect of A on C may be lower than the effect of B may be different but no less causative. Further, the best on C, given A. Let us say that A increases the likeli- causal explanation is that which is founded on, and hood of C by 20 per cent, and A increases the likeli- draws from, the most comprehensive theory (e.g. that hood of B by 35 per cent, whilst B reduces the theory which embraces intentionality, agency, interac- likelihood of C by 75 per cent. In this instance increas- tion as well as structure, i.e. micro- and macro-factors), ing examination pressure increases the student’s self- that explains all the elements of the phenomenon, that confidence rather than reduces it (see Figure 6.2). fits the explanandum (that which is to be explained) The point here is that a cause might raise the likeli- and data more fully than rival theories, and which is hood of an effect, but it may also lower that likelihood, tested in contexts and with data other than those that and the presence of other conditions or causes affects have given rise to the theory and causal explanation. the likelihood of an effect of a cause. A diagrammatic Given the complexity of probabilistic causation, it representation of these examples is in Figure 6.2 (note would be invidious to suppose that a particular inter- that the length of the lines indicates the relative strength vention will necessarily bring about the intended effect. of the influence). A cause might lower the likelihood of Any cause or intervention is embedded in a web of an effect rather than increase it. other causes, contexts, conditions, circumstances and Many cause-a nd-effect models are premised on effects, and these can exert a mediating and altering linear relationships between cause and effect (i.e. a influence between the cause and its effect. 97
the context of educational research 6.9 The role of action narratives in could have been murder or suicide, consensual sex or causation rape, fidelity or infidelity. The causal accounts are given by a woodcutter, the bandit, the wife and the Statistics, both inferential and descriptive, can indicate spirit of the dead samurai speaking through a medium. powerful relationships. However, these do not neces- Each self‑serving account protects the honour of the sarily establish unequivocal, direct causation; they may teller and tries to exonerate each. At the end, there is no establish the ‘what’ of causation but not the ‘how’. I clear statement of whose version is correct; truth floun- might assume that A and B cause C, and that C causes ders in the quagmire of epistemology, perception and D; it is a causal model, and I might measure the effects motives. of A and B on C and the effect of C on D. However, Anthropologists, lawyers and social scientists (Roth causation here lies in the assumptions behind the model and Mehta, 2002) seized on the film as an example of rather than in the statistical tests of the model, and the the multilayered, contested truth of any situation or its causal assumptions that lie behind the model derive interpretation, coining the term ‘the Rashomon effect’ from theory rather than the model itself (see Chapter to describe an event or truth which is reported or 4). Statistics alone do not prove causation. Rather, cau- explained in contradictory terms, that gives differing sation is embodied in the theoretical underpinnings and and incompatible causal accounts of an effect: a death. assumptions that support the model, and the role of sta- There is more than one causal explanation at work in a tistics is to confirm, challenge, extend and refine these situation, and it is the task of the researcher to uncover underpinnings and assumptions. Behind statistics that these, and to examine the causation through the eyes of may illuminate causation lie theories and models, and it those imputing the causation. is in the construct validity of these that causation lies. It Action narratives and agency are important in is the mechanisms of causation – the how and why – accounting for causation and effects, and, because there that might concern researchers rather than solely is a multiplicity of action narratives and individual numbers and statistical explanations – the what. motivations in a situation, there are multiple pathways Many statistics rely on correlational analysis or on of causation rather than simple input–output models. In assumptions that pre-exist the statistics, i.e. the statis- understanding the processes of causation, the power of tics might only reinforce existing assumptions and qualitative data is immense, and, indeed, mixed models rather than identify actual causation. Even methods may be useful in establishing causation. sophisticated statistics such as structural equation mod- Causal explanations that dwell at the level of aggre- elling, multiple regression and multivariate analysis gate variables are incomplete, as behind them, and succumb to the charge of being no more powerful than feeding into them, lie individuals’ motives, values, the assumptions of causation underpinning them, and, goals and circumstances, and it is these that could be indeed, they often grossly simplify the number or range exerting the causal influence; hence a theory of individ- of causes in a situation, in the pursuit of a simple, clear ual motives may be required in understanding and and easily identifiable model. explaining causation. How is it that X causes Y; what is happening in X to For example, it is commonplace for a survey to ask cause Y? In short, what are the processes of causation? respondents to indicate their sex, but it is an entirely In order to understand this involves regarding causation different matter – even if different responses are given as dynamic rather than static, as a process rather than a by males and females to rating scales in a survey – to single event, and as involving motives, volitions, say that sex causes the differences in response. How, reasons, understandings, perceptions, individuality, actually, is sex a causal factor? Similarly, does social conditions and context, and the dynamic and emerging class actually cause an effect? It is only a constructed interplay of factors, more often than not over time. It is aggregate, a sum of individual characteristics (cf. here that qualitative data come into their own, for they Kincaid, 2009). ‘get inside the head’ of the actors in a situation. Further, between aggregate independent and depend- A neat example of this is what has come to be ent variables of cause and effect respectively lie a whole known as ‘the Rashomon effect’ in social sciences (e.g. range of causal processes, and these could be influenc- Roth and Mehta, 2002). It is over sixty-five years since ing the effect and, therefore, have to be taken into Kurosawa’s film Rashomon stunned audiences at the account in any causal explanation. How macro- Venice Film Festival. It provides four discrepant structural features from society actually enter into indi- witness accounts of the same event – an encounter viduals’ actions and interactions, and how individuals’ between a samurai, his wife and a bandit, that led to the actions and interactions determine social structures – the effect of the samurai’s death – in which the causes causal processes involved – need cautious elucidation, 98
The search for causation their current status often being opaque processes in a It is not enough to say that such-a nd-such a cause black-b ox, input–output model of causation. brings about such-a nd-such an effect, for, whilst it might establish the likelihood that the cause brings 6.10 Researching causes and about an effect or that an effect has been brought about effects by a cause, this does not tell the researcher how the cause brings about the effect or how the effect has been The researcher investigating the effects of a cause or brought about by the cause, i.e. what are the causal the causes of an effect has many questions to answer, processes at work in connecting the cause with the for example: effect and vice versa. If the research really wishes to investigate the processes of causation then this requires OO What is the causal connection between the cause detailed, in-d epth analysis of the connections between and the effect (how does the cause bring about the causes and effects. effect and how has the effect been brought about by For example, it is not enough to say that smoking the cause)? can cause cancer; what is required is to know how smoking can cause cancer – what happens between the OO What are the causal processes at work in the situa- inhalation of smoke and the presentation of cancer tion being investigated? cells. I might say that turning on a light switch causes the light bulb to shine, but this is inaccurate, as turning OO What constitutes the evidence of the causal on the switch completes a circuit of electricity and the connection? electricity causes a filament to heat up such that, when white hot, it emits light. OO On what basis will the inference of causality In education, it is not enough to say that increasing be made? the time spent on reading causes students’ reading to improve; that is naive. What might be required is to OO What constitutes the evidence that a cause is a cause know how and why the increase in time devoted to and that an effect is an effect? reading improves reading. This opens up many possible causes: motivation; concentration levels and spans; OO What constitutes the evidence that a cause is the interest level of the materials; empathy between the cause (and that there is not another cause) and that reader and the material; level of difficulty of the text; an effect is the effect (and that there is not another purposes of the reading (e.g. for pleasure, for informa- effect)? tion, for learning, for a test); reading abilities and skills in the reader; subject matter of the text; ambient noise; OO Is the research investigating the effects of a cause where, when and for how long the reading is done; (an interventionist strategy) or the cause of an effect prior discussion of, and preparation for, the reading (a post hoc investigation)? material; follow-u p to the reading; choice of reading materials; whether the reading is done individually or OO How will the research separate out a range of pos in groups; teacher help and support in the reading time; sible causes and effects, and how will decisions be relatedness of the reading to other activities; the nature, made to include and/or exclude possible causes and contents and timing of the pre-test and post-test; the effects? evidence of improvement (and improvement in which aspects of reading); and so on. OO What methodology will be chosen to examine the It can be seen in this example of reading that the effects of causes? simple input variable – increasing time for reading – may bring about an improvement in reading, but that OO What methodology will be chosen to examine the may only be one of several causes of the improvement, causes of effects? or an umbrella term, or may liberate a range of other causes to come into play, both direct and indirect OO What kind of data will establish probabilistic causes. Identifying the true cause(s) of an effect is causation? extremely difficult to pin down. Take, for example, the introduction of total quality OO When will the data be collected from which causa- management into schools. Here several interventions tion will be inferred? are introduced into a school for school improvement, and, at the next school inspection, the school is found As mentioned earlier, the timing of data collection is a critical feature in establishing causation and the effects of causes. Here the greater the need to establish causal processes, the closer and more frequent should be the data-collection points. Moreover, qualitative data could hold pre-e minence over quantitative methods in estab- lishing causation and causal processes. Further, longitu- dinal studies might yield accounts of causation that are more robust than cross-s ectional studies in which the necessary temporality of causation is built out in favour of the single instance of the data-c ollection point. 99
the context of educational research to have improved. The problem is trying to decide How, then, can the researcher proceed in trying to which intervention(s) has/have brought about the uncover causes and effects? A main principle underpin- improvement, or which combinations of interventions ning how some researchers operate here is through have worked, or which interventions were counter- control, isolating and controlling all the variables productive, and so on. It is akin to one going to the deemed to be at work in the situation. By such isolation doctor about a digestion problem; the doctor prescribes and control, one can then manipulate one or more vari- six medicines and the digestion problem goes. Which ables and see the difference that they make to the medicine(s) was/were responsible for the cure, and in effect. If all the variables in a situation are controlled, what combinations, or is it really the medicines that and one of these is manipulated, and that changes the have brought about the cure; were there other factors effect, then the researcher concludes that the effect is that brought about the cure; would the digestion caused by the variable that has been manipulated. problem have cured itself naturally over time? Moreover, if the research (e.g. an experiment) can be The researcher has to identify which cause (A) or repeated, or if further data (e.g. survey data) are added, combination of causes have brought about which effect and the same findings are discovered, then this might (B), both intended and unintended, or whether the sup- give added weight to the inferred cause-a nd-effect con- posed cause (A) brought about another effect (C) which, nection (though regularity – Hume’s (2000) ‘constant in turn, became the cause of the effect (B) in question, conjunction’ – is no requirement for causation to be and whether the effect (B) is really the consequence of demonstrated). This assumes that one has identified, the supposed cause(s) (A), and not the consequence of isolated and controlled all the relevant variables, but, as something else. What looks like being a simple cause- the earlier part of this chapter has suggested, this may and-effect actually explodes into a multiplicity of causes be impossible. and effects (Figure 6.3) (cf. Morrison, 2009, p. 124). One way in which the problem of isolation and control of variables is addressed is through randomiza- ABC MODELLING tion – a key feature of the ‘true’ experiment (see Chapter 20). For example, random allocation of indi- CAUSES AND viduals to a control group or an experimental group is a widely used means of allowing for the many uncon- ? EFFECTS trolled variables that are part of the make-u p of the AC groups in question (Schneider et al., 2007). It adopts the ceteris paribus condition (all other things being B equal) that assumes that these many other variables are evenly distributed across the groups, such that there is AC no need to control for them. This is a bold and perhaps B dangerous assumption to make, not least as chaos and C complexity theory tell us that small changes and differ- ences can bring about major differences in outcome. ?? and/or A Whilst control is one prime means of trying to estab- lish causation, it does raise several problems of the pos- B sibility, acceptability or manageability of isolating and controlling variables, of disturbing and distorting the AC real work of the participants, and of operating an unde- sirable – even unethical – control and manipulation of ?A?C people. This is the world in which the researcher is king or queen and the participants are subjects – subjected to ?? A ?? ?? C control and manipulation. On the one hand the claim is made that the research is ‘objective’, ‘clean’ (i.e. not ?? B ?? C affected by the particular factors within each partici- pant), laboratory-b ased and not prone to bias; on the ?? ?? other hand it is a manipulative and perhaps unrealistic DC attempt to control a world that cannot in truth be con- trolled. Are there alternatives? ?? ?? A major alternative is one that keeps the ‘real’ world of participants as undisturbed as possible, avoids the D B C D � known cause, but not included DC ?? = Unknown cause FIGURE 6.3 Positive and negative causes on an effect (2) 100
The search for causation researcher controlling the situation, and uses qualitative On the other hand, the researcher can proceed along data to investigate causation. Here observational, inter- an entirely different track, using qualitative research to view and ethnographic methods come to the fore, and really understand the causal processes at work in a situ- these are very powerful in addressing the processes of ation and in the minds of the participants in that situ causation and in establishing the causes of an effect as ation – the ‘how’ and ‘why’ of causation. recounted by the participants or the observers them- Determining the effects of causes is often under- selves. These methods deliberately ‘get inside the taken using an interventionist strategy in educational heads’ of individuals and groups, as well as including research, installing an intervention either to test a the researcher’s own views, identifying and reporting hypothesized causal influence or a causal model, or causation in their terms. They provide considerable because it is already known that it may exert a causal authenticity to the causal accounts given or compile a influence on effects, i.e. manipulating variables in order sufficiently detailed account of a situation for the to produce effects. (Of course, a non-intervention may researcher to make informed comments on the work- also be a cause, for example, I may cause a plant to die ings of causation in the situation under investigation. by not watering it, i.e. by doing nothing.) Further, it is often the participants themselves who Manipulation takes many forms, including: identify what are the causes of effects in the situations being investigated (though the researcher would need OO action research (discussed in Chapter 22), but this to be assured that these are genuine, as participants may may raise questions of rigour brought about by a have reasons for not disclosing the real causes or lack of controls and a lack of external checks such motives in a situation or, indeed, may be mistaken). that the attribution of causation may be misplaced; These two approaches are not mutually exclusive in a piece of research, and, as Chapter 2 has indicated, OO a range of experimental approaches (discussed in there is an advantage in adopting a mixed methods Chapter 20), which assume, perhaps correctly or approach, or, indeed, in a mixed methodology incorrectly, acceptably or unacceptably, that vari approach, in which positivist and experimental ables and people can be isolated, controlled and approaches might yield accounts of the ‘what’ of cau- manipulated; and sation – which variables are operating to produce an effect – whilst an interpretive approach might be used OO participant observation in qualitative research. to yield data on the ‘how’ and ‘why’ of causation – how the causal processes are actually working. In addressing these approaches, however, serious atten- Researchers examining causes and effects have to tion has to be paid to a range of issues: decide whether they are researching the effects of causes (e.g. in which they introduce an intervention and OO the context of the intervention and the power of the see what happens as a consequence) or the causes of situation could affect the outcomes and behaviours effects (e.g. backtracking from an observed situation to of participants (the Hawthorne effect or the Lucifer try to discover its causes). These are discussed below. effect (Zimbardo, 2007a)); 6.11 Researching the effects of OO the same causes do not always produce the same causes effects, even with the same people; In trying to investigate the effects of one or more causes, OO inappropriate timing of the pre-test and post-test the researcher can commence with a theory of causality measurements of effects could undermine the relia- operating in a situation (e.g. bringing pressure to bear on bility of the statement of the effects of the cause; students causes them to work harder, or dropping out of school reduces income at age 50 by a factor of five, or OO there are problems of accuracy and reliability, as improving self-e steem improves creativity), operational- groups and individuals cannot both be in a group ize it, and then test it, eliminate rival theories and expla- that is and is not receiving an intervention (Hol- nations using data other than those which gave rise to the land’s (1986, p. 947) ‘fundamental problem of explanation, and then proceed to the drawing and delim- causal inference’, which may not be sufficiently iting of conclusions. The use of continuous rather than attenuated by randomization) (see Chapter 20); categorical variables might be more effective in estab- lishing the nature and extent of causation as they indicate OO process variables and factors, and not only input vari- the magnitude of causes and effects. ables, as these feature in understanding causation; OO the characteristics, personae and specific individual features of participants and their agency, as these influence interventions and their effects. Experimental techniques, particularly randomized con- trolled trials (RCT), have some potency in establishing 101
the context of educational research causation, and it is here that the identification, isolation intervention other than those in which the researcher and control of independent variables is undertaken, might be initially interested. For example, a researcher manipulating one independent variable to see if it might find that pressuring students to learn improves makes a difference to the outcome. The other variables their mathematics scores but leads to an enduring are held constant and, if a change of outcome is dislike of mathematics. found by manipulating the one independent variable, In the context of moves towards judging ‘what then the change can be attributed to that independent works’, deciding ‘what works’ is as much a matter of variable (it becomes the cause), as the other variables values and judgement as it is of empirical outcomes of have been held constant, i.e. their influence has been causation. Success is a value judgement, not simply a ruled out. measure or a matter of performance. Judging ‘what In experimental approaches, randomization is an works’ in terms of cause and effect is an incomplete important element in determining causation in order to analysis of the situation under investigation. A more overcome the myriad range of variables present in, and fitting question should be ‘what works for whom, under operating in, participants (the ceteris paribus condition what conditions, according to what criteria, with what discussed earlier), to overcome within-g roup and ethical justifiability, and with what consequences for between-group differences (cf. Fisher’s The Design of participants?’ Experiments (1966)). RCTs and experiments (see A range of issues in judging the reliability and valid- Chapter 20) are an example of interventionist ity of experimental approaches in establishing causa- approaches that seek to establish the effects of causes tion includes the acceptability of laboratory experiments by introducing one or more interventions into a situa- that are divorced from the ‘real world’ of multiple tion and observing the outcomes of these under control- human behaviours and actions. Here field experiments led conditions. and natural experiments (see Chapter 20) may attenuate However, RCTs are often not possible in education the difficulties posed by laboratory experiments, though and, indeed, are not immune to criticism. For example, these, too, may also create their own problems of relia- the assumptions on which they are founded may be bility and validity. suspect (e.g. over-simplifying the variables at work in a As an alternative to action research and experimen- situation, and overriding the influence of mediating or tal methods in determining effects from causes, obser- process variables). They may have limited generaliza- vational approaches can be used, employing both bility (Cartwright and Hardie, 2012) and the measures participant and non‑participant approaches (see Chapter used in RCTs focus on average results rather than out- 26). Whilst these can catch human intentionality, liers or important sub-s ample differences. They fre- agency and perceptions of causality and events more quently do not establish the causal processes or causal fully than experimental methods, nevertheless they chains that obtain in the situation. They neglect partici- encounter the same difficulty as action research and pants’ motives and motivations. They neglect the experiments, as they, too, have to provide accounts of context in which the action is located, and they might causal processes and causal chains. Further, in address- neglect the moral agency of participants and the ethics ing intentionality and agency in causal processes and of researchers. Indeed context can exert a more power- chains, it is also possible that, whilst perceptions might ful causal force than the initial causal intervention, as be correct, they might also be fallacious, partial, incom- evidenced in the examples of the Stanford Prison plete, selective, blind and misinformed. I might think Experiment and the Milgram experiments on obedience that there is a mouse in the room (a cause), and act on (see Chapters 7 and 30). the basis of this (an effect), but, in fact, there may be Caution must be exercised in supposing that RCTs, no mouse at all in the room. for some people the epitome of causal manipulation in Interventionist approaches, and the determination of the determination of the effects of causes, will yield the effects of causes, risk mixing perception with fact, sufficient evidence of causation, as these overlook the and, regardless of evidence, human inclinations may be significance of context and conditions, of processes, of to judge data and situations on the basis of personal human intentionality, motives and agency, over- perceptions and opinions that, indeed, may fly in the determination etc., in short, of the contiguous causal face of evidence (the ‘base rate fallacy’; Morrison, connections between the intervention and its putative 2009, pp. 170–1; see also Kahneman, 2012). This is effects. Indeed even the issue of when and whether only one source of unreliability, and it is important to an effect has an effect (short-term to long-term, consider carefully what actually are the effects of immediate or delayed) is problematic, and attention had causes rather than jumping to statements of causation to be given to effects that have been caused by the based on premature evidence of connections. 102
The search for causation 6.12 Researching the causes of b At the same time, class differentials in educational effects attainment have remained stubbornly stable and resistant to change, i.e. though students from all Determining the causes of effects is even more provi- classes have participated in expanded education, sional, tentative and inferential than determining the class origins and their relationship to the likelihood effects of causes, as data are incomplete and backtrack- of them staying on in education or entering higher ing along causal chains and/or searching within causal education has only reduced slightly, if at all, and nets is difficult, as it requires a search for clues and this applies to most societies. testing rival hypotheses about causation. It is possible to generate a huge number of potential causes of He is establishing social regularities that any causal and observed effects, and the problem is in deciding which theoretical account should seek to explain: the creation, one(s) is/are correct. Morrison (2009) suggests that one persistence and continued existence of class stratifica- approach which can be adopted in tracing causes from tion in modern societies, and the continuing class- effects is ex post facto research (see Chapter 20), but it relatedness of educational inequality and life chances poses challenges in the sometime inability to control (p. 24). and manipulate independent variables or to establish randomization in the sample. Another approach is to Stage 2: Set out possible theoretical adopt a seven-stage process of tracing causes from foundations for the investigation effect, thus: Goldthorpe’s work is premised on the view that theo- Stage 1: Establish exactly what has to be explained. ries are necessary to provide explanatory foundations Stage 2: Set out possible theoretical foundations for for how established regularities come to be as they are (2007, p. 21). He initially suggests four theoretical the investigation. foundations: Marxist theory, liberal theory, cultural Stage 3: Examine, evaluate and eliminate rival theoret- theory and rational choice theory. ical foundations, selecting the most fitting. Stage 3: Examine, evaluate and eliminate Stage 4: Hypothesize a causal explanation on the basis rival theoretical foundations of the best theoretical foundation. For several reasons which he gives (2007, pp. 22–34), Stage 5: Set out the assumptions underlying the causal Goldthorpe rejects the first three of these and argues that rational choice theory provides a fitting theoretical explanation. foundation for his investigation of the causes of the Stage 6: Test the causal hypotheses empirically. effects observed (pp. 34–41). True to rational action Stage 7: Draw conclusions based on the test. theory, Goldthorpe places emphasis on aspirations, in particular noting their relative rather than their absolute A worked example is provided here, from Goldthorpe status, that is to say, aspirations are relative to class (2007). Goldthorpe seeks to explain the causes of ‘per- position, as working-c lass aspirations may not be the sistent differentials in educational attainment’ despite same as those of other classes (p. 31). Different social increased educational expansion, provision and uptake classes have different levels and kinds of aspiration, across the class structure (p. 21), i.e. in the context of influenced – as rational action theory suggests – by the increased educational opportunity and its putative constraints under which they operate, and the perceived weakening influence on class-b ased determination of costs and benefits that obtain when making decisions life chances. He proceeds in the seven stages indicated (p. 32). Taking relative rather than absolute views of above. Only after that test does he provide a causal aspiration enables accounts to be given that include the explanation for his observed effects. fact of increased provision of, and participation in, edu- cation by students from all social classes, i.e. class dif- Stage 1: Establish what it is that has to be ferentials have not widened as education provision and explained participation have widened. Goldthorpe (2007, p. 32) suggests that cultural First, Goldthorpe observes some ‘regularities’ (effects) theory may account for what Boudon (1973) terms (2007, p. 45): ‘primary effects’, i.e. initial levels of achievement and ability in the early stages of schooling. However, he a In all economically advanced societies there has is more concerned with Boudon’s ‘secondary effects’, been an expansion over time of education provision and in the numbers of students staying on in full- time education beyond the minimum schooling age (e.g. going into higher education). 103
the context of educational research i.e. those effects which come into play when children with manual labourers more prone to unemployment reach ‘branching points’ (transition points, e.g. from than professional or managerial workers, i.e. the costs primary to secondary schooling, from secondary educa- of education are still a factor for less advantaged fami- tion to university) (p. 32) and which have increasingly lies, particularly at the end of the period of compulsory powerful effects as one progresses through schooling. schooling. At the time when their children come to the ‘Secondary effects’ take account of the aspirations and end of compulsory schooling, the income of manual values that children and their parents hold for educa- workers will already have peaked (e.g. when they are tion, success and life options, i.e. the intentionality and in their forties), whereas for professional and manage- agency of rational action theory in a way that ‘primary rial workers it will still be rising, i.e. costs are more of effects’ do not. Goldthorpe notes that, at each succes- a problem for manual workers than for professional and sive ‘branching point’ (p. 32), children from more managerial workers, i.e. the costs of higher education advantaged backgrounds remain in the educational relative to income, and the consequent effects on family system and those from less advantaged backgrounds lifestyle if families are having to finance higher educa- either leave school or choose courses that lead to lower tion, are much higher for manual workers. This qualifications (hence reducing their opportunities for increased proportion of family income to be spent on yet further education). education for less advantaged families is coupled with Goldthorpe (2007, p. 33) argues that more ambitious the fact that, if children from these families are to options may be regarded less favourably by those from succeed, then they need even more ambition than their less advantaged class backgrounds as they involve: (a) professional and managerial class counterparts, i.e. they greater risk of failure; (b) greater cost; and (c) relatively are at a potential double disadvantage, i.e. relative less benefit. In other words, the level of aspiration may advantage and disadvantage are not disturbed, a feature vary according to class and the associated levels of on which liberal theory is silent (p. 36). assessed cost and risk by members of different classes, Goldthorpe makes the point that class position con- and children from less advantaged backgrounds have to ditions educational decisions made by members of dif- be more ambitious than those from more advantaged ferent classes. These different class positions influence backgrounds if they are to meet the aspirations and different evaluations of the costs and benefits of educa- success levels of those from more advantaged back- tion, and these are socially reproductive, i.e. the social grounds. Class origins influence risk assessment, cost class position is undisturbed. assessment and benefit assessment – all aspects that are Another element of his argument concerns risk aver- embraced in rational action theory. These determine the sion. His view is that a major concern of members of choices made by children and their parents. different classes is to minimize their risk of downward class mobility, and to maximize their chances for Stage 4: Hypothesize a causal explanation upward class mobility or, at least, to maintain their on the basis of the best theoretical existing class location (p. 37). This exerts greater pres- foundation sure on the already-a dvantaged classes (e.g. the sal- ariat) to have their children complete higher education Goldthorpe (2007, p. 34) argues that class differentials (in order to preserve intergenerational class stability) in educational attainment have persisted because, even than it does on the children from less advantaged though there has been expansion and reform of educa- classes (e.g. the waged). It costs more for the children tion, and even though the overall costs and benefits that of the advantaged classes to preserve their class posi- are associated with having more ambitious options have tion than it does for children of the less advantaged encouraged their take-u p, in practice there has been classes to preserve theirs. little concurrent change in the ‘relativities between With regard to families in the less advantaged class-specific balances’: different classes view the classes, Goldthorpe (p. 38) suggests that they regard costs, risks and benefits differently (p. 34). This is his higher education much more guardedly. Not only does working hypothesis in trying to establish cause from it cost less for them to maintain their class position, but effect. it costs relatively more to achieve upward class mobil- ity; their best options might be for vocational educa- Stage 5: Set out the assumptions underlying tion, as it is cheaper and gives a strong guarantee of not the causal explanation moving downwards in class situation (e.g. to be unem- ployed or unskilled). Goldthorpe tests his theory by drawing initial attention Further, for children in this class, the costs (and to the ongoing income differentials between classes; likelihood) of failure in higher education could be indeed he argues that they have widened (2007, p. 35), 104
The search for causation proportionately greater than those for children from OO actual choices made by members of different more advantaged families. For example, in terms of: classes; the relative costs of the higher education; lost earning time; lost opportunity to follow a vocational route in OO fear of downward social mobility (pp. 53–4); which they have greater likelihood of being successful OO the need to preserve, or improve on, intergenera- (p. 38); loss of social solidarity if working-class chil- dren pursue higher education, the consequences of tional mobility (pp. 53–4); which may be to remove them from their class origin OO financial costs (p. 56). and community (pp. 38–9). These factors combine to suggest that children and families from less advantaged He indicates that students from lower socio-e conomic backgrounds will require a greater assurance, or expec- groups either cannot afford, or cannot afford to take tation, of success in higher education before commit- risks in, higher education, and he identifies three clus- ting themselves to it than is the case for children and ters of possible explanations of persistence of class dif- families from more advantaged backgrounds (p. 68). ferentials in educational attainment, including (but not Goldthorpe then offers his causal explanation of the limited to): effects observed: the persistence of class differentials in Cluster 1: Differences in aspirations and decisions educational attainment despite expansion of educational are caused by perceptions of costs: (a) loss of earnings provision and participation (p. 39): during study time (a bigger drawback for families and students from low-income households than for those 1 Class differentials in the uptake of more ambitious from privileged backgrounds); (b) students from low- educational options remain because the conditions income households have to work harder than privileged also remain in which the perceived costs and bene- students in order to compete with them; (c) students fits of these options operate, and these lead to chil- from low-income households must have greater ambi- dren from less advantaged families generally tion than privileged students in order to be successful requiring a greater assurance of success than chil- in a higher social class; (d) the financial costs of higher dren from more advantaged families before they (the education, proportional to income, are higher for less former) pursue more ambitious educational options; advantaged students than for more advantaged students and families. 2 There is a rational explanation for the persistence of Cluster 2: Differences in aspiration and decisions these different considerations of ambitious options are caused by relative risk aversion: (a) the risk of by class over time, which is rooted in class-b ased failure in higher education is greater for students from conditions. disadvantaged classes; (b) the risk of loss of further educational opportunities if failure ensues or incorrect These are the two main hypotheses that he seeks to test. options are followed is greater for students from disad- vantaged classes than for students from more privileged Stage 6: Test the hypotheses empirically classes; (c) the risk of loss of social solidarity is greater for students from working-c lass groups than for stu- Goldthorpe (2007) then proceeds to test his two hypoth- dents from more privileged classes; (d) less advantaged eses (pp. 39–44, 53–6, and his chapters 3 and 4), adduc- students must have greater ambition than privileged ing evidence concerning several factors, for example: students in order to be successful in higher social classes. OO the greater sensitivity of working-class families to Cluster 3: Differences in aspiration and decisions the chances of success and failure in comparison to are caused by perceptions of relative benefit: (a) the middle-c lass families (p. 40); opportunity for upward social mobility through higher education is an attraction for students from lower-class OO different levels of ambition in working-class and backgrounds; (b) higher education is differentially nec- middle-c lass families (p. 40); essary for preferred or likely employment for those from privileged and less privileged groups. OO relative (class-b ased) risk aversion in decision making: for example, the risk of failure and/or of Stage 7: Draw conclusions based on closing options (pp. 55–6); the test OO the loss of forgone earnings (pp. 53–5); Goldthorpe (2007) indicates that class differentials OO expectations of success (pp. 55–6); have continued to affect the take-up of educational OO evaluation of the potential benefits, value and utility options. He finds that class differentials in terms of the take-u p of more ambitious educational options have of higher education (pp. 38–9); OO influences on choices and decision making in differ- ent classes (his chapter 3); 105
the context of educational research been maintained because so too have the conditions in OO identifying several causal chains, mechanisms and which the perceived costs and benefits of these options processes in a situation; lead to children from less advantaged families requir- ing, on average, a greater assurance of success than OO combining micro- and macro-levels of analysis; their more advantaged counterparts before they decide OO addressing both agency and structure; to pursue such options. There are class differences in OO underpinning the data analysis and causal explana- terms of relative ambition, risk aversion, perceived costs and benefits, amounts of effort required, assur- tion with theory; ances of success (and the significance of this), fear of OO using different kinds of ex post facto analysis; downward social mobility, income, occupational OO using correlational and causal-c omparative, criterion choices and the need for qualifications. He concludes that the results of empirical tests group analysis; support his explanation of the factors of relative risk OO ensuring matching of groups in samples and that aversion and fear of downward social mobility exerting causal power on educational decision making which, in similar causes apply to both groups; turn, lead to class differentials in educational attain- OO adopting the seven-s tage process set out above, of ment being maintained (p. 99). Goldthorpe argues that this hypothesis is better sup- generation, testing and elimination of hypotheses ported than alternative hypotheses (e.g. educational and rival hypotheses; choices being predetermined by culture, class identity OO ensuring clarity on the direction of causation; and the class structure). OO using empirical data to test the causal explanation; This lengthy example here offers a robust account OO identifying which is cause and which is effect, and/ of how to track backwards from an effect to a cause or which effect then, subsequently, becomes a and how to evaluate the likelihood that the putative cause; cause of the effect actually is the cause of that effect. In OO avoiding the problem of over-s elective data; summary, for researchers seeking to establish the OO ensuring that the data fairly represent the phenome- causes of effects, the task has several aspects: non under investigation; OO recognizing that cause and effect may be blurred; OO Indicate what needs to be done to test the theory and OO accepting that effects may become causes in a cycli- to falsify it. cal sequence of causation; OO seeking out and recognizing over-d etermination at OO Identify the kinds of data required for the theory to work in causal accounts; be tested. OO keeping separate the explanans (the explanation) from the explanandum (that which is to be OO Identify the actual data required to test the theory. explained); OO Identify the test conditions and criteria. OO ensuring that alternative theories and causal expla- OO Construct the empirical test. nations are explored and tested; OO Consider the use of primary and secondary data. OO drawing conclusions based on the evidence, and the OO Consider using existing published evidence as part evidence alone. of the empirical test. In seeking to establish the causes of effects, there is a OO Ensure that action narratives and intentionality are need to review and test rival causal theories and to retain those with the greatest explanatory potential and included in causal accounts. which fit the evidence most comprehensively and securely. Testing rival hypotheses must be done with The fundamental problem in determining causes from data that are different from those that gave rise to the effects is the uncertainty that surrounds the status of the hypotheses, in order to avoid circularity. putative cause; it can only ever be the best to date, and The determination of causes from effects does not the researcher does not know if it is the best in absolute have the luxury afforded to causal manipulation in terms. One effect stems from many causes, and to try to determining effects of causes. Whilst this renders the unravel and support hypotheses about these may determination of causes from effects more intractable, present immense difficulties for the researcher. Morri- nevertheless this is not to say that it cannot be son (2009, p. 204) suggests that there are several ways attempted or achieved, only that it is difficult. in which causes may be inferred from effects: Morrison (2009) argues that, in seeking to identify the causes of effects, there is a need for a theoretical OO recognizing that a high level of detail may be required foundation to inform causal explanation. Possible in order to establish causation: high granularity; causal explanations should be evaluated against rival theories and rival explanations, being operationalized 106
The search for causation in considerable detail (high granularity), and tested OO identifying the causal processes at work in determin- against data that are different from those that gave rise ing social and macro‑structures from the actions and to the causal explanation. Causal explanations should interaction of individuals (the micro‑worlds) and, link micro- and macro-factors, include agency and conversely, in determining the actions and interac- intentionality as well as structural constraints, and tions of individuals from the structures of society contain a level of detail that is sufficiently high in gran- and its institutions (the macro‑worlds), their ontolo- ularity to explain the phenomenon to be explained gies and epistemologies. without concealing or swamping the main points with detail overload, i.e. the researcher must be able to dis- The researcher has to decide whether the research is tinguish the wood from the trees. investigating the cause of an effect, the effect of a The companion website to the book presents a fully cause, or both, and when causation is demonstrated, worked example of working with a range of challenges given that absolute certainty is illusory. If one is inves- in causality, for example, counterfactuals, ‘before-a nd- tigating the effects of causes then the methodologies after’ comparisons, multiple causes, over-d etermination, and approaches to be used might include experiments, causal forks, preceding causes, causal links, causal action research, survey analysis, observational direction and what can and cannot be inferred about approaches or a combination of these (and indeed causality. We advise readers to look at that in-depth others). If one is investigating the causes of effects worked example. then, in the context of the likelihood of greater uncer- tainty than in establishing the causes of effects, one can 6.13 Conclusion employ numerical and qualitative data in backtracking from effects to causes and in testing hypothesized In approaching causal research, then, the researcher is causes of effects. In all of these approaches, this chapter faced with a range of challenges, including, for has suggested that probabilistic rather than determinis- example: tic causation is a more fitting description of the nature of the conclusions reached. It has suggested that, even OO focusing more on causal processes than input/ if it sounds simplistic at first, nevertheless it is both output/results models of causation; important yet difficult to establish what actually consti- tutes a cause and an effect. The chapter has suggested OO establishing causation other than through reduction that causal processes, with high granularity, are often and recombination of atomistic, individual items closer to identifying the operations of causes and and elements; effects and the links between them, and that here quali- tative data might hold pre-e minence in educational OO regarding causation as the understanding of the research. However, the chapter has also suggested that emergent history of a phenomenon or a whole; there is an important role for numerical approaches, for examining the ‘regularities’ that might be evidenced in OO investigating multiple and simultaneous causes and survey approaches, and in the isolation and control of their multiple and simultaneous effects in a multiply variables in experimental approaches. In short, the connected and networked world; chapter is arguing for the power of mixed methodolo- gies and mixed methods in investigating and establish- OO separating causation from predictability, and ing causation. drawing the boundaries of predictability for an None of the preceding discussion takes us very far understanding of the frequent uniqueness of a causal from the difficulty in actually defining causation. Is it, sequence, which may not be repeatable, i.e. living like time, space, existence, not defined in terms of pre- with uncertainty and unpredictability; viously defined concepts, hence is a ‘primitive concept’, irreducible to anything else? OO learning to work with causation in a situation in The companion website to the book provides addi- which randomness often ‘trumps’ causation (cf. tional material and PowerPoint slides for this chapter, Gorard, 2001a, p. 21); which list the structure of the chapter and then provide a summary of the key points in each of its sections. OO indicating the utility of understanding causation if it This resource can be found online at: www.routledge. has little subsequent predictive strength; com/cw/cohen. OO understanding how to investigate causation in holis- tic webs of connections, i.e. how is it possible to discover or demonstrate causation when looking at events holistically; OO understanding causation and causal processes in a multi‑causal, multi-e ffect, non‑linear and multiply connected world; 107
the context of educational research Companion Website The companion website to the book includes PowerPoint slides for this chapter, which list the structure of the chapter and then provide a summary of the key points in each of its sections. These resources can be found online at www.routledge.com/cw/cohen. 108
Part 2 Research design The planning of educational research is not an arbitrary ensure that the project is practicable. This part suggests matter; the research itself is an inescapably ethical several ways that researchers can approach the choice enterprise. Nor are the planning and conduct of educa- of a research project, and comments on the need for the tional research simply a matter of cranking out recipes project to be significant (and what this means), to con- and following them. On the contrary, educational sider its purposes and intended outcomes, feasibility, research is a deliberative and reflexive exercise. We research questions, literature review and overall design. place ethical issues at a very early point in the book to signal this. The research community and those using This part also provides an augmented chapter on the findings have a right to expect that research is con- sampling issues, with attention to statistical power. ducted rigorously, scrupulously and in an ethically Sampling, reliability and validity are key matters in defensible manner. This is thrown into sharp relief with research; without due attention to these the research the rise in online research, and we discuss this in a new could turn out to be worthless. Hence this part chapter. All this necessitates careful planning, and this addresses these issues in detail. These are complex part introduces some key design and planning issues. It matters, and we take readers through them systemati- contains chapters on how to choose a research project cally. The chapter on sensitive educational research is and a comprehensive set of considerations in the design included here, to underline the point that not only is the and planning of educational research, including ensur- very decision to conduct research a sensitive matter, ing that the research provides warrants for interpreting but that often access itself is difficult and sensitive, and data and drawing conclusions. This part includes this could be the major issue to be faced in planning another entirely new chapter on research questions and research. This part sets out a range of planning possi- hypotheses. The entire part contains a wide range of bilities so that the eventual selection of sampling proce- worked examples. dures, together with decisions on reliability and validity, are made on the basis of fitness for purpose, In designing research, we need to consider the issues and so that sensitivities in research are anticipated and of how to choose a research project, how to plan it, how addressed. to conduct a literature search and review, and how to 109
The ethics of educational CHAPTER 7 and social research 7.1 Introduction review boards, legislation, regulations and regulatory frameworks may raise issues for consideration and Ethics concerns that which is good and bad, right and provide advice for researchers on what to do and not to wrong. Ethical research concerns what researchers do. However, ethical issues are rarely as straightfor- ought and ought not to do in their research and research ward as rule‑following would suggest, and it is for indi- behaviour. A cursory glance at recent literature throws viduals to take responsibility for the decisions that they up a vast field of issues in considering ethics in educa- take on ethical matters and the actions connected with tional research, for example: those decisions (Brooks et al., 2014, p. 153). Ethical decisions are contextually situated – socially, OO informed consent; politically, institutionally, culturally, personally – and OO confidentiality and anonymity; each piece of research raises ethical issues and dilem- OO identification and non-traceability; mas for the researcher. Ethical norms vary in different OO non‑maleficence; parts of the world, and what is acceptable in a western OO beneficence and duty of care; culture may not apply elsewhere. Ethical issues are not OO responsibilities (for what and to whom); a once-a nd-for-a ll matter which can be decided before OO gaining access; the research commences or when the proposal is put to OO overt and covert research; an ethics committee, and then forgotten (cf. Brooks et OO disclosure and public versus private knowledge and al., 2014, p. 154); rather, they run throughout the entire research process. For example, Wax (1982, p. 42) spaces; makes the telling point that informed consent in many OO relationships and differential power relations in kinds of research is not a ‘one-s hot, once-a nd-for-a ll’ affair, but has to be continuously negotiated, particu- research; larly in qualitative, emergent research. Ethics are OO interests at stake in the research (in whose interests present at every turn, and we indicate key issues to be faced at each stage. the research is operating); What starts as being an apparently straightforward OO rights, permissions and protections; ethical matter quickly raises non‑straightforward ethical OO ownership and control of data; decisions for the researcher. Each research undertaking OO access to data (and its archiving); is an event sui generis, and the conduct of researchers OO the roles and power of research sponsors and cannot be, indeed should not be, forced into a procru- stean system of ethics. When it comes to the resolution commissioners; of a specific moral problem, each situation frequently OO sensitive research; offers a spectrum of possibilities. Ethics are ‘situated’, OO gender, age, colour, (dis)ability and ethnicity issues; i.e. they have to be interpreted in specific, local situ OO researching with children; ations (Simons and Usher, 2000). OO avoidance of selective, partisan and skewed data Each stage in the research sequence raises ethical issues. Sikes (2006) notes that ethics touch ‘researchers analysis; and their research choices, research topics, methodolo- OO value positions in data interpretation; gies and methods, and writing styles’ (p. 106). Ethical OO responsibilities to different parties; issues may arise from the nature of the research project OO being judgemental. itself; the context for the research; the procedures to be adopted (e.g. creating anxiety); methods of data col- Each of these, in turn, raises many questions and con- lection (e.g. covert observation); the nature of the siderations and we introduce them in this chapter. There are rarely easy, ‘black-a nd-white’ decisions on ethical matters. Rather, researchers must take informed decisions on a case-by-case basis. Codes of practice, ethical guidelines, ethics committees and institutional 111
Research design participants; the type of data collected (e.g. personal Ethical problems in educational research can often and sensitive information); what is to be done with the result from thoughtlessness, oversight or taking matters data (e.g. publishing in a manner that may cause partic- for granted. A student whose research is part of a ipants embarrassment or harm); and reporting the data course requirement and who is motivated wholly by (e.g. in a way that the participants will understand) self-interest, or academic researchers with professional (Oliver, 2003, p. 17). advancement in mind, may overlook the ‘oughts’ and How, then, can the researcher, particularly the ‘ought nots’. It is unethical for the researcher to be novice researcher, begin to address the scope of ethical incompetent in the area of research. Competence may issues? One way is to follow the stages of research, require training (Ticehurst and Veal, 2000, p. 55). from initial considerations to research planning, choice Indeed an ethical piece of research must demonstrate of topic, design, methodologies, data collection, data rigour and quality in the design, conduct, analysis and analysis, interpretation, to reporting and dissemination, reporting of the research (Morrison, 1996b). and this is how the chapter is organized. We review Kimmel (1988) has pointed out that it is important issues in the ethical field in the sequence in which they we recognize that the distinction between ethical and may be encountered in planning, conducting, reporting unethical behaviour is not dichotomous, even though and disseminating research: the normative code of prescribed (‘ought’) and pro- scribed (‘ought not’) behaviours, as represented by the OO ethical principles and the nature of ethics in educa- ethical standards of a profession, seem to imply that it tional research; is. Judgements about ethics lie on a continuum that ranges from the clearly ethical to the clearly unethical. OO sponsored research; The point here is that ethical principles are not abso- OO regulatory contexts of ethics: codes of practice, lute, generally speaking (though some may maintain otherwise), but must be interpreted in the light of the ethical review boards and ethics committees, legis- research context and of other values at stake. lation, ethical frameworks and guidelines; Whilst many of the issues addressed in this chapter OO choice of research topic and research design; concern procedural ethics, ethics concerns right and OO ethical dilemmas in planning research: informed wrong, good and bad, and so procedural ethics may not consent, non-m aleficence, beneficence and human be enough; one has to consider how the research pur- dignity, privacy, anonymity, confidentiality, betrayal poses, design, contents, methods, reporting and out- and deception; comes abide by ethical principles and practices. OO gaining access and acceptance into the research A deontological view of ethics concerns what one setting; has a duty or obligation to undertake, what ought to be OO power and position; done, i.e. there are universalizable ‘categorical impera- OO reciprocity; tives’ (Kant’s phrase) to behave in certain ways and not OO ethics in data analysis; to behave in other ways (prescriptive and prohibitive OO ethics in reporting and dissemination; respectively), which override the consequences of such OO responsibilities to sponsors, authors and the research actions. These are universalizable in the sense that it community. should apply to all persons, and categorical in that they must be obeyed without exception. This view involves These are intended to guide the reader through a maze treating people as ends in themselves – with equal of ethical concerns in educational research, and the respect, dignity and value – rather than as means (Howe foundations on which they are built. The chapter pro- and Moses, 1999, p. 22). As Brooks et al. (2014, p. 23) vides practical examples of ethics considerations. remark, the deontological view requires us to treat others as we would wish others to treat us, regardless 7.2 Ethical principles and the nature of their personal characteristics, status or backgrounds. of ethics in educational research This extends to considerations such as: do no harm and do prevent harm (non-m aleficence); do good (benefi- Ethics has been defined as ‘a matter of principled sensi- cence); be honest; be sincere; be grateful. We discuss tivity to the rights of others’ (Cavan, 1977, p. 810). Edu- these issues in detail later in the chapter. cational researchers must take into account the effects of By contrast, a consequentialist view of ethics con- the research on participants; they have a responsibility to cerns the outcomes of actions, for example, the utilitar- participants to act in such a way as to preserve their ian view that ethical behaviour is that which produces dignity as human beings. Ethical decisions are built on the greatest good (and happiness) for the greatest ethical principles, but different ethical principles may conflict with each other, and we explore this below. 112
The ethics of educational and social research number. This replaces the deontological emphasis on knowledge. Marshall and Rossman (2016, p. 51) note unexceptionable rules and what is always right with a that virtue ethics concern relationships, and researchers focus on consequences. In this view a costs–benefits have to consider their relationships with participants, analysis is considered, but this is problematical, as (a) stakeholders, sponsors, the research community, indi- it is unclear which costs and which benefits should be viduals, groups, the institution and so on – a complex- factored into the analysis, and it assumes that all costs ity of relationships (cf. Ary et al., 2002). and all benefits are of the same strength (Howe and These different views of ethics may not sit comfort- Moses, 1999, p. 23); and (b) there is disagreement on ably together; for example, a utilitarian view might what constitutes goodness and happiness, how it will be argue that a person who has a healthy body and healthy calculated and who decides. Brooks et al. (2014) note organs should be killed, and his organs used to save that rights-b ased ethics concern people’s liberty, and five or six lives of those who otherwise would die, these trump consequential ethics. whereas a virtue ethics viewpoint (Hammersley, 2009, From the consequentialist position, a major ethical p. 213) would argue that this is murder and cannot be dilemma is that which requires researchers to strike a justified. Another example of values clashing is where balance between the demands placed on them as pro- a deontological view might argue that a failing school fessional scientists in pursuit of truth, and the partici- should be closed, whereas a utilitarian view would pants’ rights and values potentially threatened by the argue against its closure because those 2,000 students research. This is known as the ‘costs/benefits ratio’, the would go to an even worse school. essence of which is outlined by Frankfort-N achmias A source of tension in considering ethics is that and Nachmias (1992) in Box 7.1. generated by the competing absolutist and relativist Deontological and consequentialist views of ethics positions. The absolutist view holds that clear, set concern actions and behaviour. By contrast, a third principles should guide the researchers in their work view is a virtue ethics basis which concerns people, and and that these should determine what ought and what in which one pursues what is good simply because it is ought not to be done (see Box 7.2). To have taken a what is expected of a good and right person: ‘what kind wholly absolutist stance, for example, in the case of of person we ought to be’ (Brooks et al., 2014, p. 24). the Stanford Prison Experiment (see Chapter 30), A virtuous person might possess characteristics includ- where the researchers studied interpersonal dynamics ing loyalty, integrity, respect, sincerity, modesty etc., in a simulated prison, would have meant that the i.e. virtues. Views of what the virtues are, and what the experiment should not have taken place at all or that it virtuous person is, may vary by time, place, culture, should have been terminated well before the sixth day. society etc. For the researcher, Hammersley and Tra- Indeed Zimbardo (1973), the author of the Stanford ianou (2012) identify key virtues of dedication, objec- Prison Experiment, has stated that the absolutist tivity and independence (respecting academic freedom, ethical position, in which it is unjustified to induce professionalism and minimizing negative influences on any human suffering, would bring about the end of the research) (pp. 46–51) in the disinterested pursuit of much research, regardless of its possible b enefits to Box 7.1 The costs/benefits ratio The costs/benefits ratio is a fundamental concept expressing the primary ethical dilemma in social research. In planning their proposed research, social scientists have to consider the likely social benefits of their endeavours against the personal costs to the individuals taking part. Possible benefits accruing from the research may take the form of crucial findings leading to significant advances in theoretical and applied knowledge. Failure to do the research may cost society the advantages of the research findings and ultimately the opportunity to improve the human condition. The costs to participants may include affronts to dignity, embarrassment, loss of trust in social relations, loss of autonomy and self-determination, and lowered self-e steem. On the other hand, the ben- efits to participants could take the form of satisfaction in having made a contribution to science and a greater personal understanding of the research area under scrutiny. The process of balancing benefits against possible costs is chiefly a subjective one and not at all easy. There are few or no absolutes and researchers have to make decisions about research content and procedures in accordance with professional and personal values. This costs/benefits ratio is the basic dilemma residual in a great deal of social research. Source: Adapted from Frankfort-N achmias and Nachmias (1992) 113
Research design Box 7.2 Absolute ethical principles in social research Ethics embody individual and communal codes of conduct based upon a set of explicit or implicit principles and which may be abstract and impersonal or concrete and personal. Ethics can be ‘absolute’ and ‘relative’. When behaviour is guided by absolute ethical standards, a higher-order moral principle is invoked which does not vary with regard to the situation in hand. Such absolutist ethics permit no degree of freedom for ends to justify means or for any beneficial or positive outcomes to justify occasions where the principle is suspended, altered or diluted, i.e. there are no special or extenuating circumstances which can be considered as justifying a departure from, or modification to, the ethical standard. Source: Adapted from Zimbardo (1984) society. In other words, ethical absolutism over- Both sides have a weakness. If, for instance, as the constrains research (Hammersley and Traianou, 2012, absolutists usually insist, there should be informed p. 138); let not the perfect stand in the way of the consent, it may leave relatively privileged groups good or the ‘good enough’. under‑researched (since they will say ‘no’) and By an absolute principle, the Stanford Prison Exper- underprivileged groups over‑researched (they have iment must be regarded as unethical because the partic- nothing to lose and say ‘yes’ in hope). If the indi- ipants suffered considerably. In absolutist principles a vidual conscience is the guide, as the relativists deontological model of research is governed, inter alia, insist, the door is wide open for the unscrupulous – by universal precepts such as justice, honesty and even immoral – researcher. respect. In the utilitarian ethics, ethical research is judged in terms of its consequences, for example, (Plummer, 1983, p. 141) increased knowledge, benefit for many. Those who hold a relativist position would argue He suggests that broad guidelines laid down by profes- that there can be no absolute guidelines and that ethical sional bodies which afford the researcher room for per- considerations will arise from the very nature of the sonal ethical choice offer some way out of the problem. particular research being pursued at the time: the situ We consider these later in this chapter. ation determines behaviour. Indeed they would argue Whilst ethical research concerns principled behav- that it is essential to respect the context in which the iour, as we will see in this chapter, it is often the case research takes place, culturally, ethnically, socio‑eco- that the research draws on different ethical principles nomically, and that these should be judged in their own when considering a specific situation, i.e. ethics are terms (Oliver, 2003, p. 53). This underlines the signifi- situated. cance of ‘situated ethics’ (Simons and Usher, 2000; Hammersley, 2015b), where overall guidelines may 7.3 Sponsored research offer little help when confronted with a very specific situation. From the standpoint of situational ethics (e.g. Sponsored research does not absolve the researcher Simons and Usher, 2000; Oliver, 2003; Hammersley from ethical behaviour. For example, it may be consid- and Traianou, 2012; Hammersley, 2015b), what we ered unethical for the sponsor to control the research or should do or what is right to do depends on the situa- to tell the researcher: (a) how to conduct the research; tion in question, i.e. judging what to do cannot simply (b) what results he/she should look for and what find- be determined, calculated or logically derived from ings should be suppressed; (c) what should and should principles but has to be decided in respect of the not be reported; (d) to conceal who the sponsor is; and presenting situation (i.e. ‘bottom-up’ rather than ‘top- (e) what are the purposes of the research (e.g. Ham- down’): ethical principles inform but do not mersley and Traianou, 2012). simplistically determine. However, this could be chal- Sponsors may have the right to remain confidential; lenged on the grounds that it sanctions relativist ethics they may have the right to non‑disclosure of who they over absolutist principles. are and the purposes and findings of the research. There are some contexts where neither the absolutist Further, researchers will need to consider the effects on nor the relativist position is clear-cut. Writing of the the participants of any disclosure of who the sponsors application of the principle of informed consent with are; for example, if they are told that the research is respect to life history studies, Plummer (1983) says: sponsored by a government office, how will that affect what they say and do? 114
The ethics of educational and social research Sponsors may be oriented towards certain kinds of attention to agreed confidentiality and non-traceability research, for example that which is related to govern- etc.) and the researcher owns the research, not the ment policy or which has immediate or direct practical sponsor. concerns or consequences, and this may suppress ‘criti- cal’ or sensitive research and prefer user-friendly, 7.4 Regulatory contexts of ethics policy-a ffirmative research, or it may exert pressure for a particular style of research to be conducted (e.g. rand- Regulatory contexts of ethics have risen massively in omized controlled trials) or quantitative research. Spon- influence. There have been moves from guidelines and sors may wish to define the research topic, its limits codes of ethics which advise and inform the profes- and how it can or should be done, thereby rendering sional integrity of researchers and research to regula- researchers the instruments of the sponsor. Whilst tions, regulatory frameworks and increasing power and sponsored research is usually contractual between the legalism to govern, control and police them in a liti- researcher and the sponsor, and between the researcher gious age (‘ethics creep’) (Haggerty, 2004). and the participants, and whilst the research may be for Codes of practice, institutional review boards, uni- the sponsor alone and not for the public, this does not versity ethics committees, legislation, ethical frame- privilege the sponsor in dictating how the research works and guidelines exist to oversee research in should be conducted and what it should find; in short, universities and other institutions and can constitute a ‘fixing’ the study. major hurdle for those planning to undertake research. The researcher’s responsibilities may lie only in Ethical codes of the professional bodies and associations conducting the study and providing the sponsor with a as well as the personal ethics of individual researchers report. What happens to the report after that (e.g. are all important regulatory mechanisms. They have a whether it is released completely, selectively or not at ‘gatekeeping’ function, to prevent unethical research all to the public or other parties within the sponsor’s from taking place, to ensure that no harm comes to organization) is a matter for the sponsor. However, this participants, and to ensure that due attention has been does not absolve the researcher from decisions about given by research proposers to the ethical dimensions of the conduct of the study, and the researcher must retain their research: for example, risk assessment; acceptable the right to conduct the study as she or he thinks fit, and unacceptable burdens on people and institutions; informed by, but not decided by, the sponsor. The benefit to different parties; informed consent; researcher’s integrity must be absolute. It is often the confidentiality, non-traceability, privacy, disclosure and case that researchers will negotiate (a) publication protections; control of data; beneficence and non- rights with the sponsor in advance of the research and maleficence; appropriacy of methodology etc. (b) what confidentiality the researcher must respect. The celebrated Belmont Report (National The sponsor has a right to expect high-quality, rig- Commission for the Protection of Human Subjects of orous and useable research. The researcher should not Biomedical and Behavioral Research, 1979) picks out succumb to pressure to: three key ethical principles that codes of ethics are designed to protect: respect for persons, beneficence OO betray the confidentiality of the respondents; and justice (e.g. participants are not denied access to OO tamper with data, their analysis or presentation to potentially beneficial interventions); as seen below, these are echoed by many organizations. Hammersley meet a particular objective; and Traianou (2012) identify key principles as: OO present selective and unrepresentative data and minimization of harm; respect for autonomy (and informed consent); and the protection of privacy conclusions; (addressing confidentiality and anonymity) (discussed OO make recommendations that do not arise from the later in this chapter). Ethical codes of practice are designed to protect the data themselves; interests of individuals and institutions, to ensure OO use the data for non-negotiated personal interests, suitably informed consent and to ensure that the proposed research abides by legal requirements and agendas, purposes and advancement; does not violate ethical principles. They address OO conduct a study in which personal research objec- adherence to data protection laws, including that: (a) data will only be used in the way intended by the tives influence the nature, contents and conduct of research and are important for, rather than superfluous the research. to, the research; (b) data are kept securely and only for The researcher has obligations to the sponsor, but not to doctor or compromise the research. Research is not the same as consultancy in that the researcher may be able to publish from the research (often with due 115
Research design as long as necessary; and (c) control and ownership of OO responsibility (e.g. social, professional, personal) to data are clear, with suitable attention to anonymity researchers and others; (Denscombe, 2014, pp. 319–20). Professional societies and associations have formu- OO voluntary participation. lated codes of practice which express the consensus of values within a particular group and which help indi- Further, in an increasingly information-rich world, it is vidual researchers in indicating what is desirable and essential that safeguards be established to protect what is to be avoided. Of course, this does not solve all research information from misuse or abuse. The UK’s problems, for there are few absolutes and ethical prin- Data Protection Acts of 1984 and 1998 are designed to ciples may be open to a wide range of interpretations. achieve such an end. These cover principles of data Researchers must take cognizance of ethical codes protection, responsibilities of data users and rights of and regulations governing their practice. Failure to data subjects. Data held for ‘historical and research’ meet these responsibilities on the part of researchers is purposes are exempted from the principle, which gives perceived as undermining the scientific process and individuals the right of access to personal data about may lead to legal and financial penalties and liabilities themselves, provided the data are not made available in for individuals and institutions. a form which identifies individuals. Such research data Brooks et al. (2014) note a distinction between may be held indefinitely and their use for research pur- ‘foundational principles’ and ‘practical implications’ poses need not be disclosed at the time of data collec- that have been addressed in regulations, legislation and tion, notwithstanding the Freedom of Information Acts ethical review boards. The former concerns respect for which may give the public access rights to data. Per- persons, justice, beneficence and non-m aleficence, sonal data (i.e. data that uniquely identify the person whilst the latter concerns risk assessment, informed supplying them) shall be held only for specified and consent, benefits and selection of participants and lawful purposes, and appropriate security measures projects (pp. 27–32). The authors analyse several shall be taken against unauthorized access to, or altera- published research guidelines and codes of practice tion, disclosure or destruction of, personal data and from research associations, review boards, research against accidental loss or destruction of personal data. sponsors and governments across the world, and note Regulatory contexts also include organizations some recurrent ethics matters in them (pp. 30–1), which set legally binding regulations. For example, the presented here in alphabetical order: United Nations Convention on the Rights of the Child (UNICEF, 1989) sets out fifty-four Articles which are OO academic freedom; legally binding standards in the fields of, inter alia: OO accountability; non-discrimination; development of full potential; OO adherence to scientific standards; respect for children, their dignity and their views; OO beneficence and non-maleficence (personal and acting in the interests of the child; protection from harm; active and voluntary participation; and voice. social); These include statements that place the interests of the OO compliance with the law; child as the primary consideration (Article 3.1), that the OO concern for welfare; child is assured of the right to express his/her own OO confidentiality; views in matters that affect him/her and that these will OO democracy; be accorded due weight (Article 12.1) and that the OO duty of care; child’s freedom of expression is protected, including OO fairness; the freedom to ‘seek, receive and impart information OO full information; and ideas’ in any media that the child chooses, OO honesty; regardless of frontiers (Article 13.1). OO impartiality; Echoing this, the United Nations Children’s Fund OO independence; (UNICEF ) (Graham et al., 2013) produced a wide- OO integrity; ranging document on Ethical Research Involving Chil- OO justice; dren which recognizes the cultural location of research OO objectivity; (p. 13) and places relationships ‘at the core of ethical OO professional competence; research’ (p. 13). Its key principles are respect, benefit OO quality of research; and justice, with their implications for considerations of OO reliability; harm and benefits (non-m aleficence and beneficence), OO respect for persons, including rights, dignity and informed consent, privacy, confidentiality and payment. Its ‘charter’ (p. 23) includes seven key statements: diversity; 116
The ethics of educational and social research 1 Ethics in research involving children is everyone’s consent; ethical regulation; the gravity of the ethical responsibility. issues; conflicts between the ethical principles, between different interpretations of them, and how to resolve 2 Respecting the dignity of children is core to ethical these differences; having to work with different stake- research. holders and groups simultaneously; the thrust towards worthwhile research; and situated ethics and judge- 3 Research involving children must be just and ments (pp. 4–6). equitable. Hammersley and Traianou also include a compre- hensive bibliography of many aspects of, and topics in, 4 Ethical research benefits children. educational research: diverse perspectives; randomized 5 Children should never be harmed by their participa- controlled trials and experimental research; survey research; action research; researching children; Internet tion in research. research; visual research; narrative and discourse anal- 6 Research must always obtain children’s informed ysis; relationships with funders; anonymity; archiving of data; data protection; philosophical literature on and ongoing consent. ethics (including feminist ethics and radical 7 Ethical research requires ongoing reflection. approaches); and literature on ethical regulation (with extensive websites). This is an extremely useful refer- (Graham et al., 2013, p. 23) ence document, with multiple website references. The American Educational Research Association’s Many institutions of higher education have their own (2011) Code of Ethics is organized into principles and ethics committees, with their own codes of ethics ethical standards: against which they evaluate research proposals. In addi- tion, some important codes of practice and ethical OO P rinciples: professional competence; integrity; pro- guidelines are published by research associations, for fessional, scientific and scholarly responsibility; example the British Educational Research Association, respect for people’s rights, dignity and diversity; the British Psychological Society, the British Sociolog- and social responsibility. ical Association, the Social Research Association, the American Educational Research Association, the OO E thical standards: scientific, scholarly and profes- American Psychological Association and the American sional standards; competence; use and misuse of Sociological Association. We advise readers to consult expertise; fabrication, falsification and plagiarism; these in detail. avoiding harm; non-d iscrimination; non-exploitation; The British Educational Research Association’s harassment; employment decisions; conflicts of inter- (2011) Ethical Guidelines for Educational Research est; public communications; confidentiality (includ- are devolved onto: ing privacy and electronic data storage and communication); informed consent (including decep- OO P rinciples: ‘all educational research shall be con- tion); research planning, implementation and dissem- ducted within an ethic of respect for: the person; ination; authorship credit; publication process; knowledge; democratic values; the quality of educa- responsibilities of reviewers; teaching, training and tional research; academic freedom’ (p. 4). administering education programmes; mentoring; supervision; contractual and consulting services; and OO G uidelines: these address: (a) responsibilities to par- adherence to the ethical standards of the American ticipants (including sections on: voluntary informed Educational Research Association. consent; openness and disclosure; rights to with- draw; children, vulnerable young people and vulner- These standards run right through the entire research able adults; incentives; detriment arising from process, from planning to dissemination. participation in research; privacy; disclosure); (b) The UK’s Economic and Social Research Council’s responsibilities to sponsors of research (including (2015) research ethics framework sets out six key methods and publication); (c) responsibilities to the principles: community of educational researchers (including sections on misconduct and authorship); and (d) 1 Research participants should take part voluntarily, responsibilities to educational professionals, policy free from any coercion or undue influence, and their makers and the general public. rights, dignity and (when possible) autonomy should be respected and appropriately protected. Hammersley and Traianou (2012), writing for the British Educational Research Association, set out five ethical principles: minimizing harm; respecting auton- omy; protecting privacy; offering reciprocity; and treat- ing people equitably. In light of these principles they raise several issues (pp. 4–12): full and free informed 117
Research design 2 Research should be worthwhile and provide value On the other hand, Hammersley raises the argument that outweighs any risk or harm. Researchers should that principles may be ‘too determinate’ and prescrip- aim to maximise the benefit of the research and min- tive in the force of their injunctions and in seeking to imise potential risks of harm to participants and derive or deduce from general principles (in a ‘quasi- researchers. All potential risk and harm should be logical way’) (2015c, p. 445) those ethical practices mitigated by robust precautions. which obtain in a specific situation (pp. 443–8). Further, he advances the argument for ‘particularism’ 3 Research staff and participants should be given rather than ‘principlism’, holding that, since research- appropriate information about the purpose, methods ers have to focus on specific projects, their decisions and intended uses of the research, what their partici- are ‘situated’ and their ethical decisions are taken with pation in the research entails and what risks, bene- reference to the case in hand rather than being derived fits, if any, are involved. from sets of principles (p. 441) (see also Hammersley and Traianou, 2012). Whilst principles raise considera- 4 Individual research participant and group prefer- tions for researchers, they do not necessarily determine ences regarding anonymity should be respected and what ethical decisions should be taken in a specific participant requirements concerning the confidential case; rather, they inform such decision making. nature of information and personal data should be The British Psychological Society’s Code of Human respected. Research Ethics (2014) sets out four principles: respect for the autonomy and dignity of persons (including 5 Research should be designed, reviewed and under- rights to privacy, self‑determination, personal liberty taken to ensure recognised standards of integrity are and natural justice) (p. 8); scientific value; social met, and quality and transparency are assured. responsibility; and maximizing benefit and minimizing harm. It also states that ‘[r]esearchers should respect 6 The independence of research should be clear, and the rights and dignity of participants in their research any conflicts of interest or partiality should be and the legitimate interests of stakeholders such as explicit. funders, institutions, sponsors and society at large’ (Economic and Social Research Council, 2015, p. 4) (p. 4). These principles inform its discussions of: risk; valid consent; confidentiality; giving advice; deception; One can note here the use of the word ‘should’, i.e. it has debriefing; ethics review; safeguards for, and working the power of an injunction, a demand (Hammersley and with, vulnerable populations. Traianou, 2012, p. 7). The document then provides com- With regard to respecting dignity, there is the need prehensive coverage of eleven minimum requirements to treat participants as equals, not as objects or as sub- informed by these principles, included in which is the role ordinate to the researcher. This may mean avoiding of Research Ethics Committees to ‘protect the dignity, treating them as ‘subjects’ rather than as equals or par- rights and welfare of research participants’ (p. 14). The ticipants.1 It also means avoiding stigmatizing groups document sets out: (a) a definition of risk and how to (e.g. the unemployed; the homeless; religious groups; approach risk assessment (pp. 27–9); (b) key issues in ethnic groups; those considered deviant by virtue of considering voluntary informed consent (pp. 29–33); and their sexual orientation, dress, beliefs). The document (c) issues to be addressed in an ethics review (pp. 38–9). also indicates that ethical research extends to concerns Further, it carries an extensive list of relevant organiza- of: involvement with vulnerable groups; sensitive tions and their publications on research ethics (pp. 45–9). topics; deception; access to personal or confidential This is a comprehensive document with useful advice to records or sensitive data from third parties (e.g. researchers in considering ethical matters. employers); the potential to induce stress, anxiety, pain It is important to note the references to ‘principles’ humiliation; intrusive interventions that are outside the in these documents. Hammersley (2015c) argues that routine lives of individuals; and that which could lead the inclusion of principles (e.g. ‘general considera- to negative labelling. tions’) (p. 435) is helpful, as: (a) it overcomes the risk From this very brief excursus into the contents of that ethics codes and guidelines could become outdated codes of ethics, it can be seen that they cover similar with advances coming on apace in research methodolo- topics. Ethical codes are a guide, but they cannot gies and topics; (b) they facilitate agreement among dictate to the researcher what to do in a specific, unique researchers about essential matters; (c) being at a high situation, nor can they absolve the researcher of respon- level of abstraction, they are likely to be inclusive of sibility for action taken in the research. The issue here most research projects (though such abstraction might is that ethics are ‘situated’ (Simons and Usher, 2000): lend itself to varying interpretation); and (d) they avoid too tight a level of prescription which often encourages researchers ‘to follow the letter rather than the spirit of ethical codes’ (p. 434). 118
The ethics of educational and social research while ethics has traditionally been seen as a set of may over-simplify complex research situations (cf. general principles invariantly and validly applied to Hammersley and Traianou, 2012; Brooks et al., 2014, all situations, … on the contrary, ethical principles pp. 34–42). are mediated within different research practices and Ethical regulation of research is often conducted by thus take on different significances in relation to university ethics committees. Whilst the intentions here those practices. might be honourable in protecting individuals and insti- tutions, in a blistering paper against ethical regulation (Simons and Usher, 2000, p. 1) by research ethics committees, Hammersley (2009, pp. 212–19) argues that: The authors state that this implies that situated ethics are ‘immune to universalization’, because: a they are incapable of making sound or ‘superior’ ethical decisions, such that their work will not researchers cannot avoid weighing up conflicting improve the ethical quality of research. This is considerations and dilemmas which are located in because: (i) ethical issues are contentious and there the specificities of the research situation and where is a lack of consensus among social scientists on there is a need to make ethical decisions but where ethical matters, principles, priorities of principles those decisions cannot be reached by appeal to and practices, or consequences; (ii) ethical issues unambiguous and univalent principles or codes. and practical research are complex (e.g. secrecy and deception in research); (iii) ethical answers cannot (Simons and Usher, 2000, p. 2) simply be cranked out, mechanistically or algorith- mically, but are framed in specific contexts (which Against ethical codes and regulation may be unknown to ethical committees); (iv) the remit of ethical committees is unclear, for example, Whilst ethics committees and regulations may be a whether to approve, prevent, control methodology useful safeguard, for example ‘gatekeeping’, or topics; (v) ethical committees only need to be Farrimond (2013) suggests that ethics committees can persuaded that the researcher has the ethical capabil- be too time-c onsuming and bureaucratic, unfairly use ity to conduct the research, but this confuses ethical models from one discipline (e.g. medicine) in another audit with ethical decision making and confuses (e.g. social science), are part of a wider ‘audit culture’, substance and procedures of ethical review; are over-concerned with protecting institutional reputation, make it difficult for researchers to b they have no legitimacy or moral superiority/exper- challenge decisions, and lack consistency (pp. 49–51). tise to control researchers, and that this is inherent in Brooks et al. (2014) report that they can also ethical principles themselves: (i) researchers should undermine and discourage professional expertise and have their autonomy respected; (ii) it is the research- reflection. They can be unduly restrictive and lack ers themselves – and not ethics committees – who cultural relevance. In their endeavour to protect even have the responsibility for the ethical conduct of minimal harm from coming to individuals, they may research and such responsibility cannot and should overlook the principle of the greatest good for the not be passed to a committee; (iii) ethics committees greatest number. They may require socio-culturally must apply the principle of ‘informed consent’ to insensitive or inappropriate ways of ensuring informed researchers, and not just to those being researched; consent, for example, requiring such consent to be (iv) ethics committees operate prospectively, not given on paper-copy pro formas, whereas this is only retrospectively, and this kind of prospective inappropriate in some (e.g. non-w estern) cultures, regulation is highly unusual in most areas of life; (v) and, indeed, they may require individuals to sign such there is almost no evidence that researchers operate forms when, in reality, in some cultures it is not the unethically apart from some illegal cases, and so the individual but the family head, the community and its processes of ethics committees are unnecessary, i.e. leaders who should give consent (Lie and Witteveen there is no problem which needs to be fixed; (2017) comment on the value of filmed informed consent rather than in written form). c they lead to undesirable consequences in research: Further, ethics committees, codes of practice and (i) the bureaucratization of research; (ii) the time regulations may emphasize rule-following in situations and effort required to meet bureaucratic require- where following rules and regulations is insufficient ments will deter many researchers from proceeding; for taking ethical decisions in situ. Indeed they may (iii) research will avoid sensitive, difficult or con- protect the university or institution rather than the tested yet important areas and marginalized or research participants, they may prevent important powerful groups, i.e. where informed consent may sensitive research from being undertaken, and they 119
Research design not be possible; (iv) we may not discover important to consider whose interests are involved in, or at stake data; (v) researchers will avoid important research in, conducting the research. Why focus on such-and- areas because they may consider it difficult to obtain such in proposing the research? Choice of research the consent of the ethics committee. topic also raises issues of privacy, sensitivity and access, and we address these later in the chapter. Hammersley argues that ethics committees’ roles Here we do not rehearse the issues of ‘partisan should be limited to providing advice, providing a research’ that we addressed in Chapter 3, though forum for discussion on ethical matters and initiating readers may find it useful to review that chapter. such discussions. This echoes the comment from Howe Rather, we address issues of the kind of research pro- and Moses (1999, pp. 46–5) that research ethics com- posed. Researchers may find themselves having to mittees have no special expertise to judge many educa- defend not only their research design but their method- tional research issues about such-a nd-such a project, ology in the face of, for example, questions from ethics that they are bureaucratic and tend to discharge their committees or sponsors who lean towards quantitative duties in a perfunctory manner. They may provide methods and experimental approaches and away from advice and guidance, but not prospective judgements what they perceive to be ‘soft’, unscientific qualitative about specific research projects (Howe and Moses, research. This may extend to their being uncomfortable 1999, p. 53). with covert research or research which, as it unfolds The difficulty with, and yet the strength of, ethical over time or is conducted in non-m ainstream or ‘differ- codes is that they cannot and do not provide specific ent’ cultures, may raise questions of informed consent advice for what to do in specific situations. And the dif- or where the research will go (e.g. ‘blue-s kies’ ficulty for ethics committees is that their operations are research) and what the research will ‘deliver’ (e.g. seen as impractical and, ultimately, anti-ethical, anti- issues which routinely face ethnographic research). research and anti-researchers. Ultimately, it is research- The issue goes further, into instrumentation: some ers themselves, their integrity and their conscience, sponsors may not be friendly towards survey research informed by an acute awareness of ethical issues, (Brooks et al., 2014), particularly if it might find underpinned by guideline codes and regulated practice, ‘unwelcome data’ (p. 72); they may either bar research- which should decide what to do in a specific situation, ers from identifying individuals or, by contrast, posi- and this should be justified, justifiable, thought through tively require parties and individuals to be identified. and defensible. As discussed earlier, this is illegitimate, as sponsors It was observed earlier that many ethical codes and cannot dictate to researchers how to go about their guidelines themselves avoid univalency and unambigu- business. ity, arguing, for example, that deception, covert Even apparently innocuous instruments such as research and the lack of informed consent may be justi- questionnaires broach ethical issues, as respondents fied. The need for polyvalency (multiple interpretations typically have to answer in terms of categories already of what is worthwhile, acceptable and valuable) and decided by the researcher, and this risks reducing the situated ethics arises from the practicality of conduct- participants to data objects rather than agentic people ing research, the need for sensitivity to socio-political (Hammersley and Traianou, 2012c, p. 12). Observa- contexts and fairness to disadvantaged groups, and to tion, participant or non-p articipant, intrudes into take account of the diversity and uniqueness of differ- people’s lives and privacy. ent research practices. What this suggests, then, is that, Researchers will also need to consider where their whilst codes, guidelines and committees may be useful research will take place, for example, in classrooms, in raising issues and orienting researchers, they cannot off-site (e.g. for confidentiality and ‘neutral territory’), decide what should and should not be done in a specific as context often influences the research. For example, situation; that is for individual researchers and their interviewing children in school may be more comforta- informed conscience to decide. ble for children than outside school or, indeed, the opposite may be the case (Morrison, 2013a). Conduct- 7.5 Choice of research topic and ing interviews in school may make students feel research design obliged to participate when in fact they would prefer not to participate, or might discourage them from being Ethical issues enter into considerations of choice of honest in favour of saying or doing what they think the topic and the design and operationalization of the school would require. Schools as hierarchical institu- research. The decision on what to research may become tions may exert an influence on students (Brooks et al., a political act, deliberate or not, and the researcher has 2014, p. 76). 120
The ethics of educational and social research In setting research agendas, the researcher may also instrumentation; transparency, usefulness and validity; face dilemmas in deciding whose agendas to serve: the scholarly and scientific merit; significance and advance- insiders’ (the participants, the sponsors) or the outsid- ment of the field (e.g. substantively, conceptually, ers’ (the researcher, the reader, the audiences of the methodologically); scientific value; risk assessment and research, the public). Whilst this may lead to some minimization; and transparency. Research which does negotiation of research agendas and methodologies (e.g. not advance the field or which is of poor quality may in sensitive research, see Chapter 13), this is an ethical be a waste of time and resources, and may be literally issue and the integrity of the researcher cannot be sacri- useless, all of which violate ethics. ficed to insider power. On the one hand it may be better Ethics in educational research also affect those not not to do the research than to compromise ethical prin- directly involved in the study but who may be affected ciples in deciding the agenda and the conduct, method- by it, for example, children and parents. Ensuring the ology and reporting of the research, how the research quality of research is an ethical matter, as poor-quality will be conducted and reported, and to whom. On the educational research can cause harm, particularly if other hand it may be better to give ground in some used for decision making, for example, by funding quarters in order to gain ground in others: better to do bodies, parents, governments and policy makers. In some research rather than no research at all or better to other disciplines such as medicine, poor-q uality make some matters public rather than no matters at all. research would not pass muster and would be prevented This returns to the issue of ‘situated ethics’ introduced from taking place (and this might be a legitimate role earlier: researchers have to decide how to behave ethi- for ethics committees). In poor-q uality research: the cally in each specific situation. evidence is weak; it uses inappropriate designs, In identifying the research topic and the design of methods and data analysis; conclusions made do not the research, the researcher moves to planning the follow from the evidence; no warrants are made to link conduct of the research, and this raises a huge range of evidence and conclusions (Gorard, 2013); reporting is issues, which we consider below. These ethical con- biased (we discuss bias in data analysis, reporting and cerns need to be addressed very early on in the plan- dissemination later in this chapter); and claims made ning, design and conduct of the research, including: from the research are not supported by the research and informed consent; non-m aleficence; beneficence; data provided. Indeed taxpayers’ money is wasted. human dignity; privacy; anonymity; confidentiality; In poor-q uality research, claims concerning ‘impact’ betrayal and deception; Internet ethics; ethics and eval- are made from inappropriate sampling, lack of baseline uative research. data for enabling comparisons to be made, attrition and missing data, lack of counterfactual analysis and data, Ethics and the quality of the research and inadequate consideration of alternative explanations of findings. Ethical researchers have a duty of care to The research design, and indeed the research itself, ensure that their research is of the highest quality. have an ethical duty to demonstrate quality. Put simply, Robson (1993, p. 33) raises ten questionable badly designed research is a breach of ethics (Farri- practices in social research: mond, 2013, p. 75). As Hammersley and Traianou (2012) argue, research ethics is not only about treating OO involving people without their knowledge or people correctly, i.e. procedural matters such as rights, consent; interests and duties, but about the ‘primary obligation’ (p. 1) to answer worthwhile questions; to pursue, OO coercing them to participate; produce, test and defend valid factual knowledge (e.g. OO withholding information about the true nature of the descriptions, explanations, interpretations, conclusions, theories etc.) on the basis of full evidence; and to make research; a significant, relevant contribution to knowledge. The OO otherwise deceiving participants; fundamental purpose of research is the production of OO inducing them to commit acts diminishing their self- valid, relevant, worthwhile and significant knowledge. It does not have as a necessary goal the practicality or esteem; political attractiveness of such knowledge; there may OO violating rights of self-determination (e.g. in studies be motives for undertaking the research but these are not the purposes of the research itself (p. 134). seeking to promote individual change); Producing knowledge includes, inter alia: a rigorous OO exposing participants to physical or mental stress; and coherent research design that demonstrates fitness OO invading their privacy; for purpose; appropriate sampling, methodology and OO withholding benefits from some participants (e.g. in comparison groups); OO not treating participants fairly, or with considera- tion, or with respect. 121
Search
Read the Text Version
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
- 18
- 19
- 20
- 21
- 22
- 23
- 24
- 25
- 26
- 27
- 28
- 29
- 30
- 31
- 32
- 33
- 34
- 35
- 36
- 37
- 38
- 39
- 40
- 41
- 42
- 43
- 44
- 45
- 46
- 47
- 48
- 49
- 50
- 51
- 52
- 53
- 54
- 55
- 56
- 57
- 58
- 59
- 60
- 61
- 62
- 63
- 64
- 65
- 66
- 67
- 68
- 69
- 70
- 71
- 72
- 73
- 74
- 75
- 76
- 77
- 78
- 79
- 80
- 81
- 82
- 83
- 84
- 85
- 86
- 87
- 88
- 89
- 90
- 91
- 92
- 93
- 94
- 95
- 96
- 97
- 98
- 99
- 100
- 101
- 102
- 103
- 104
- 105
- 106
- 107
- 108
- 109
- 110
- 111
- 112
- 113
- 114
- 115
- 116
- 117
- 118
- 119
- 120
- 121
- 122
- 123
- 124
- 125
- 126
- 127
- 128
- 129
- 130
- 131
- 132
- 133
- 134
- 135
- 136
- 137
- 138
- 139
- 140
- 141
- 142
- 143
- 144
- 145
- 146
- 147
- 148
- 149
- 150
- 151
- 152
- 153
- 154
- 155
- 156
- 157
- 158
- 159
- 160
- 161
- 162
- 163
- 164
- 165
- 166
- 167
- 168
- 169
- 170
- 171
- 172
- 173
- 174
- 175
- 176
- 177
- 178
- 179
- 180
- 181
- 182
- 183
- 184
- 185
- 186
- 187
- 188
- 189
- 190
- 191
- 192
- 193
- 194
- 195
- 196
- 197
- 198
- 199
- 200
- 201
- 202
- 203
- 204
- 205
- 206
- 207
- 208
- 209
- 210
- 211
- 212
- 213
- 214
- 215
- 216
- 217
- 218
- 219
- 220
- 221
- 222
- 223
- 224
- 225
- 226
- 227
- 228
- 229
- 230
- 231
- 232
- 233
- 234
- 235
- 236
- 237
- 238
- 239
- 240
- 241
- 242
- 243
- 244
- 245
- 246
- 247
- 248
- 249
- 250
- 251
- 252
- 253
- 254
- 255
- 256
- 257
- 258
- 259
- 260
- 261
- 262
- 263
- 264
- 265
- 266
- 267
- 268
- 269
- 270
- 271
- 272
- 273
- 274
- 275
- 276
- 277
- 278
- 279
- 280
- 281
- 282
- 283
- 284
- 285
- 286
- 287
- 288
- 289
- 290
- 291
- 292
- 293
- 294
- 295
- 296
- 297
- 298
- 299
- 300
- 301
- 302
- 303
- 304
- 305
- 306
- 307
- 308
- 309
- 310
- 311
- 312
- 313
- 314
- 315
- 316
- 317
- 318
- 319
- 320
- 321
- 322
- 323
- 324
- 325
- 326
- 327
- 328
- 329
- 330
- 331
- 332
- 333
- 334
- 335
- 336
- 337
- 338
- 339
- 340
- 341
- 342
- 343
- 344
- 345
- 346
- 347
- 348
- 349
- 350
- 351
- 352
- 353
- 354
- 355
- 356
- 357
- 358
- 359
- 360
- 361
- 362
- 363
- 364
- 365
- 366
- 367
- 368
- 369
- 370
- 371
- 372
- 373
- 374
- 375
- 376
- 377
- 378
- 379
- 380
- 381
- 382
- 383
- 384
- 385
- 386
- 387
- 388
- 389
- 390
- 391
- 392
- 393
- 394
- 395
- 396
- 397
- 398
- 399
- 400
- 401
- 402
- 403
- 404
- 405
- 406
- 407
- 408
- 409
- 410
- 411
- 412
- 413
- 414
- 415
- 416
- 417
- 418
- 419
- 420
- 421
- 422
- 423
- 424
- 425
- 426
- 427
- 428
- 429
- 430
- 431
- 432
- 433
- 434
- 435
- 436
- 437
- 438
- 439
- 440
- 441
- 442
- 443
- 444
- 445
- 446
- 447
- 448
- 449
- 450
- 451
- 452
- 453
- 454
- 455
- 456
- 457
- 458
- 459
- 460
- 461
- 462
- 463
- 464
- 465
- 466
- 467
- 468
- 469
- 470
- 471
- 472
- 473
- 474
- 475
- 476
- 477
- 478
- 479
- 480
- 481
- 482
- 483
- 484
- 485
- 486
- 487
- 488
- 489
- 490
- 491
- 492
- 493
- 494
- 495
- 496
- 497
- 1 - 50
- 51 - 100
- 101 - 150
- 151 - 200
- 201 - 250
- 251 - 300
- 301 - 350
- 351 - 400
- 401 - 450
- 451 - 497
Pages:
