Goh (2020) Engaging students in feedback

ENGAGING STUDENTS IN FEEDBACK Rachel Goh Overview 1. How is teachers’ feedback conceptualized? 2. How does feedback enhance learning? 3. What constitutes student feedback literacy? 4. How do student feedback literacy enabling practices look like in the classroom? Introduction: How would I want my students to find ways to remember and act on my feedback? Have I done anything in particular to help them remember and act on my feedback? What should be achieved with or from feedback? These questions and others are what teachers grapple with when thinking about what assessment and feedback should do for student learning. Feedback can achieve several things: learning at a higher or different level, or at the same level and to a better progressive outcome. The purpose of feedback in addressing learning gaps and improving teaching practices is clearly articulated in the Singapore Curriculum Philosophy. Providing feedback to students to improve their learning is thus viewed as an important aspect of the assessment competency of teachers in the Singapore classroom. A principal once commented to a teacher, “Math always tick tick cross cross, but what do the ticks and crosses mean to the students?” The principal’s comment got the teacher thinking about whether her students could understand the ticks and crosses as she had intended, and be able to act on the feedback on their own. 1

As educators, our understanding of feedback is particularly important if we believe assessment to be integral to the teaching and learning process, and is ultimately aimed at helping students become self- directed learners (Singapore Curriculum Philosophy, 2018). Students becoming self-directed learners who question, reflect, persevere and take responsibility for their own learning is one of the four Desired Outcomes of Education (DOE) of the Singapore education system (Desired Outcomes of Education, 2018). Amongst other processes, feedback lies at the heart of formative assessment in interpreting information from assessment, and adapting instructional practices accordingly, to address student learning gaps and improve teaching practices. So how can teachers provide feedback to students effectively and efficiently? How do we know that feedback has been effective? What differentiates effective feedback from ineffective feedback? The answers to these questions may be found in understanding the role of student engagement with feedback in making assessment feedback effective. The focus of effective feedback practice is, therefore, not only on what teachers do, but also on students’ understanding and their uptake of feedback information to improve their learning. Developing student feedback literacy is the sine qua non of helping students in their learning. This chapter aims to provide readers with a deeper understanding of the following aspects in engaging students in feedback: • Conceptualizations of teachers’ feedback • Feedback that enhances learning • Student feedback literacy • Student feedback literacy enabling practices in the classroom 2

Theory/Principles 1. Conceptualizations of teachers’ feedback Learning theories are helpful in unpacking the differences in conceptions of feedback. The following section presents different ways teachers’ feedback has been defined and understood from the lens of two learning theories, cognitivism and social constructivism. An argument is made for a wider conception of feedback that supports our view of learning which “takes place individually and collaboratively” and our belief in the role of the students in taking ownership of their learning and (Singapore Curriculum Philosophy, 2018). 1.1 Feedback as information product The idea of involving students in the process of feedback can be traced back to Sadler’s (1989) proposition of the three essential conditions for students to benefit from feedback. He proposed that students need to “(a) possess a concept of the standard (or goal, or reference level) being aimed for, (b) compare the actual (or current) level of performance with the standard, and (c) engage in appropriate action which leads to some closure of the gap” (p. 121). The notion of having students compare how their actual performance relates to good performance was influenced by Ramaprasad’s (1983) definition of feedback in organizational systems as “information about the gap between the actual level and the reference level of a system parameter which is used to alter the gap in some way” (p. 4). This conceptualization of feedback from a cognitivist point of view perceives feedback as a product (Chong, 2018), specifically as information to be conveyed to students, for example, on the errors to be corrected, or the accuracy of student understanding (Gipps & Simpson, 2014). This is aligned to how learning, from a cognitivist standpoint, is understood to 3

be about the acquisition of knowledge, developing internal mental structures, and individual sense-making. Years later, Hattie and Timperley (2007) in a widely-cited review of feedback literature, expanded on the conception of feedback as product by including the role of agents in their definition of feedback, describing feedback as: “information provided by an agent (e.g., teacher, peer, book, parent, self, experience) regarding aspects of one’s performance or understanding” (p. 81). Thurlings, Vermeulen, Bastiaens, and Stijnen (2013) proposed that the characteristics of the process of feedback is related to the specific learning theory from which student learning is facilitated. The authors depict the feedback processes related to cognitivism in a linear manner, as shown in Figure 1. They indicate that the feedback process underpinned by this learning theory begins with the teacher giving feedback, which is then processed by the students, and finalizes in particular learning outcomes. Feedback Students Learning outcomes Figure 1. The feedback model for cognitivism. Adapted from Thurlings et al. (2013). 1.2 Feedback as an interactive process From a social constructivist standpoint, feedback is viewed not as a product, but as an interactive process (Chong, 2018). Feedback is conceptualized as dialogue between teachers and students, and with peers, rather than as a commodity to be imposed in a unilateral manner 4

on students (Beaumont, O’Doherty, & Shannon 2011; Carless, 2016; Tan, 2014). This is aligned to how learning, from a social constructivist standpoint, is understood to be about creating meaning from experience, and is negotiated through interaction with others. Thurlings et al. (2013) depict the feedback processes related to social constructivism in a cyclical manner, as shown in Figure 2. The authors indicate that the starting point of learning underpinned by social constructivism is prior knowledge, and the feedback process begins with the students at the beginning stage. Multiple sources of feedback, peers and teacher, give feedback, and as learning is continuous, students progress to another stage, which serves as a beginning stage of the next learning cycle. Students Multiple Feedback Students Learning at sources of at another outcomes feedback beginning stage of stage (e.g., learning peers, teacher) Figure 2. The feedback model for social constructivism. Adapted from Thurlings et al. (2013). Engaging students in assessment feedback dialogue is said to help them make sense of feedback information, and clarify their understanding of assessment criteria (Tan, 2015). A good understanding of intended learning outcomes and standards allows students to monitor their own 5

progress, and helps them focus on achieving learning outcomes, instead of merely pursuing marks as proxy for success. 1.3 Feedback as an internal dialogue Other than just being viewed as an interactive process involving teacher and peers, Carless (2016) proposed that dialogic feedback can also operate as an “inner dialogue or self-monitoring in which students are engaged when they are tackling a task” (p. 4). The conceptualization of feedback as an internal dialogue extends the notion of feedback as an interactive process as described in Section 1.2. Chong (2018) proposed that when feedback is perceived as such an internal process, the underlying premise is that “students are not regarded as passive recipients but active agents [emphasis added] who make use of feedback to connect their present performance and future expected performance” (p. 191). The view of students’ responsibility in sense- making of feedback on their own is aligned to cognitivism which regards learning to be about students’ individual sense making through structuring new knowledge to tap their own prior knowledge so that it is assimilated or accommodated within their internal cognitive structures. 1.4 An expanded conceptualization of teachers’ feedback Most recently, Carless and Boud (2018) offer a more expanded definition of feedback that supports their theory of student feedback literacy. They defined feedback as “a process [emphasis added] through which learners make sense [emphasis added] of information [emphasis added] from various sources and use it to enhance their work or learning strategies [emphasis added]” (p. 1315). Their definition of feedback appears to encompass all three conceptualizations of feedback: as an interactive 6

process, an internal dialogue, and feedback as formative information, rather than a binary contrast of either process or product. Their definition of learning also includes an important distinction that feedback information influences students with regard to, not only to their current work performance, but also to their future learning. Their ideas provide a great expansion to a theoretical framing of feedback that could support our efforts in developing students in taking ownership of their learning (Singapore Curriculum Philosophy, 2016), while acknowledging our roles as teachers in modelling good feedback practices. This section on the conceptualizations of teachers’ feedback demonstrates how the theory of learning shapes the characteristics of the feedback processes, and consequently the enactment of teachers’ feedback practice. The tripartite definition of feedback as an interactive process, an internal dialogue, and formative information (Carless & Boud, 2018) can help us frame our understanding of how feedback enhances learning in the classroom, as elaborated in the next section. 2. Feedback that enhances learning In this section, Hattie and Timperley’s (2007) model of feedback is unpacked to draw insights on how feedback could enhance learning at different levels. The three questions that effective feedback answers are presented, before explaining the notions of feed up, feed back, and feed forward. Then, the four levels at which feedback comments can be directed are discussed and illustrated with examples. Finally, three misconceptions that some teachers may have in interpreting the levels of feedback are addressed. 7

2.1 Effective feedback answers three questions Hattie & Timperley (2007) states that feedback works by reducing the discrepancy between students’ current understanding and desired understanding. In explaining the theoretical background of their model, they echoed Sadler’s (1989) three conditions of effective feedback stated earlier. The authors indicated that effective feedback must answer the following three major questions asked by a teacher and/or by students. Table 1 lists the three questions from the students’ and teachers’ perspectives. Table 1 Three questions addressed by effective feedback. Asked by Asked by teacher students Where am I What are the goals? going? How am I What progress is being made toward the goal? going? Where to What decisions and actions need to be undertaken to next? make better progress? e.g., how to plan the next class, what follow up tasks Such a framing of effective feedback emphasizes both the role of students and teachers. It helps overcome the binary conceptions of feedback discussed in Section 1 by creating a more nuanced understanding of feedback as a process. 8

2.2 Where am I going? (Feed up), How am I going? (Feed back), and Where to next? (Feed forward) Hattie and Timperley (2007) suggest that the above three questions “correspond to notions of feed up, feed back, and feed forward” (p. 86) as the focus of effective feedback. “Feed up” practice calls for the communication of achievement standards with students. Success criteria can be communicated in many forms such as through unpacking rubrics, reviewing checklists, and using various exemplars of student work to help students notice different standards. As for “feed back” practice, it would require the design of an appropriate task for students to complete in order to generate feedback on their current performance. With regard to “feed forward”, scholars agree that the only way to tell if learning results from feedback or whether feedback information is effective is for students to take some action to close the feedback loop, and demonstrate it by producing improved work (Boud, 2000; Sadler, 1989). This view of improved performance as an indicator of learning is consistent with Sadler’s (2007) definition of learning. He posits that students can be said to have learned when three conditions are satisfied: “They must be able to do, on demand [emphasis added], something they could not do before. They have to be able to do it independently [emphasis added] of particular others, those others being primarily the teacher and members of a learning group (if any). And they must be able to do it well [emphasis added]” (ibid, p. 392). As such, “feed forward” practice calls for opportunities for students to act on the feedback. One way to do so may be to have students submit improved versions of particular portions of their work. For example, in the teaching of writing, students are asked to rewrite the conclusion to 9

an essay based on the feedback given such that it makes a stronger link back to the introduction. It is not always the case that students’ corrections of their previous works, would immediately be a form of effective feed forward. For instance, in re-doing or re-answering a Math question by just copying the right answer without understanding is counter-productive. A very useful suggestion for feeding up is offered by some Math educators in a professional development course on assessment that I conducted. Instead of asking students to re-attempt a general set of questions as a follow-up task, the questions in the identified worksheet or workbook could be regrouped into Set A questions and a parallel Set B questions, based on identified learning outcomes or issues. For example, in the teaching of division of a 2-digit number by a 1-digit number without regrouping, Set A could involve questions with divisors of 2, 3, 4, and 5. Set B questions would have the same divisors. After the teaching instruction which may involve using a checklist of steps to make explicit the long division algorithm, students work on Set A questions. During the task, students refer to the checklist to self-assess their progress. After the task, they may also be asked to peer assess and give feedback aimed at the task-level with reference to the checklist or mark scheme to correct each other’s errors. Students act on the peer feedback before submitting their work. When the teacher assesses students’ work, feedback, written and/or verbal, can be directed at the process level and/or self-regulation level to help students develop a better understanding of the algorithm checklist/process and/or a more robust judgement of their work. (The different levels at which feedback can be directed will be explained further in the next section.) Should the identified issue be students’ lack of the pre-requisite mathematical knowledge such as the 3 times table, giving feedback may not be as useful as having students master their mathematical facts. Following 10

that, Set B questions are then assigned to students to assess if they are able to close the identified learning gap. However, it may not be a simple case of students applying the feedback that teachers provide. Relational and emotional factors can be at work which influence how students receive, interpret, and use feedback, as will be discussed in Section 3 in unpacking the notion of student feedback literacy. 2.3 Four levels of feedback In their review of the literature, Hattie and Timperley (2007) looked at evidence of how feedback answers the three questions described above, and they concluded that each feedback question can work at four levels: task, process, self-regulation, and self-level. The levels of feedback are described in Table 2, along with examples of teacher’s written comments that illustrate the level which the feedback is directed at. Table 2 Levels of feedback Level of Description (including Example feedback strength/weakness) Task level • About whether students’ work You have not answered the is correct or incorrect. question. Focus on • Tells students directly what to the causes and not the consequences. correct. • Effective if the issue is with student misconceptions. • Not as effective if the issue is with students’ lack of 11

necessary subject matter knowledge. Process • For processing of information Have you considered level and requires understanding. all aspects of the 3C • Tells students how to do it framework? better. • Builds student capacity beyond the immediate task. Self- • For developing students’ What have you learnt regulation assessment capacity beyond about level current task and process. consensus/politics? What would help you • Prompts students to self- prepare for and regulate and refine the perform future judgement of their work challenges? against the success criteria. • Builds self-initiated development of capacity of students. Self-level • General praise of students. Good effort! • Usually ineffective. Well done! Try harder! Hattie and Timperley (2007) argue that feedback directed at the self- level is least effective in enhancing student learning. Feedback aimed at the task level or corrective feedback is helpful in correcting students’ faulty interpretations, but students’ learning may not be transferred to answering other questions beyond the current task. The authors maintain that “feedback aimed to move students from task to processing 12

and then from processing to regulation is most effective” (p. 91). The work by Hattie & Timperley (2007) has important implications on extending the repertoire of teachers’ feedback comments as well as in thinking about student progression in their response to feedback in their entry behavior and exit behavior in a course of study. 2.4 Misconceptions about levels of feedback In my consultation work with teachers in schools, it has been observed that some teachers may have some misconceptions in their reading of the work by Hattie and Timperley (2007). One misconception is that the notion of the four levels of feedback is about the quality assurance level of feedback. Some teachers seem to believe that the writing of feedback comments aimed at a particular level will necessarily lead to observed student outcomes at the intended level. They believe, for example, feedback comments aimed at the self-regulation level will directly result in student outcomes or actions at the self-regulation level. In fact, Hattie and Timperley (2007) only wrote that: “The focus of feedback is critically important, and in this article, we claim that there are four major levels and that the level at which feedback is directed influences [emphasis added] its effectiveness” (ibid, p. 90). The authors’ claim is clearly that the idea of the levels of feedback is more about the teachers’ intent rather than student outcomes per se. Addressing this misconception is important to help teachers understand the difference between intent and outcomes so that they will not be unduly disappointed or frustrated when students do not respond at a level at which the feedback comments are intended. 13

Hattie and Timperley (2007) go on to describe six factors that mediate feedback effectiveness (p. 94). These aspects are summarized in the form of following questions: • Can students internalize feedback cognitively? • Can students self-assess? • Are students willing to seek and use feedback information? • Were students confident about initial work? • Can students understand how feedback explains poor performance and informs improvement? • Are students proficient at seeking help? Years later, the theory of student feedback literacy proposed by Carless and Boud (2018) appear to build on and extend the idea of these mediating factors of feedback effectiveness, as would be discussed later in Section 3. Secondly, some teachers seem to think that task level feedback and process level feedback are good enough for students’ learning in the long term. However, if we agree with Sadler’s (2007) definition of learning, what should be our ambition level of feedback eventually? The line of argument is this: To help students be able to do something they could not do previously well enough, self-level feedback only is not good enough. Minimally, task level feedback is needed. To help students be able to do it on demand, self-level feedback and task level feedback only may not be good enough. Minimally, process level of feedback is needed. Finally, to help students be able to do it independently of particular others, self-, task, and process level feedback only may not be good enough. I would argue that self-regulated level feedback is needed to increase students’ capacity to make use of opportunities for learning, and to ascertain that they have truly learned. 14

Thirdly, some teachers believe that the use of feedback aimed at these different levels will result in students’ greater receptivity of feedback, and lead to their immediate learning. Teachers with this misconception tend to get frustrated with students who fail to demonstrate the desired outcomes right away despite their well-intentioned attempts in using differentiated levels of feedback. In this regard, the meta-analysis of feedback literature by Kluger and DeNisi (1996) comes to mind. The study showed that while feedback improves student performance, its effects are very variable. Feedback effectiveness decreases as attention moves closer to the self-level feedback and away from the task. But the findings are further moderated by task characteristics that are still poorly understood. The lesson here is perhaps about teachers needing to persevere in their feedback practice, while at the same time figuring out what level and kind of feedback will best suit particular students at a certain time horizon to bring about their learning eventually. Such findings give direction to the kind of differentiated instruction work that is needed as the students’ learning progress becomes less homogenous. The key messages to address the misconceptions of the levels of feedback are summarized in Table 3. 15

Table 3 Key messages to address misconceptions of levels of feedback Misconception How it may be Key message to address misconception unproductive Many factors mediate the effectiveness of It’s about the Thinking that it’s about feedback, particularly student feedback quality student outcomes rather literacy. assurance than teachers’ intent may level of lead to teachers’ feedback. frustration when students do not respond at a level at which the feedback comments are intended. Task level and Thinking so limits teachers’ Moving students process level ambition level of feedback towards self-regulated feedback are and the potential of level feedback is needed good enough feedback to develop self- to increase their for students’ directed learners. capacity to make use of learning in the opportunities for long term. learning, and to ascertain that they have truly learned. It’s about Thinking so may lead to Teachers need to learning teachers’ frustration with persevere in their immediately. students who fail to feedback practice, while demonstrate the desired at the same time outcomes right away figuring out what level despite their well- and kind of feedback intentioned attempts in will best suit particular using differentiated levels students at a certain of feedback. time horizon to bring 16

about their learning eventually. 3. What constitutes student feedback literacy? Engaging students in feedback goes beyond giving correct answers. It involves teachers providing personalized feedback with actionable steps as well as school-wide processes to support quality feedback. To ensure meaningful and productive assessment feedback dialogues, student feedback literacy is arguably a necessary condition (Careless & Boud, 2018). Moreover, at the end of the day, it is only the students who can take action to improve their learning. Poorly-developed student feedback literacy can prove to be a barrier to the enactment of effective feedback practices and students’ uptake of feedback in the classroom and beyond school. So, what constitutes student feedback literacy? Carless and Boud (2018) define student feedback literacy as “the understandings, capacities and dispositions needed to make sense of information and use it to enhance work or learning strategies” (p. 1316). According to the authors, students who are feedback literate “appreciate their own active role in feedback processes; are continuously developing capacities in making sound judgments about academic work; and manage affect in positive ways” (ibid, p. 1318). Affect is understood to encompass students’ emotions and attitudes. The authors proposed that a combination of these three enablers: ‘understandings’, ‘capacities’, and ‘dispositions’ maximizes students’ potential to take action on feedback information. These inter- related aspects that serve as a framework underpinning student feedback literacy are shown in Figure 3. 17

Appreciating Making Managing affect feedback judgement (dispositions) (capacities) (understandings) Taking action Figure 3. Features of student feedback literacy. Adapted from Carless and Boud (2018) Drawing from the work by Carless and Boud (2018), the following statements in Table 4 may be helpful in helping students reflect on and self-assess themselves in the area of feedback literacy. 18

Table 4 Student feedback literacy: statements for student self-assessment. Appreciating • I understand and appreciate the role of feedback in feedback improving my work. • I understand and appreciate the active role I need to play in the feedback process. • I recognize that feedback information comes in different forms, for example, verbal and written. • I recognize that feedback information comes from different sources, for example, from teachers, peers, books, parents, self, or experience. • I can use technology to access, store, and revisit feedback. Making • I can tell if my work is up to par against certain judgement standards. • I can tell if the work of my peers is up to par against certain standards. • I participate actively in peer feedback discussions. • I refine my self-assessment to make more robust judgement about my own work and the work of my peers. Managing • I avoid being defensive when receiving critical affect feedback. • I am proactive in eliciting suggestions from peers or teachers, and continuing dialogue with them as needed. • I strive for continuous improvement based on the external feedback from others and my own sense- making of feedback. 19

Taking • I am aware that I have to take action in response to action feedback. • I am able to draw inferences from a range of feedback experiences to improve my work. • I have a repertoire of strategies for acting appropriately on feedback. Having students self-assess their own feedback literacy helps raise their awareness of the enablers that maximizes their potential to take action from feedback. Dialogue over students’ perception of feedback could reveal how some students may perceive feedback negatively as a judgement upon them. Data on the students’ self-assessment can also provide information for teachers to identify aspects that students struggle with, and offer insights into areas of priority for feedback practice. For example, if many students indicate that they have difficulty in telling if their work is up to par against certain standards, the teacher may want to prioritize class time to help students develop their understanding of a rubric or checklist, and offer opportunity for them to compare their work against the standard, as part of feedback practice. Section 4 gives more examples of such practices. 4. How do feedback literacy enabling practices look like in the classroom? The big idea of developing student feedback literacy is perhaps not just in helping students to clarify their understanding of standards and feedback comments, but also in helping them see the quality of their work through the teacher’s eyes. Sadler (1989) argues that “the indispensable conditions for improvement are that the student comes to 20

hold a concept of quality roughly similar to that held by the teacher, is able to monitor continuously the quality of what is being produced during the act of production itself, and has a repertoire of alternative moves or strategies from which to draw at any given point. In other words, students have to be able to judge the quality of what they are producing and be able to regulate what they are doing during the doing of it” (p. 121). Such feedback literacy enabling dialogue need not happen only after the task completion. In fact, such productive dialogue probably lies at the heart of all formative assessment processes. Feed up can take place, for instance, as part of introducing a formative assessment task, and as students are working on the task as part of in-task guidance. Two established learning practices, peer feedback and analyzing various exemplars of student work, are discussed below to illustrate ways in which feedback dialogue can be operationalized to enhance student feedback literacy. 4.1 Peer feedback How can the use of peer feedback develop students’ feedback literacy? Nicol, Thomson, and Breslin (2014) describe peer review as a “reciprocal process whereby students produce feedback reviews on the work of peers and receive feedback reviews from peers on their own work” (p. 102). Based on students’ accounts of their experience of constructing feedback reviews, the above study found that giving peer feedback develops students’ evaluative judgement about the peers’ work, and through reflection, about their own work. A feedback literate student demonstrates evaluative judgement which is understood as “the capability to make decisions about the quality of work of oneself and others” (Tai, Ajjawi, Boud, Dawson, & Panadero, 2018, p. 467). When 21

students apply criteria to explain their evaluative judgement, it shifts the locus of control of feedback processes from the teacher to students, and the shift can lead to them relying less on external feedback eventually (Nicol et al., 2014). McConlogue (2012) reported in her study of implementations of peer assessment that while students indicated a deeper understanding of quality in student writing, they complained that the marks given by peer assessors were not “fair” (p.113). The study by Sridharan, Tai, and Boud (2019) found that assessment bias towards low-performing students was more pronounced when peer marks counted towards the final grade. They concluded that while students are capable of judging the work of their peers accurately, consistently, and without bias, the use of marks as part of summative assessment inhibits their good judgement. The finding suggests that student feedback literacy could be inhibited by premature summative assessment practice. We should heed this finding by being cautious about introducing peer grading within peer summative assessment. Merely pairing students or putting them into groups would not engender productive peer feedback dialogue. In the sub-sections that follow, two practices, establishing protocols to focus peer response, and training students to peer review, are described as structures to support feedback dialogue amongst students in the context of formative assessment. 4.1.1 Focusing peer response: PQP protocol In the teacher workshops I conduct, a protocol I employ to have teachers give peer feedback to one another is “PQP”- Praise-Question-Polish (Neubert & McNelis, 1990). Teacher participants begin by first affirming the work of their peers by identifying something praise-worthy before 22

raising questions to clarify their understanding of the presented work. Examples of clarifying questions could be, “What do you mean when you wrote …? Were you trying to say …?” Finally, they offer a suggestion to improve the work presented. Such peer response protocols may also be helpful in focusing peer response with students and supporting the peer feedback process in the classroom. 4.1.2 Training students to peer review In the study by Min (2006) on the teaching of expository essay writing skills, the case teacher used a think-aloud method to model to students how to give feedback during the peer review process before having them do a paired peer review. The procedure for peer review involved a four- step procedure: clarifying writers’ intentions, identifying the source of problems, explaining the nature of problems, and making specific suggestions. Table 5 shows the procedure for peer review and the prompts provided to guide peer comments. 23

Table 5 Procedure for peer review. Procedure for Questions and comments to guide peer comments peer review When you are certain of When you are uncertain of what the writer is trying the writer’s intention to convey Clarifying Ask clarifying questions: Locate specific phrases or writers’ cohesive gaps, and ask intentions • Do you mean that… questions to prompt the • Are you saying … writer to explain or revise ideas: • I do not get this … • What do you mean by … Identifying Identify the problem and Refer to the sentences the source of explain why it is immediately preceding or problems problematic. following the problematic area and say what you Explaining the expect to read given the nature of surrounding contexts. problems Making Depending on the problems, specific • provide a specific definition of a misused phrase suggestions and a more appropriate one according to the context • remind the writer to discuss ideas from the same personal perspective • suggest a specific idea to enrich the content 24

As described above, teacher modelling to students on how to give feedback comments on written work, establishing a procedure for peer review, guiding students with prompts for peer comments, and establishing protocols for focusing peer response are different ways of supporting peer feedback dialogue. 4.2 Analyzing exemplars How can the use of various exemplars of student work develop students’ feedback literacy? The analysis of exemplars can help students get acquainted with academic standards and develop their capacities to make informed academic judgements (Carless & Chan, 2017). Getting students to analyze exemplars of different standards helps develop their tacit knowledge, understood as “aspects of knowledge that are difficult to transmit through speaking and writing” (Chong, 2019, p. 751-752). The dialogic use of exemplars in class can also promote the development of students’ evaluative judgement (Chong, 2019). The following sections discuss the different ways of using exemplars for student self-assessment and offer a description of a classroom procedure for organizing such feedback dialogue. 4.2.1 Different ways of using exemplars Exemplars can be used at different levels: self-marking, sound standard model (Cowan, 2002), and metacognitive reflection (Taras, 2010). At the self-marking level, students mark their own work against a rubric, checklist, or exemplar. A rubric can draw students’ attention to particular criterion-specific examples while an exemplar helps illustrate some or all the criteria in a rubric. In the sound standard model, which operates at a more relational thinking level (Biggs & Collis, 1982), students are 25

provided with not just a model answer, but exemplars of different standards. Students judge their own work against the exemplars. To do so, students would have to first judge and rank the exemplars in terms of standards, before judging where their work stand in relation to the exemplars. The metacognitive reflection model which extends the sound standard model operates at a more extended abstract level (Biggs & Collis, 1982), by having students write down their justifications of why they graded themselves as such by giving a breakdown of their strengths and weaknesses. The teacher then gives feedback on the student work and their reflection. Of the three ways of using exemplars, the self-marking model is the most common way of using exemplars in the classroom. Taras (2010) however argues that it is a weaker model of self-assessment when placed on a continuum of stronger and weaker models of enabling student decision- making in assessment. The author suggests that the self-marking model can be made stronger by allowing students to participate in deciding the assessment criteria at the beginning of the task assignment. Compared to the self-marking model, the sound standard model is stronger in developing student feedback literacy as it requires students to make a judgment of the exemplars of different standards. As for the metacognitive reflection model, the teacher provides feedback on both the work and the self-assessment. The feedback helps students refine their initial self-assessment to make a robust judgement of their work. Together, these three ways of using exemplars present increasingly stronger ways of developing student feedback literacy. 26

4.2.2 Procedure for organizing the analysis of exemplars The study by Carless and Chan (2017) describes a procedure for organizing the analysis of exemplars in class. The procedure for organizing such feedback dialogue is presented in Table 6. Table 6. Procedure for organizing the analysis of exemplars. When Procedure Pre- Students analyze the strengths and weaknesses of the lesson exemplars, and submit their analysis to the teacher. They make their judgement with reference to assessment criteria established in the task rubric and/or other success criteria co-constructed or agreed by the class. During The teacher elaborates the rationale for analyzing the exemplars. lesson Students discuss their accounts of the strengths and weaknesses of the exemplars in pairs. Students elicit views from classmates, and summarize them in the form of mini-presentations. At the Students submit an exit slip reflecting on their learning end of from the discussion of exemplars. the lesson Dialogue over analyzing exemplars is said to be productive when “multiple viewpoints are explored to facilitate the co-construction of reasoned judgements about the exemplars” (Carless & Chan, 2017, p. 939). The authors listed the following teacher actions needed to facilitate such productive exemplars dialogues: 27

• preparation of key potential messages prior to the exemplars discussion • flexible responses to students’ evolving thinking • skill in making use of student views to build shared understandings about quality • skills related to elicitation, including use of wait time, attentive listening and the use of non-evaluative discourse. • going beyond surface features of exemplars, and indicate how they relate to wider intended learning outcomes (p. 939-940) In our effort to develop student feedback literacy, it may be helpful to borrow from the corporate world, the idea of a ‘capital investment’. The term ‘capital investment’ refers to money allocated to purchase long- term assets with the expectation of increasing the capacity to generate more income over time as opposed to money spent on day-to-day business operations. When first introducing peer review processes in class, a ‘capital investment’ of time and effort is clearly needed to train students to peer review, set up useful classroom routines, and provide students opportunities to practise feedback dialogue. This may seem to slow down the pace of lesson, but over the course of time, the ‘capital investment’ would begin to pay off as students become more familiar with established routines and develop greater confidence in giving and receiving peer feedback as well as more skill and will in making sense of and acting on teachers’ feedback. This is a ‘win-win’ situation for all, as teachers attempt to empower students to take up their responsibility of learning by a ‘slow release’ process (Fisher & Frey, 2013). This section attempts to show how having students peer review and analyze exemplars help them develop capacities in making judgement to assess their work and the work of their peers against certain standards. Such experiences also enable them to refine their self-evaluative capacities to make more robust judgement. The ideas of engaging 28

students in peer review and orchestrating exemplars dialogue are underpinned by a social constructivist learning paradigm which views that learning as creating meaning from experience, and is negotiated through interaction with others. The characteristics of these student feedback literacy enabling practices, related to social constructivism, are premised on the active involvement of students in the assessment feedback process. The extent to which teachers see the relevance of these practices for Singapore classrooms and in different subject disciplines is often dependent on the teachers’ view of what constitutes learning and feedback processes, their perceived time pressures, and the co-operation of students. 4.3 Putting it together: An example of an extended/expanded assessment feedback model In the courses I teach, a closing activity to move teachers from multi- structural thinking to relational/extended abstract thinking (Biggs & Collis, 1982), is to have teachers put together the ideas of feed up, feed back, and feed forward in their own version of an extended assessment feedback model in the context of their subject discipline situated in the teaching of particular skills or concepts. In the final section of this chapter, such an example of a feedback model is presented to illustrate how feedback can be practised as dialogue in enabling student feedback literacy in the Singapore classroom. The feedback model was developed by a group of Senior Teachers in putting together the ideas they have learnt from a three-day course on the Theory and Practice of Assessment as part of the Teacher Leaders Programme. Table 7 describes how feedback dialogue is orchestrated in the context primary science teaching of process skills, specifically in teaching students to use the Claim-Evidence-Reason (CER) protocol in answering 29

open-ended questions. A gradual release of responsibility was also adopted in the scaffolding approach (Fisher & Frey, 2013). Table 7. Extended assessment feedback model in Science teaching Stage Procedure Preparatory Teacher defines CER, and explains when to apply guidance the strategy and the criteria of success. In-task Teacher demonstrates the use of C, E, R (I do) in guidance analyzing the question requirements and planning to write the answer. Students do a guided practice question with the teacher and indicate C, E, and R in their answers. (We-do together) Students do a question on their own (You-do) Teacher collects student work to identify different standards of student work as exemplars. In the next lesson, teacher gets students to rank exemplars based on standards/rubrics. Students self-assess their work against the rubrics and exemplars. They work on identified gaps together with others in same-level groupings. Performance Students work on practice 2 to assess their feedback transfer of learning. Feedback is given in verbal or written form in summative fashion. 30

The feedback model above brings together a number of ideas presented in this chapter. Where am I going? (Feed up): It begins with engaging students during the preparatory guidance stage in developing a concept of the standards to aim for. During the in-task guidance stage, dialogic feedback interaction is crucial in helping students clarify their understanding of the referenced standards, and has been well-designed by employing a gradual release of responsibility approach (Fisher & Frey, 2013) in scaffolding such interaction dialogue through a I-do and We-do-together phase followed by a You-do phase. How am I going? (Feed back): The use of exemplars was used in helping students compare their current work with good performance. The analyzing of exemplars follows a sound standard model (Cowan, 2002) whereby students, in pairs or groups, evaluate exemplars of different standards. They rank them based on the referenced criteria before self-assessing their own work. Students have the responsibility to do sense-making of feedback, from peers and from the analysis of exemplars, on their own to self-assess their own work, and afterwards, take appropriate action, as discussed below. Where to next? (Feed forward): To enable students to apply feedback from their sense-making, they have an opportunity to submit an improved version of their work after discussions with peers in the same-level groupings. Students are also assigned a parallel task to assess their transfer of learning in practice 2. Reserving teacher feedback till after practice 2 is intended to have students focus on thinking about improving their work from an informed position using peer feedback and self-assessment, and without the 31

undue power pressures of teacher’s comments and the emotional baggage of grades. In summary, the various ideas on engaging students in feedback presented in this chapter are premised on an expanded conceptualization of feedback as an interactive dialogue between students, their peers, and teacher over formative feedback information, and involving students’ individual sense-making internal dialogue. Developing student feedback literacy is key in sustaining productive feedback dialogue. Students’ appreciation of feedback, their capacities in making judgement, and their dispositions in managing affect are believed to work together in maximizing their potential in the uptake of feedback in improving their learning. Peer review and analyzing exemplars, when done well, can support the development of student feedback literacy through increased opportunities for students to dialogue over the concept of referenced standards, to compare their current level performance against the standards, and to engage in taking appropriate actions in moving towards the standards of good performance. To conclude, students’ feedback literacy is arguably a key competency in their pursuit of sustainable learning. Engaging students in feedback and assessment to develop their capacity for self-assessment, for reflective and self-directed learning should be a priority area for all teachers in the Singapore classroom. Teachers with the support of middle and school leaders as well as school-wide support structures are more likely to focus on assessment for learning and employ feedback as a dialogic practice more often and be proficient at it over time. These are definitely assessment competencies that teachers would need to hone in order to 32

well prepare our students to meet the challenges of an increasingly complex world. Points to Ponder • What is your conception of feedback and how does it shape your enactment of feedback practice? • What helps, and what hinders, your feedback practice from being more dialogic with students? • Which features of student feedback literacy do your students demonstrate more difficulty with, and how can you prioritize that in your feedback practice? 33

References Beaumont, C., O’Doherty, M., & Shannon, L. (2011). Reconceptualising assessment feedback: a key to improving student learning?. Studies in Higher Education, 36(6), 671-687. Biggs, J.B., and Collis, K.F. (1982). Evaluating the quality of learning-the SOLO Taxonomy (1st ed) New York: Academic Press. Boud, D. (2000). Sustainable assessment: rethinking assessment for the learning society. Studies in continuing education, 22(2), 151-167. Carless, D. (2016). Feedback as Dialogue. Encyclopedia of Educational Philosophy and Theory 1–6. Accessed http://link.springer.com/referenceworkentry/10.1007/978-981-287- 532-7_389-1 Carless, D., & K. K. H. Chan. (2017). Managing dialogic use of exemplars. Assessment & Evaluation in Higher Education, 42(6), 930–941. Carless, D., & Boud, D. (2018). The development of student feedback literacy: enabling uptake of feedback. Assessment & Evaluation in Higher Education, 43(8), 1315-1325. Chong, I. (2018). Interplay among technical, socio-emotional and personal factors in written feedback research, Assessment & Evaluation in Higher Education, 43(2), 185-196, Chong, S. W. (2019). The use of exemplars in English writing classrooms: from theory to practice, Assessment & Evaluation in Higher Education, 44(5), 748-763, Cowan, J. (2002). Plus/minus marking: A method of assessment worth considering? York, UK: The Higher Education Academy. 34

Desired Outcomes of Education. (2018, October 11). Retrieved from https://www.moe.gov.sg/education/education-system/desired- outcomes-of-education Fisher, D., & Frey, N. (2013). Gradual release of responsibility instructional framework. IRA Essentials, 1-8. Gibbs, G., and Simpson, C. (2004). Does your assessment support your students’ learning?. Journal of Teaching and Learning in Higher Education 1(1), 3–31. Hattie, J., & Timperley, H. (2007). The power of feedback. Review of Educational Research, 77(1), 81-112. Kluger, A. N., & DeNisi, A. (1996). The effects of feedback interventions on performance: A historical review, a meta-analysis, and a preliminary feedback intervention theory. Psychological Bulletin, 119(2), 254-284. Min, H. (2006). The effects of trained peer review on EFL students’ revision types and writing quality. Journal of Second Language Writing, 15, 118-141. McConlogue, T. (2012). But is it fair? Developing students’ understanding of grading complex written work through peer assessment. Assessment & Evaluation in Higher Education, 37(1), 113- 123. Neubert, G. A., & McNelis, S. J. (1990). Peer response: Teaching specific revision suggestions. The English Journal, 79(5), 52-56. Nicol, D., Thomson, A., & Breslin, C. (2014). Rethinking feedback practices in higher education: a peer review perspective. Assessment & Evaluation in Higher Education, 39(1), 102-122. 35

Ramaprasad, A. (1983). On the definition of feedback. Behavioral science, 28(1), 4-13. Sadler, D. R. (1989). Formative assessment and the design of instructional systems. Instructional science, 18(2), 119-144. Sadler, D. R. (2007) 'Perils in the meticulous specification of goals and assessment criteria', Assessment in Education: Principles, Policy & Practice, 14(3), 387-392 Singapore Curriculum Philosophy. (2018, October 11). Retrieved from https://www.moe.gov.sg/about/singapore-teaching- practice/singapore-curriculum-philosophy Sridharan, B., Tai, J., & Boud, D. (2019). Does the use of summative peer assessment in collaborative group work inhibit good judgement?. Higher Education, 77(5), 853-870. Tai, J., Ajjawi, R., Boud, D., Dawson, P., & Panadero, E. (2018). Developing evaluative judgement: enabling students to make decisions about the quality of work. Higher Education, 76(3), 467-481. Tan, K. H. K. (2014). Assessment feedback practices for enhancing learning. In W. S. Leong, Y. S. Cheng, & K. H. K. Tan (Eds.). Assessment and Learning in Schools (pp. 129-139). Singapore, Pearson Education South Asia Pte Ltd. Taras, M. (2010). Student self-assessment: processes and consequences, Teaching in Higher Education, 15(2), 199-209. Thurlings, M., Vermeulen, M., Bastiaens, T., & Stijnen, S. (2013). Understanding feedback: A learning theory perspective. Educational Research Review, 9, 1-15. 36

Research • Tan, K. H. K. (2011). Assessment for learning in Singapore - Unpacking its meanings and identifying some areas for improvement. Educational Research for Policy and Practice, 10(2), 91-103. Abstract This article examines the meanings and impact of ‘Assessment for Learning’ initiatives in schools against the back drop of assessment reform in Singapore since 1997. It is argued that Assessment for Learning’ is understood in different ways, and these different meanings do not always benefit students’ learning. The different meanings of ‘Assessment for Learning’ in Singapore are unpacked, and three areas for improvement for Assessment for Learning are suggested—clear standards for effective feedback practices, assessment for sustainable learning and assessment for integrating holistic learning. • Tan, K. H. K. (2013). A framework for assessment for learning: Implications for feedback practices within and beyond the gap. ISRN Education: 1-6. Abstract Three recurring emphases in the literature on formative assessment are (a) the importance of assessment design in prompting and sustaining students’ learning, (b) giving students feedback that enables them to improve their work, and (c) clarity of standards to articulate the gap between past and desired performances. Much has been written on how each of these is important in designing and using assessment for learning. But there is considerably less attention (if any) on how these emphases should confluence each other. In this paper, I propose a framework 37

for assessment for learning (AfL) wherein assessment standards (vertical axis), task design (horizontal axis) and feedback practices (incline) form a “triangle of practices” that construct the “space” for enhancing learning. Implications for formative assessment to extend learning beyond its triangulated frame are discussed. • Tan, K., & Wong, H. (2018). Assessment Feedback in Primary Schools in Singapore and Beyond. In A. Lipnevich & J. Smith (Eds.), The Cambridge Handbook of Instructional Feedback (Cambridge Handbooks in Psychology, pp. 123-144). Cambridge: Cambridge University Press. doi:10.1017/9781316832134.008 Abstract This study attempts to answer the question: Can feedback in primary schools enhance students’ “independent” learning, and at the same time assist students to achieve requisite learning outcomes for high-stakes examinations? • Butler, R., & Nisan, M. (1986). Effects of no feedback, task-related comments, and grades on intrinsic motivation and performance. Journal of educational psychology, 78(3), 210. Abstract This study was designed to test the hypothesis that intrinsic motivation would be maintained after receipt of nonthreatening, task-related evaluation and undermined after repeated non- receipt of feedback or receipt of controlling normative grades. Nine classes comprising 261 sixth-grade pupils were randomly assigned to one of these three feedback conditions and were given two interesting tasks, one quantitative and one qualitative, on three sessions over 2 days. The manipulation was applied after Sessions 1 and 2, and no feedback was expected or received after Session 3. 38

Experimental measures consisted of Session 3 performance scores and of the results of a questionnaire, given after Session 3, which tapped interest and patterns of attribution of success and effort. The results confirmed the hypothesis and revealed significant group differences in intrinsic motivation as reflected in both performance and attitudes. • Butler, R. (1988). Enhancing and undermining intrinsic motivation: The effects of task-involving and ego-involving evaluation on interest and performance. British Journal of Educational Psychology, 58, 1–14. Abstract This study was designed to test the hypothesis that intrinsic motivation will be differentially affected by task-involving and ego- involving evaluation, and that provision of both kinds of evaluation will promote ego-involvement rather than task-involvement. Twelve class of fifth and sixth grade pupils were randomly assigned to one of the three feedback conditions. Pupils receive either ego- involving numerical grades or task-involving individual comments or both after performing interesting tasks, one convergent and one divergent, on each of two sessions. Interest and performance for 132 randomly selected pupils of high or low school achievement were measured at pre-test, during the manipulation and at a third session, where no further evaluation was anticipated. As hypothesized, interest and performance on both tasks at both levels of school achievement were highest after comments, both when further comments were anticipated and when they were not. Grades and grades plus comments had similar and general undermining effects on both interest and performance, although high achievers who received grades maintained high interest and 39

convergent thinking when further grades were anticipated. These results are discussed in terms of the contribution of the distinction between task and ego-involvement to further understanding of intrinsically motivated activity. • Kluger, A. N., & DeNisi, A. (1996). The effects of feedback interventions on performance: A historical review, a meta- analysis, and a preliminary feedback intervention theory. Psychological Bulletin, 119(2), 254-284. Abstract A meta-analysis of feedback literature (607 effect sizes; 23,663 observations) showed that feedback improved performance with an average effect size of 0.41; however feedback effects are very variable. The authors found that viewing feedback as punishment and reward did not account for the results in early feedback studies. This led them to propose a preliminary FI theory (FIT), the central assumption of which is that FIs change the locus of attention among three general and hierarchically organized levels of control: task learning, task motivation, and meta-tasks (including self-related) processes. The results suggest that FI effectiveness decreases as attention moves up the hierarchy closer to the self and away from the task. These findings are further moderated by task characteristics that are still poorly understood. • Hattie, J., & Timperley, H. (2007). The power of feedback. Review of Educational Research, 77(1), 81-112. Abstract 40

Feedback is one of the most powerful influences on learning and achievement, but this impact can be either positive or negative. Its power is frequently mentioned in articles about learning and teaching, but surprisingly few recent studies have systematically investigated its meaning. This article provides a conceptual analysis of feedback and reviews the evidence related to its impact on learning and achievement. This evidence shows that although feedback is among the major influences, the type of feedback and the way it is given can be differentially effective. A model of feedback is then proposed that identifies the particular properties and circumstances that make it effective, and some typically thorny issues are discussed, including the timing of feedback and the effects of positive and negative feedback. Finally, this analysis is used to suggest ways in which feedback can be used to enhance its effectiveness • Lipnevich, A. A., & Smith, J. K. (2009). Effects of differential feedback on students’ examination performance. Journal of Experimental Psychology: Applied, 15(4), 319. Abstract The effects of feedback on performance and factors associated with it were examined in a large introductory psychology course. The experiment involved college students (N = 464) working on an essay examination under 3 conditions: no feedback, detailed feedback that was perceived by participants to be provided by the course instructor, and detailed feedback that was perceived by participants to be computer generated. Additionally, these conditions were crossed with factors of grade (receiving a numerical grade or not) and praise (receiving a statement of praise or not). The task under consideration was a single-question essay examination administered at the beginning of the course. Detailed 41

feedback on the essay, specific to individual’s work, was found to be strongly related to student improvement in essay scores, with the influence of grades and praise being more complex. Generally, receipt of a tentative grade depressed performance, although this effect was ameliorated if accompanied by a statement of praise. Overall, detailed, descriptive feedback was found to be most effective when given alone, unaccompanied by grades or praise. It was also found that the perceived source of the feedback (the computer or the instructor) had little impact on the results. These findings are consistent with the research literature showing that descriptive feedback, which conveys information on how one performs the task and details ways to overcome difficulties, is far more effective than evaluative feedback, which simply informs students about how well they did. 42

feedback on the essay, specific to individual’s work, was found to be strongly related to student improvement in essay scores, with the influence of grades and praise being more complex. Generally, receipt of a tentative grade depressed performance, although this effect was ameliorated if accompanied by a statement of praise. Overall, detailed, descriptive feedback was found to be most effective when given alone, unaccompanied by grades or praise. It was also found that the perceived source of the feedback (the computer or the instructor) had little impact on the results. These findings are consistent with the research literature showing that descriptive feedback, which conveys information on how one performs the task and details ways to overcome difficulties, is far more effective than evaluative feedback, which simply informs students about how well they did. 42