Important Announcement
PubHTML5 Scheduled Server Maintenance on (GMT) Sunday, June 26th, 2:00 am - 8:00 am.
PubHTML5 site will be inoperative during the times indicated!

Home Explore Workshop NexGen Science

Workshop NexGen Science

Published by dgage, 2019-06-07 10:27:00

Description: Workshop NexGen Science

Search

Read the Text Version

23 coherence of ideas is important, and it makes sense to call on students who have relevant contributions at the moment. Handing off In “handing off,” the student who spoke last calls on the next speaker. This technique gives the students more control over the discussion, and makes it more likely students will talk directly to one another, linking their contributions and building on one another’s ideas. This makes the discussion feel more “kid-run” and sometimes motivates more students to want to contribute their ideas. There are a couple of downsides to this strategy. It gives you less control over who speaks and in what order. As a result, turns at talk might become less equitably distributed. It also can become a distraction if students orient to speakers based on friendship, taking their time looking around the group, trying to make up their mind about whom to choose. If you do turn this responsibility over to the students, it is important to establish the norm of equitable participation. They should understand that it is important that those who have something important to contribute get a chance to talk and that no one should dominate the conversation. Jumping in (or Popcorn) This strategy allows anyone who wants to talk to do so, without raising their hand, as long as the current speaker has finished. In addition, the next speaker MUST link their contribution to the previous speaker or something that has occurred earlier in the conversation. This has some of the same benefits of handing off in that it gives the students a sense of agency and ownership over the discussion process. The downside to this approach is that it requires a certain amount of restraint among the students if two or more students start to talk at once. They have to be able to negotiate -- on their own -- a turn-taking order. This approach tends to favor the dominant talkers, as well as those students who can formulate their ideas quickly. For this strategy to work (and not promote highly inequitable participation), you have to discuss with students the importance of giving quieter students a chance to enter the conversation. For example, if two students start to talk at once, the rule should be that the student who has participated less often should go first. The Talk Ball This is the term we’ve given to a strategy where the teacher designates an object that can easily be passed around from speaker to speaker. The object indicates who has the floor. It serves as a pseudo-microphone, signaling that only the person with the object has the right to speak, and needs to be heard. We have found that a soft object, such as a small nurf ball or foam block works well because it is easy to toss, easy to catch, and impossible to hurt anyone with. We have found that the “talk ball” makes it easy to introduce and enforce the “one person talks at a time” rule in situations where the students find it very hard not to talk over one another. We also find that students like the sense of having the floor (often holding up the ball if others start to speak and saying, “I have the floor”), and that the process of passing the ball slows the conversation down, giving everyone a bit more time to think. We have seen this strategy work well with students as young as pre-schoolers and as old as high school students. However, with older students, it’s possible for the talk ball to become a distraction if the students start to throw the ball hard or have to chase after it if dropped.

24 Round-Robin Talking This strategy is used to get everyone, in turn, to contribute their ideas to the conversation. A question is posed and everyone has to go on line with their best thinking. It’s best used when a good deal of talk has already occurred and it is time to take stock of the group’s viewpoints. For example, in science, the strategy can be used after an experimental apparatus has been discussed (“What will happen when I pump this volleyball with 20 pumps of air? Will it be heavier, lighter, or weigh the same?”). Before determining the outcome, the round-robin strategy can be used. Everyone has a chance (and the obligation) to go public with their prediction and reasoning. The value of this strategy is that everyone has a chance to talk and even shy, ELL, or struggling students are expected to contribute. However, because this occurs after a good deal has occurred, and it is fine to use others’ ideas, the entry threshold is low. The Gender Rule This strategy involves alternating boys and girls and can be used selectively and for a relatively short period of time in a discussion. It’s introduced simply by saying something like, “OK for now, let’s alternate boy, girl, boy, girl — just to make sure everyone’s getting a chance to talk. Once one person has spoken, the teacher (or the last student to speak) must select someone from the other gender. This strategy is particularly helpful if a small group of boys or girls seems to dominate the conversation. We have found that in math and science, especially in the upper grades, girls are sometimes more reticent to volunteer to speak. And we have found consistently that the girls LOVE the gender rule, even if it slows the conversation down a bit, in selecting the next speaker. Threats to equitable participation In many classrooms, over a long series of discussions, one does find enduring patterns of inequality. Some student consistently dominate the conversation and the same few students rarely participate. Some of the students have begun to opt out of the conversation all together. What are the causes of these patterns of inequitable participation and what can be done to counter them? Some causes of inequity lie in student characteristics: students are shy, or domineering; confident or insecure. We have to find ways to balance these features. Other causes of inequity lie in us as teachers. We need to reflect on these, explore them, and address them. In every case, it is important to act consistently to maintain fairness. Often where we see enduring patterns of inequitable participation, we find that the teacher has simply failed to establish norms for academically productive talk at the outset. Over time, a pattern has taken hold in which the “talky kids” or the so-called “smart” kids dominate. This can become a case where the “rich get richer.” Those kids who are good talkers to begin with get more practice and more support. Gradually others stop trying to get a turn, stop engaging deeply with the ideas, and have less to actually contribute. Whatever the reason, the solution here is simply to establish norms and hold the students accountable to them, following the guidelines set forth in this chapter. Quiet and shy students Research had shown that a student who doesn’t talk in a given discussion CAN be engaged and learning at high levels. Some have called this phenomenon

25 “silent participation.” In these cases, the student can hear what others are saying and is IN the conversation as an engaged listener, silently identifying with speakers who take up positions he or she agrees with. At the same time, it is important that — over time — students have a variety of opportunities to formulate complex ideas (even if merely repeating a complex idea that someone else has originated). It is important that students get a chance to explicate and elaborate their ideas and do the hard but rewarding work of making sure that their ideas are clear to others. For students who are by nature very shy, for English language learners, or for students new to academic discussions, a good deal of scaffolding and support (from the teacher and peers) will be required to help them formulate arguments and explanations in a way that others can hear and understand. But all students must get the time and support they need to publicly express complex ideas. Only then can others build upon or challenge their ideas. This in turn will motivate these students to develop their ideas further. Students who tend to dominate Some students are by nature comfortable and confident talking in front of a group. They may come from families where this kind of talk is practiced regularly at mealtime, during car trips, or bedtime. Some kids are simply particularly knowledgeable about a topic under discussion. And finally, some students, no matter what their background or preparation, are simply gifted in the verbal realm. In these cases, there are a number of ways to promote equitable participation without squelching these willing and gifted participants. First of all, these students can provide interesting material for others to repeat in their own words. “Wow, did everyone hear what Juan just said? Does anyone think they understood Juan’s point well enough to put it into their own words?” This kind of teacher move has many benefits. It credits Juan with a substantive contribution. It models complex thinking for others and gives someone else a chance to put this complex idea into words publicly. (This is often a helpful way to support ELL students in acquiring academic English.) Finally, it gives everyone a second chance to hear and think about a complex idea, serving as a kind of “wait time.” Another strategy to use is to ask for a contribution from someone “who hasn’t yet had a chance to speak.” Still another is to invoke the “gender rule” which asks for a boy to speak after a girl. This gives the students who are anxious to speak practice waiting for others to take a turn. English Language Learners Sometimes, students are reluctant to participate in a whole group discussion because they lack proficiency with academic English. They may be learning English as a second or third language or they may lack experience engaging in academic conversations. For these students, it is highly beneficial simply to listen in on and participate in academically productive talk discussions. These student require practice in an environment that supports them to try, to take a chance, even if only to repeat what someone else has said, and to be given wait time or a chance to “say more about that.” If a reluctant or struggling speaker has offered a substantive contribution (even if poorly formulated), it is helpful to ask someone if they can put that idea into their own words. Often the next student will expand or clarify the thought and at the same time credit the original student as having made a substantive contribution to the conversation. The reluctant or ELL student will

26 also then be hearing a revoicing of their contribution, perhaps in more grammatical English, a more academic register, or in a more elaborated form. This then provides additional linguistic input that is highly accessible for that student, because it expands or rephrases their own original idea. This serves as a kind of “linguistic amplification” rather than simplification (Walqui, 2000), and supports language acquisition within the context of an academically rigorous curriculum. 3 Students with conflicting home-based norms Some students are reluctant to participate vocally in large group discussions because it is not something they do regularly in their home or home community. Among some Native American groups, for example, it is considered unseemly to set yourself apart from others by taking center stage and speaking out in a large group. In some communities, it is considered rude or arrogant to be fully explicit about your ideas, to state the obvious, so to speak. If you did that at home, you might be chastised for “hitting the other guy over the head with your point.” Here, stating the obvious is considered bad form because you are acting as if your audience is too stupid to read between the lines, or is not listening carefully. These home-based norms are in direct conflict with the norms for academically productive talk, where students are expected to take the floor and fully explicate their reasoning. This means stating a claim and providing evidence for that claim, as well as providing an explicit link between the evidence and the claim. In practicing academically productive talk, we are asking students to talk in ways they would not be expected to talk at home. How can you work with students from a wide variety of home communities which will have many different norms for public speaking? First of all, it is helpful simply to acknowledge that reluctance to speak in a group does NOT indicate lack of intelligence or a dearth of good ideas. It might well be a cultural norm or value that creates harmony in group discussion in one’s home community. But a willingness to recognize or acknowledge that cultural differences exist will only get you so far. What can you do to get these kids to participate more? Some students will participate more vocally in small group settings because they do not feel that they are calling attention to themselves or acting like they are more important than others. For these students, the question is how can you help them move from contributing in a small group to contributing in a whole group setting. One strategy is to encourage them within a large-group context simply to repeat what someone else has said, in order to help clarify a classmate’s idea. In this situation, they may feel more comfortable as they would be seen to be helping others, rather than tooting their own horn. With students who are reluctant to fully explicate their ideas, because of a cultural norm that prohibits stating the obvious and potentially insulting a listener, a number of teacher moves can be very helpful. “Why do you think that?” “What’s your evidence?” or “How does that evidence relate to your claim?” encourages students to unpack their reasoning. Gradually, these students 3 Walqui, A. (2000). Access and Engagement: Program Design and Instructional Approaches for Immigrant Students in Secondary Schools. Washington D.C.: Center for Applied Linguistics & Delta Systems.

27 begin to do so spontaneously. Inequities in our own expectations Sometimes the root cause of inequitable participation is subtler and harder to talk about. Teachers may unwittingly be expecting less from certain students because of deeply held prejudices or misunderstandings about cultural differences. If teachers act on these differential expectations and ask less of certain students, they may unwittingly create fewer opportunities for these students to participate and practice academically productive talk. If that happens, the teacher will have created a self-fulfilling prophecy. “My boys are more scientifically inclined and better at math than my girls. That’s why they have so much more to say in math discussions than the girls.” “Some of my kids have a hard time reading and discussing books because they don’t speak Standard English.” The trouble with these kinds of differential expectations (gender bias or differential expectations for children of different ethnic or social class backgrounds) is that it’s hard to recognize bias in oneself. To some extent we may not even be aware we work with these differential expectations. Or if we are aware of differential expectations, they may not seem like bias at all; they seem like simple, self-evident truth! What can be done in cases like this? First of all, it is helpful to pay attention to one’s assumptions and expectations for children from a particular ethnic, class, or gender group. There is a great deal of research that suggests that decreased expectations of children often lead directly to decreased opportunities to learn for these children. For this reason, it is helpful to look closely at one’s own practice – of calling on children or scaffolding their participation. Sometimes it is helpful to tape record a discussion and simply look closely at who talks, and for how long. It is also helpful to look at how much support for talk (“Say more about that.” “Who can repeat…?”) different children receive. We all have ideas about individual children and make efforts to individualize our support. But if we have ideas about girls as a group, or African American children, or poor children, or English language learners, and those ideas lead us to expect less from these children in certain contexts, it is worthwhile to look closely at how these children participate in discussions. It may turn out that WE are part of the problem and that we can intervene strategically to create more equitable opportunities for participation. Calling on ‘reliable’ talkers One last cause of inequitable participation is a common problem that many of us have, or have had at one time or another. We call on the same students again and again, students who we know reliably say something “smart.” Why do we do this? Because leading discussions that support academically productive talk in science is really hard to do. And being human, we have a hard time with the uncertainty and unpredictability of these discussions, where so much of the content is provided by the students themselves. We feel the need to stick with reliable right-answerers to avoid the discomfort of wait time. (What if I wait and no one else raises their hands? How long should I wait?) Another reason we may call too much on the ‘reliable’ talkers is that we are uncertain about how to deal with wrong answers. (Should I correct a wrong answer? Let it hang in the air and see if anyone else notices? Will a student feel humiliated if an incorrect answer is exposed publicly?)

28 We feel the need to control the direction of the conversation, so that it proceeds in a coherent manner. (I don’t want the majority of the students to get confused if someone proposes an off- base idea.) We fear looking foolish or ignorant if a student comment takes the conversation off in a direction we are unprepared for. (I’m not sure where this is going. What if he knows more about this subject than I do???) And many of us worry about “coverage.” We fear wasting time while kids struggle to build meaning. (The state tests are coming up…) All of these concerns are legitimate and all of us entertain them from time to time as we attempt to lead discussions. Nonetheless, it is important to trust the process of opening up the conversation to all participants while using teacher moves to maintain the academic rigor and focus of the conversation. How do I decide whether or not my discussions are equitable? Because the goal is not to make sure that every student speaks in every discussion, it is not a good idea to assess the degree of equity in your discussions after a single session. Rather, it makes sense to look over a stretch of discussions, over the course of a week or two. It is helpful to keep track of participation in some way, and then to look over your “results” during a number of sessions. Some teachers find it helpful to use a seating chart (a simple drawing of students’ names in the seating arrangement they are in for that discussion). Every time a student speaks, you can make a tick mark next to their name, or even jot down a word or two to help you remember their contribution. Seating charts serve a number of purposes in addition to providing you with a rough tally of contributions and contributors. It shows the students that you are very serious about their ideas and want to think hard about them and remember them. It helps to slow the conversation down a bit, and give you a moment to collect your thoughts, as you are jotting down a note about what was just said. Another advantage of taking notes and using a seating chart is that it takes your gaze off of the students and makes it easier for you NOT to respond with an evaluative comment (such as, “yes, that’s right,” “no, that’s not quite right,” “great point,” “brilliant idea,” or even “OK.”). These comments often orient the students to correctness of the idea, putting yourself in the role of authority. This motivates students to come up with an answer that YOU will want, rather than allowing the students to grapple with the ideas themselves, learning to make their own judgments about how compelling or convincing some ideas are compared to others. If you do use a seating chart (or even a listing of names with tick marks), you can look over a set of these records (from 5 or so discussions) and ask yourself if there are some obvious patterns or imbalances. Are some students not talking at all? Are some students talking way more than others? Are the boys talking much more than the girls? Are ELL students not contributing?

29 5. Can I really do this? The main worries that teachers have about their own capacity to orchestrate academically productive talk in science tend to be expressed in two questions: Can I make progress in the academic content while using this kind of talk? Can I manage student interactions in ways that will work for all my students? Why should these worries emerge? The main reason is that when you use discussion you are opening the door to unpredictability. In a lecture or direct instruction format, you control all the ideas and questions. When you allow students to explain their reasoning, and to talk to one another in partner talk, and to ask questions that occur to them, you are introducing a source of uncertainty. This uncertainty is exciting but at the same time it may provoke anxiety. Will you be able to stay on track in terms of content? What if students want to talk about ideas that are incorrect? What if the unpredictability and open character of the talk brings out behaviors in students that are undesirable? Once you have begun to use academically productive talk strategies, you will have a better sense of how to manage these issues, and the anxieties you have now will fade. As you start the process, there are several things to keep in mind about your own preparations. Starting slow: build confidence in using talk moves Before you begin ‘officially’ using academically productive talk in your class, you can build your confidence and that of your students by starting to use some of the teacher moves yourself. Many teachers have told us that the first time they consciously used revoicing, asking students to repeat, and so on, they found themselves distracted and discombobulated, unable to focus full attention on the academic content of students’ contributions. This makes sense: any time we do something new it increases our overall processing load. Therefore, we think the best way to begin is by starting slow: practice using a few of the talk moves even before you introduce the full set of talk norms to your students. To do this, you need to plan an activity in which you will be asking your students substantive and open-ended questions on a topic that you are fully confident about. For example, you may revisit a previous lesson and ask them how they solved a complex problem, or how they interpret a piece of text, or how they explain or describe a complex procedure. In short, you must find a somewhat familiar activity in which the questions call for more than a single word or yes/no response. One of the easiest ways to start is by practicing the teacher revoicing move. Whenever you ask a question and a student answers it with more than a minimal answer, revoice their contribution, making sure to leave room for them to agree or disagree with your revoicing. This is particularly helpful in building your confidence that you can deal with unintelligible contributions. It also gets students used to the fact that you will respond to their statements with full attention, taking

30 them seriously and following up. When a student provides only a minimal answer, you can begin to encourage them to go further by saying something like “I’m not sure I totally understand. Can you say more?” If they refuse to say more, you can tell them in a friendly way that you will give them time to think more and will come back to them. Then make sure that you do return to them later in the same class session. It may take several days of patient and gentle persistence, but eventually you will get responses from even the most reluctant students. During this breaking-in period, make sure that you make full use of wait time. Students will gradually come to believe that you actually will wait for them to think through their answer. This will increase their willingness to participate. Many English language learners and other students who need time to process have learned that their most reliable strategy is simply not to respond. Then teachers will not call on them. It may take time to change this perception. You need persistence and kindness, but eventually, they will get the message that you do want to hear from them. Some students will find this level of attention intimidating in itself. Some students have learned to ‘fly beneath the radar,’ in the words of one teacher. As you gradually introduce the discourse- intensive practices of academically productive talk, these students will find that they are expected, perhaps for the first time, to express their reasoning aloud. You may want to do this gradually, giving each student the experience of having their answers revoiced and discussed a number of times before you go into the full range of student talk moves. Academic issues: plan ahead to integrate talk with content One of the chief sources of uncertainty is that students will say things that you don’t expect or don’t understand. Whether you are teaching science, math, ELA or social studies, you can address this ahead of time by considering how the talk might interact with student knowledge and misconceptions. In your lesson planning, look at the materials you will use in the lesson. Review the texts or problems or hands-on materials you will be using. Identify the places where you will ask questions to create the opportunity for discussion or explanation or other reasoning. Write down the questions you plan to ask. You are making a lesson plan that explicitly incorporates academically productive talk. Consider where you might want to use Partner Talk. Plan where you will pose an open-ended question that may evolve into a discussion, an opportunity for students to externalize their reasoning. Now sit back and think about what students know about the material at each point. If you are experienced in this area you will have a good idea of common student misconceptions and likely misunderstandings. You will know which vocabulary items are likely to be difficult. If you are not experienced, ask other colleagues who have more experience, or make some working hypotheses. Until you feel comfortable with the process, make explicit notes for yourself about this. Next, try to imagine a scenario in which a student says something completely unexpected or

31 wrong or uninterpretable. What will you say? What are the range of options you have? Considering how you will deal with mistakes and unclear responses is tremendously important. Many students believe that mistakes are a reason for embarrassment and indicate only that the mistake-maker lacks understanding and awareness. Thus mistakes must be avoided at all costs, even at the cost of not participating. The way you handle mistakes and unclarity can make the difference between full and enthusiastic participation, and a quiet, reluctant class dominated by a few confident students. It’s important to know that there are several “no-fail strategies” you can always use when a student’s contribution is problematic. These strategies will socialize students into valuing mistakes as a resource for learning, rather than as evidence of a personal failure. •If the talk strikes you as productive but confusing, you can always use the talk move “Can anyone repeat?” Invariably, this will take the conversation in a productive direction. Usually someone can actually explicate and elaborate the idea, and if they can’t, the original student is motivated to try again and work hard to make the ideas clear to everyone. •If the talk is completely unclear (even after you have asked students to repeat) and it is unclear to you that it is worth a good deal of time to unpack the contribution, you can always simply move on by soliciting additional viewpoints. You can say, “Does anyone have another idea?” or “Does anyone else have something to say?” •If the talk is clear but likely to take the conversation in the wrong direction, you can acknowledge the contribution and say, “I’m not sure we can go there now. Remind me and we can talk about this later.” [It’s important to follow up with this student. You might write the comment on the board to remind you to return to it later.] •If the student’s answer is wrong but wrong in a way that might well be productive to unpack, you can always ask someone to repeat it. This may sound counterintuitive, but opening up the idea to the group has several advantages. First of all, it buys you a little bit of time to think. Second, it allows students time to fully process the idea. This makes it more likely that one of the students will see the problem and be able to comment productively on it. Finally, every time you allow students to think about an idea that turns out to be not completely correct, you are socializing them to truly think, rather than to just orient to your cues about what is correct. Students who have learned that their teacher will pursue ideas that need to be repaired tend to be far better at thinking critically and at persevering when presented with a complex idea. In any subject matter area, you can think ahead of time about various ways to move through the material. There are multiple ways to solve a problem, interpret a text, understand a science demonstration or model, or contextualize a historical document. These will help you understand the content from the point of view of various students, which is in itself a useful outcome. As you think ahead and imagine what students might say that would throw you off, you are actually improving your ability to respond to what they might say, and thus you are improving your ability to help them learn. Academic issues: reflect afterwards and adjust your course Particularly in the first few days

32 and weeks of incorporating academically productive talk strategies into your classroom, you will benefit greatly from finding time to reflect after a class. It is always difficult to squeeze one more thing into your day, but allocating even ten minutes for reviewing what worked and what didn’t will bring great rewards. During class, you can take notes about things you want to remember. At first this may feel strange, but jotting down notes about what students are saying actually gives everyone a bit more time to think. It also indicates to your students that you are taking their words seriously. You can even let them know why you are doing this: “I’m going to take a few notes about what you’re saying, because that will help me think about what is the best way for us all to learn this material.” It is difficult in the blur of classroom interaction to remember everything that was said. Jotting down student comments and questions that are unexpected or problematic or fruitful will help you greatly as you go back to planning. As you reflect on what worked and what didn’t, adjust your course. Incorporate into your plans what you have learned that day about the material and about your students. Your ability to predict what is going to happen and your confidence will rapidly improve. Social and behavioral issues: plan ahead for equity and respect Before you carry out your official presentation or negotiation of the talk norms in your classroom, you may want to consider the challenges your classroom will present. Do you have many students who are very quiet and unlikely to participate? Do you have a few students who are used to ridiculing and teasing other students? Do the boys dominate the girls, or vice versa? Is there a small group of very able students and a large group of students who have opted out, leaving active participation to that small group? Academically productive talk norms tend to look fairly similar across classrooms. Nevertheless, your unique circumstances will lead you to want to emphasize certain aspects of the norms. If your class is made up of many raucous students who compete to shout out their thoughts, you will not have to spend much time worrying about making sure that each speaker is audible. But you will perhaps have to spend extra time on making sure that turn-taking norms are followed. At this point it would also be wise to consider how you will deal with violations of the norms. Any experienced teacher (or parent!) knows that as soon as norms are agreed upon, the limits will be tested by some. This limits-testing will continue throughout the year, no matter how enthusiastically your students are participating in academically productive talk. Therefore, it is important to have established beforehand how you will enforce the talk norms. Check with colleagues and the administrators at your school at this point, just to make sure that your behavioral system will be backed up by appropriate parties in the school. Social and behavioral issues: reflect and assess your progress After a week or two of officially incorporating these talk strategies into your classroom, it is important to assess your progress. Are students largely adhering to the norms? Are their contributions to classroom talk becoming more interesting, more extensive, and more thoughtful? Are students spontaneously saying that they agree or disagree with their classmates’ ideas? If so, congratulations. You are seeing the benefits already. If not, don’t give up. Continue to work at it, and assess your progress at the end of each week. Again, we encourage you to take notes during class so that you can remember the high and the low points, and adjust your instruction accordingly.

33 Can my students really do this? The main worries that teachers have about their students’ capacity to participate in academically productive talk tend to be expressed in these questions: Will my students be willing to participate? Will my students have anything worthwhile to say? Will the talk be dominated by the one or two most able students? These concerns are reasonable. As you incorporate academically productive talk into your instruction, students will be called on to talk in ways that they may never have done, at least in class. They will have anxieties and fears about being embarrassed, about having their lack of knowledge revealed, perhaps even about being ridiculed. You want to allay their fears, but you know these fears are not unrealistic. How can you encourage them to participate and make sure that things stay under control so that no student has a truly negative experience? In addition to your anxieties about their willingness to participate, you may have concerns about their ability to participate. You probably have a few students whose abilities are obvious, and who you know will shine. But what about the others? Will this kind of talk turn out to be a showcase for these few most able students? If it does, then other students will eventually turn off, inferring that this talk is not really meant for them. For these reasons it’s important that you acknowledge these concerns at the very outset and plan to deal with them proactively. If you do, it is very likely that you—like virtually all of the teachers we have worked with—will be surprised by the degree to which your students like participating in academically productive talk. We wager that you will also be surprised by their ability to do so. Starting slow: give students a chance to get used to APT If you follow our advice above, you will be trying out some of the teacher talk moves before you officially introduce the APT norms. As you do this, notice that your students need time to get used to the pressures to talk just as you do. Many teachers have pointed out to us that it is important to build a positive talk environment. Specifically, they have told us that it is very important how one first begins to use the student repeating move. Early on, when you ask a student if they can repeat what another student said, very often they cannot. While eventually you will be able to expect that students should not violate this norm, at the beginning, it will take them some time to get used to. Teachers have told us that it is very important to use this move in a positive way: when a student makes a contribution, don’t look around for the one student who is not paying attention, and ask him or her to repeat. Instead, ask “Who understands what Rita said well enough to repeat it for us?” Make the ability to repeat another student’s utterance a positive accomplishment. Don’t hesitate to praise a student for their ability to repeat or rephrase a complex contribution. Later, when the

34 obligation to do this is clearly established as a positive norm, you can begin to sanction students for not being able to repeat due to attention problems. Make sure everyone has a chance to participate from the beginning. As you start to use academically productive talk, it will be awkward at times. You will inevitably want to rely on students who are competent and articulate. However, if you limit yourself to these students in order to avoid social awkwardness, you will be setting up tacit norms that will be very hard to dismantle later. Review the section on Turn-taking strategies above. Why isn’t this working? Trouble-shooting and FAQs Inevitably, problems will crop up. Here we have assembled a collection of some of the most common, along with suggestions about how to deal with them. The bottom line, however, is that you will need to diagnose and solve your problems in ways that fit the circumstances of your particular classroom. (a) My students won’t talk—they just sit there Particularly at the beginning, you may have trouble getting students to speak up. Here are the questions to ask yourself, and strategies to use. 1) Do they really understand the question? Sometimes teachers underestimate the amount of time it takes students to process a question, and then formulate an answer to it. The students may not yet have understood the question. As you wait for hands to go up, a ten second pause may feel like an eternity. First, use wait time skillfully. Ask the question, then silently count to ten. Then repeat the question using other words. Count to ten again. If there are still no hands up, ask someone if they can put the question—not the answer, but the question itself—in their own words. If no one can do it, rephrase the question again. Work to make your question understandable. Resist the urge to make the question into a simple, yes-no question. You can only get good discussion if you have a good question to begin with. 2) Are they having trouble putting together an answer? It may be that the question is understandable, but the answer is elusive. It’s time to build up to an answer by letting them talk about it. Try partner talk. Announce that they should turn and talk to the person next to them about their answer. They will likely burst into talk. Walk around the room with a clipboard and listen in, taking notes. When you hear students who seem to have the beginning of an answer, note that and then when you return to the front of the class, call on one of them. 3) Are they afraid of being ridiculed? Occasionally a teacher will tell us that his or her students will not talk. On investigation, it turns out that there are one or two

35 students in the class who have found clever ways to tease or make fun of other students but have not been disciplined for it. Students who have fear of ridicule will simply make up their minds never to talk. To figure out whether this is the case in your classroom, you may have to honestly examine your attitudes about fear, teasing, and creating an environment in which all students will feel safe to express their thinking. Students must be convinced that you care about their potential loss of face or they will not ever be willing to abandon their guarded stance. 4) Is talking in class culturally strange for them? Some students find it very difficult to talk in class because they have been taught that students should never talk: teachers do the talking. Often students who are recent immigrants from countries with very teacher-centered education practices find academically productive talk moves very strange and even objectionable. You will need to talk specifically about why these practices are helpful, and about how they are a special part of education in your classroom. Take care not to cast aspersions on cultural attitudes about silence or speaking. Eventually, students will be able to adopt new norms, even if these are only used in your classroom. (b) A few always want to talk but most won’t The same questions should be asked here as in example (a) above. However, in addition, you should ask another question. 1) Am I relying overly much on a few students? As we’ve said above, it is all too easy to continually call on students who want to talk. You feel that you are responding to their initiative, and that you are not putting pressure on students who don’t want to take. Moreover, it’s rewarding to listen to a few very able students move through the material. This approach, however, will not result in your students all receiving the benefits of using academically productive talk. Therefore, you need to consider another approach. You might want to address both problems in private conversations with the relevant students. You need to let the active students know that you appreciate their contributions, but you would like them to hold back a bit so that other students may contribute. There are many reasons that students jump in all the time. Some just want the attention. Others have a more altruistic motivation: they want to rescue you from long silences. You need to let them know that you are prepared to wait for students to talk, and that silence is not a bad thing: it allows for thinking. Then you need to approach the silent students individually and let them know that they must be willing to participate, just as everyone in the class does. Ask them if there are any reasons you should know about that they don’t want to speak up. Then negotiate a way that they are willing to start to participate. One teacher we know got students to agree that they would participate at least once each class period, even if that participation was only to raise their hand and ask someone to repeat. After these shy or hesitant students got a bit of practice at speaking in public, they became used to it and soon fit right in with the rest.

36 (c) My students are disrespectful and out of control This is a frequent complaint or concern, particularly among teachers with very large classes. In addressing this problem, it’s important to ask a number of questions: 1) Are the students just excited or testing limits of the new practices? It’s possible that the students are excited about a new approach and are overly exuberant. It’s possible that the students are testing you and are simply pushing the limits to see what will happen in this new practice. If this is the case, you will find that in the context of a rigorous and rich task with a good framing question, the students will quickly (after a session or two) settle down into a respectful discussion. You will find that being very clear about the rule “one person talks at a time” and trying a specific strategy such as the “talk ball” will have a noticeably positive effect. Then the experiences students begin to have motivate them to participate and work hard at explicating their ideas. Getting a chance to express complex ideas or the honor of having one’s ideas taken seriously by others is a heady and highly motivating experience. Students realize that the “one person talks at a time” rule – though restrictive — nonetheless protects their right to the floor and is fair to all. There is a kind of snowball effect. As students listen harder, they have more to say, and more motivation to be heard, speak clearly, and listen hard to one another. We have found that as students adjust to the norms of academically productive talk, they begin to take one another more seriously as academic colleagues, and behave accordingly. 2) Do I have a more serious classroom management problem? If, however, you have carefully introduced your students to the ground rules, established clear consequences for breaking the rules, and applied sanctions when violations occur, and your students are STILL not taking turns, speaking respectfully or listening to one another, you have a more serious classroom management problem. This may be a problem with the school culture at large or it may have to do with specific students in your class. It may also have to do with your management style and strategies overall. Whatever the cause, major classroom management problems are beyond the scope of this introductory text. It’s critical that you seek assistance from a colleague, coach, or administrator in your building. You must assess the nature of your management problems and seek specific interventions to address them. Academically productive talk will NOT solve major classroom management problems. Instead, stable management of the classroom is a pre- condition for effective academically productive talk and learning in any classroom. (d) My discussions quickly become slow and boring If you find that your discussions start on a good note, but quickly turn into something less rewarding, ask yourself the following questions.

37 1) Do your discussions turn into ‘quizzing’ sessions? Do you notice that discussion starts off well, but you quickly revert to a more familiar role of asking known-answer questions, with students providing short answers, followed by an evaluation from you? If so, you need to look closely at how you are actually posing your questions. Sometimes teachers start out posing a question and following up in ways that quickly close off the discussion, as in the example below. Teacher: So why do you think the volleyball with more air in will weigh Student: more? Teacher: I think because you added more to it, it will have more stuff in it Teacher: and it will weigh more. Good answer! Does everybody agree? [Silence. Several students nod.] Does anybody disagree? [Silence. No response.] By responding in a way that appears to favor the student’s answer, whether or not that was intended, the teacher may have set up a barrier to alternative answers. As the teacher tries to continue the discussion, she may find herself relying more and more on known-answer questions in order to get a response. The key to avoiding this problem is to spend time thinking about the questions you are going to use, and the follow-up questions you have ready to continue to support the discussion. 2) Do your discussions seem listless? If students seem uninterested, it may be that they do not understand. It may also be the opposite: that you are using the academically productive talk moves too mechanically, requiring students to answer questions that are too low-level. Occasionally, in his zeal to use revoicing, a teacher might overuse this move. Student: I found forty-seven seeds in my pumpkin. Teacher: So you’re saying you got forty-seven? Student: Uhh, yeah. Teacher: Can anyone repeat what he said? Students: [incredulous expressions] There is nothing wrong with using traditional methods of question and answer when the material warrants it. Academically productive talk moves are best suited to classroom activities in which you want to foster rigorous thinking, coordination of theory and evidence, and externalization of complex reasoning. (e) My students are at different levels Some teachers find that their classes are very heterogeneous, and that this seems to make it difficult to use academically productive talk effectively. Some students are thinking at a much higher level, or are able to articulate

38 more complex ideas. Other students in the same class may have severe learning problems or have educational histories which put them at a disadvantage. Academically productive talk cannot fix all of these problems; it is not a panacea. However, we have observed a number of teachers who are able to skillfully use the APT moves to provide most or all of the students with more enriching instruction than might otherwise be possible. For one thing, often times able students can understand an issue more quickly or at a more sophisticated level than another student, but this does not mean that they can express that idea fully. Their role in a discussion may be to put forth an idea and then struggle to make it understood by the other students. By using the device of having students repeat the idea, it will become clear to the originator that he or she has to continue to clarify until the idea becomes fully available. This benefits the able student, and also provides a tremendous amount of clarifying input for students who have more difficulty with the material. Less able students can contribute as well, even if their contributions are repetitions, or answers to queries about whether they agree or disagree, and why. There is a place for every student, and teachers who work at using the tools skillfully will find that they are able to orchestrate intellectually productive activity across the entire class. We end with a protocol for thinking through or planning a lesson to support academically productive talk.

39 Thinking Through a Lesson in Order to Promote Academically Productive Talk: Making Thinking Visible Through Talk, Argumentation, and Models The main purpose of the Thinking Through a Lesson Protocol is to prompt you to think deeply about a specific lesson that you will be teaching that is based on a cognitively challenging task, related to core concepts you are trying to teach. Part 1: Selecting and Setting up a Task or Theorizable Situation ! What are your goals for the lesson (i.e., what is it you want students know and understand about a core concept as a result of this lesson)? ! In what ways does the task build on students’ previous knowledge? What definitions, concepts, or ideas do students need to know in order to begin to work on the task? ! What are the variety of ways (some correct, some incorrect) that this task or phenomenon under investigation may be approached, or explained? o Which of these approaches (or methods, models, or explanatory tools) do you think your students will use? o What misconceptions might students have? o What errors might students make? ! What are your expectations for students as they work on and complete this task? o What resources or representational tools will students have to use in their work? o How will the students work -- independently, in small groups, or in pairs -- to explore this task? How long will they work individually or in small groups/pairs? Will students be partnered in a specific way? If so in what way? o How will students record and report their work? How will they make their thinking visible, to themselves and to their peers? ! How will you introduce students to the activity so as not to reduce the demands of the task? What will you hear that lets you know students understand the task? Part 2: Supporting Students’ Exploration of the Task ! As students are working independently, with partners, or in small groups: o What questions will you ask to focus their thinking? o What will you see or hear that lets you know how students are thinking about key ideas? o What questions will you ask to assess students’ understanding of key concepts or representations? o What questions will you ask to advance students’ understanding of the ideas? o What questions will you ask to encourage students to share their thinking with others or to assess their understanding of their peers’ ideas?

40 ! How will you ensure that students remain engaged in the task? o What will you do if a student does not know how to begin? o What will you do is a student finishes the task almost immediately and becomes bored or disruptive? o What will you do if students focus on ancillary or non-academic aspects of the activity (e.g., spend most of their time making a beautiful poster of their work)? Part 3: Sharing and Discussing the Task ! How will you orchestrate the class discussion so that you accomplish your academic goals? Specifically: o Which ideas, predictions, or explanations do you want to have shared during the class discussion? In what order will ideas, predictions, or explanations be presented? Why? o In what ways will the order in which ideas are presented help develop students’ understanding of the ideas that are the focus of your lesson? o What specific questions will you ask so that students will: \" make sense of the ideas that you want them to learn? \" expand on, debate, and question the ideas being shared? \" make connections between the different explanations or representations that are presented? \" look for patterns? \" reflect on the status of their own knowledge (the plausibility, usefulness of their ideas, their degree of certainty, etc.) \" begin to form generalizations? ! What will you see or hear that lets you know that students in the class understand the ideas or concepts that you intended for them to learn? ! What will you do tomorrow that will build on this lesson? The Thinking Through a Lesson Protocol is adapted from the collaborative efforts (lead by Margaret Smith, Victoria Bill and Elizabeth Hughes) of the mathematics team at the Institute for Learning and faculty and students in the School of Education at the University of Pittsburgh. Smith, M.S. & Bill, V. (2004, January). Thinking Through A Lesson: Collaborative Lesson Planning as a Means for Improving the Quality of Teaching. Presentation at the annual meeting of the Association of Mathematics Teacher Educators, San Diego, CA.

41 Appendix A: Nine Productive Talk Moves Encourage individual students to share, expand and clarify their own thinking: 1. Say More: “Can you say more about that?” “What do you mean by that?” “Can you give an example?” 2. Verifying and Clarifying by Revoicing: “So, let me see if I’ve got what you’re saying. Are you saying…?” (always leaving space for the original student to agree or disagree and say more) 3. Wait Time: “Take your time; we’ll wait.” Encourage students to listen carefully to one another: 4. Who Can Repeat? “Who can repeat what Javon just said or put it into their own words?” 5. Explaining what Someone Else Means: “Who can explain what Aisha means when she says that?” Press for deeper reasoning: 6. Asking for Evidence or Reasoning: “Why do you think that?” “What’s your evidence?” “How did you arrive at that conclusion?” “How does your evidence relate to your claim?” 7. Challenge or Counterexample: “Does it always work that way?” “How does that idea square with Sonia’s example?” “That’s a good question. What do you think?” Press students to apply their own reasoning to that of others: 8. Add On: “Who can add onto the idea that Jamal is building?” 9. Agree/Disagree and Why?: “Do you agree/disagree? (And why?)” “Are you saying the same thing as Jeyla or something different, and if different, how is it different?”

42 Notes:

  Chapter  1:  Moving  beyond  “knowing”  science  to  making  sense  of  the  world     Christina  Schwarz,  Cynthia  Passmore  &  Brian  Reiser     An  Introduction  to  Scientific  and  Engineering  Practices     Sarah  is  a  conscientious  teacher.  She  has  been  teaching  science  to  middle  school   students  for  8  years.  During  that  time  she’s  gone  from  a  tentative  novice  teacher  to  a   competent  veteran.  Throughout  her  career  she’s  been  alternatively  frustrated  and   pleased  with  her  teaching.  Sometimes  things  seem  to  be  going  well  and  other  times  she   feels  she  should  be  able  to  get  more  from  her  students;  she  wants  them  to  think  more   deeply  and  she  wishes  that  she  had  more  “aha”  moments  in  her  class.  Sarah  uses  the   district-­‐wide  pacing  guide  and  follows  the  textbook  that  was  adopted  by  her  school   several  years  ago.  Of  course,  she  supplements  the  district  materials  with  things  she’s   found  online,  gotten  from  teaching  colleagues  and  picked  up  at  conferences,  but  for  the   most  part,  she  does  many  of  the  same  activities  from  one  year  to  the  next.  Recently,  she   has  begun  to  hear  about  changes  that  may  be  coming  to  the  science  standards  in  her   state.  She  is  at  once  excited  and  intimidated  by  the  big  changes  in  science  education.   The  more  she  hears  about  these  reforms,  the  more  questions  she  has  about  what  this   means  for  her  as  a  teacher  of  science.       As  she  looks  over  the  Framework  for  K-­‐12  Science  and  the  Next  Generation  Science   Standards  (NGSS)  documents,  she  has  a  lot  of  specific  questions  about  what  she  is   reading.  She  wonders:   • What  is  this  focus  on  practices  all  about?     • Is  this  just  a  new  name  for  inquiry?   • How  should  my  class  look  different  if  I  am  “doing”  NGSS?     • If  I'm  pretty  happy  with  what  I’ve  been  doing,  why  would  I  want  to  take  this  on?       In  another  school  district,  Carlos  has  been  teaching  3rd  and  4th  grade  for  many  years   and  loves  working  with  his  students.  He  tells  great  stories  and  they  enjoy  doing  hands-­‐ on  activities  like  the  ones  they  sometimes  have  in  science  –  exploring  different  rocks   and  taking  the  field  trip  to  the  local  children’s  garden.  His  state  has  recently  adopted   NGSS,  and  he  is  wondering  what  these  new  standards  mean  for  him.     • What  will  I  have  to  do  differently?     • How  is  this  different  from  the  hands-­‐on  inquiry  I  do  now?   • How  will  I  have  time  for  NGSS?  There  is  already  so  much  to  cover  with  the   literacy  and  math  curricula.     Perhaps  you  can  see  yourself  in  Sarah’s  or  Carlos’  situation.  You  may  have  some  of   the  same  questions  about  the  goals  of  this  reform  and  what  it  means  for  you  as  a   teacher  of  science.  This  is  an  exciting  time  in  science  education.  We  have  many   opportunities  before  us  to  make  significant  and  lasting  change  in  the  ways  we  teach   science  at  the  K-­‐12  level.  But  with  major  change  comes  some  anxiety.  We  hope  this  

book  can  begin  to  answer  some  of  your  questions  about  the  reforms  found  in  The   Framework  for  K-­‐12  Science  Education  and  The  Next  Generation  Science  Standards.   Even  if  your  state  is  not  adopting  NGSS,  you  and  your  colleagues  can  take  advantage   of  the  research-­‐based  recommendations  in  the  Framework  for  making  science   learning  more  meaningful  and  effective  for  all  students.         The  title  of  this  book  expresses  a  major  goal  of  the  current  science  education  reform   effort  –  that  students  make  sense  of  the  natural  and  designed  world  by  engaging  in   science  and  engineering  practices.  To  some  educators,  this  may  seem  like  nothing   new.  For  many  years,  it  has  been  a  goal  of  science  reforms  to  move  from  students  as   passive  recipients  of  knowledge  to  classrooms  in  which  students  are  active   participants  in  generating  knowledge.  Attempts  since  the  1990s  to  incorporate   inquiry  in  science  classrooms  have  been  a  step  forward  in  efforts  to  accomplish  this.   Yet,  while  these  efforts  have  made  some  inroads,  studies  of  today’s  U.S.  classrooms   and  curriculum  materials  show  that  many  of  our  classrooms  do  not  involve  students   in  very  sophisticated  versions  of  scientific  practice  (Banilower  et  al.  2013).  Instead,   in  many  classrooms,  students  are  primarily  studying  and  recounting  factual   information  and  definitions  provided  by  textbooks  and  teachers  and  reinforced   through  hands-­‐on  activities  that  may  not  be  linked  to  advancing  students’   conceptual  ideas  and  practices.       As  a  science  education  community,  let’s  embrace  an  opportunity  to  do  more.  The   reform  agenda  articulated  in  the  Framework  and  NGSS  provides  a  vision  and  way   forward  toward  making  science  education  inspiring  and  meaningful.  While  the   Framework  and  standards  do  not  tell  us  exactly  how  we  should  teach,  they  do   provide  clear  direction  for  what  we  should  be  aiming  for  in  our  science  instruction.   They  help  us  see  that  there  are  productive  ways  to  integrate  the  processes  of   science  with  the  learning  of  science  and  help  clarify  that  we  should  be  pushing  for   outcomes  related  to  what  students  should  be  able  to  do  with  the  knowledge  they   have  developed  over  time.  The  Framework  states  that:     K-­‐12  science  and  engineering  education  should  focus  on  a  limited  number  of   disciplinary  core  ideas  and  crosscutting  concepts,  be  designed  so  that  students   continually  build  on  and  revise  their  knowledge  and  abilities  over  multiple   years,  and  support  the  integration  of  such  knowledge  and  abilities  with  the   practices  needed  to  engage  in  scientific  inquiry  and  engineering  design.  (p  2)     Even  though  some  of  the  themes  in  the  Framework  and  NGSS  around  engaging   students  in  science  and  engineering  may  sound  familiar,  the  documents  offer  new   ways  to  talk  about  and  organize  instruction  to  meet  these  goals.  In  this  book  we   focus  on  one  of  the  key  innovations  of  the  Framework:  a  focus  on  the  practices  of   science  and  engineering.  The  editors  and  contributors  to  this  book  have  a  great  deal   of  experience  in  working  on  how  to  focus  our  science  classrooms  to  be  about   making  sense  of  the  world  through  engaging  in  the  practices  of  science  and   engineering.  The  contributors  include  science  education  researchers  and  teachers   who  have  explored  these  ideas  in  their  own  classrooms.  We,  along  with  many  other  

science  educators  and  researchers,  have  been  collectively  working  on  these   problems  for  years  preceding  the  publication  of  the  Framework  and  NGSS.  The  work   of  many  of  the  authors  within  this  volume  contributed  to  the  vision  put  forth  in  the   Framework.  What  you  will  read  about  in  this  volume  are  not  ivory  tower,   “experimental”  ideas,  or  the  latest  fad  some  hope  will  make  all  the  difference.   Instead,  these  are  ideas  that  have  been  tested  and  refined  in  real  science  classrooms   over  many  years.  This  book  represents  a  true  collaboration  between  practicing   teachers,  those  in  science  education,  and  learning  sciences  researchers.       From  Scientific  Inquiry  to  Practices     The  emphasis  on  science  and  engineering  practices  attempts  to  build  on  prior   reforms,  and  take  advantage  of  what  research  has  revealed  about  the  successes  and   limitations  of  inquiry  classrooms.  We  like  to  think  of  the  focus  on  practices  as  a  kind   of  Inquiry  2.0  –  not  a  replacement  for  inquiry,  but  rather  a  second  wave  that   articulates  more  clearly  what  successful  inquiry  looks  like  when  it  results  in   building  scientific  knowledge.  Inquiry  classrooms,  as  they  are  often  configured,   typically  let  students  explore  the  relationship  between  two  variables  (e.g.  how  does   the  mass  of  a  toy  car  affect  its  velocity  going  down  a  ramp),  but  often  this  empirical   exploration  is  not  taking  place  in  an  ongoing  process  of  questioning,  developing,  and   refining  explanatory  knowledge  about  the  world.  Testing  and  confirming  or   disconfirming  hypotheses  is  part  of  science,  but  these  actions  become  meaningful  by   being  a  part  of  the  broader  work  of  building  explanatory  models  and  theories.     This  attempt  to  take  our  ideas  of  inquiry  in  science  beyond  designing  investigations   and  testing  hypotheses  has  led  to  the  fuller  articulation  of  inquiry  as  the  scientific   and  engineering  practices  that  enable  us  to  investigate  and  make  sense  of   phenomena  in  the  world  by  building  and  applying  explanatory  models,  and  by   designing  solutions  for  problems.  Making  sense  of  the  world  or  sense-­‐making  for   short  is  the  fundamental  goal  of  science  and  should  be  at  the  core  of  what  happens   in  science  classrooms.       What  is  involved  in  this  sense-­‐making?  Sense-­‐making,  as  we  are  using  it  here,  is  the   conceptual  process  in  which  a  learner  actively  engages  with  the  natural  or  designed   world,  wonders  about  it,  develops,  tests,  and  refines  ideas  with  peers  and  the   teacher.  Sense-­‐making  is  the  proactive  engagement  in  understanding  the  world  by   generating,  using,  and  extending  scientific  knowledge  within  communities.  In  other   words,  sense-­‐making  is  about  actively  trying  to  figure  out  the  way  the  world  works   (for  scientific  questions),  and  exploring  how  to  create  or  alter  things  to  achieve   design  goals  (for  engineering  questions).       When  student  sense-­‐making  is  the  focus  of  the  classroom  goals  and  purposes  –  then   science  and  engineering  practices  used  to  make  sense  of  the  world  become  critical.   Science  and  engineering  practices  are  the  way  we  build,  test,  refine,  and  use   knowledge,  either  to  investigate  questions  or  to  solve  problems.  As  defined  in  the   Framework,  the  science  and  engineering  practices  are  the  different  parts  of  the  

sense-­‐making  process.  Here  are  the  eight  practices  identified  in  the  Framework  and   used  in  NGSS:     1. Asking  questions  (for  science)  and  defining  problems  (for  engineering)   2. Developing  and  using  models   3. Planning  and  carrying  out  investigations   4. Analyzing  and  interpreting  data   5. Using  mathematics  and  computational  thinking   6. Constructing  explanations  (for  science)  and  designing  solutions  (for   engineering)   7. Engaging  in  argument  from  evidence   8. Obtaining,  evaluating,  and  communicating  information.       Now  that  we  have  listed  these  practices,  we  can  be  a  little  more  specific  about  how   the  practices  are  designed  to  flesh  out  scientific  inquiry.  There  are  some  familiar   ideas  in  the  list  that  are  often  present  in  classrooms  attempting  inquiry.  Planning   and  carrying  out  investigations  and  analyzing  and  interpreting  data  are  typically   what  we  think  of  as  involving  students  in  an  inquiry  investigation.  But  the  list  of   practices  moves  beyond  these  two  practices  and  includes  others  such  as   constructing  explanations.  As  we  will  see  later  (in  Chapter  10),  using  the   explanation  practice  involves  more  than  what  sometimes  happens  with  hypothesis   testing,  where  students  figure  out  how  two  variables  are  related.  The  goal  of   constructing  explanations  (practice  6)  is  to  be  able  to  say  why  something  happens.   In  the  process  of  figuring  out  why  something  in  the  world  happens  the  way  it  does,   students  will  often  have  different  ideas,  and  will  need  to  evaluate  one  another’s   ideas  against  evidence  (practice  7).  As  students’  reach  consensus  through  this   argumentation,  they  represent  their  general  account  for  why  something  happens  as   a  general  model  (practice  2).  And  of  course  this  all  should  be  sparked  in  the   classroom  from  explanatory  questions  (Practice  1)  that  arose  from  an  attempt  to   make  sense  of  some  data  or  patterns  in  the  world.  We  will  get  to  specifics  about   these  practices  and  consider  the  engineering  thread  through  the  practices,  in   Section  II  of  this  book.  For  now,  notice  that  the  mix  of  these  practices  is  to  develop   explanations  and  models  and  involve  interacting  with  one  another  to  compare  ideas   and  reach  consensus.     One  critical  feature  of  a  sense-­‐making  classroom  is  that  if  students  are  genuinely   engaged  in  science  and  engineering  practices,  an  observer  should  be  able  to  walk  in   to  a  science  class  on  any  day  and  tap  a  student  on  the  shoulder  and  ask:  “what  are   you  trying  to  figure  out  right  now?”  The  intellectual  aim  of  any  work  in  the  science   class  should  be  clear  to  everyone.  Rather  than  stating,  “We  are  learning  about   photosynthesis  or  plate  tectonics,”  students  should  be  able  to  say  (and  believe!)   “We’re  trying  to  figure  out  how  the  tiny  seed  becomes  this  huge  oak  tree”  or  “we’re   trying  to  better  understand  why  volcanoes  and  earthquakes  happen  more  often  in   some  parts  of  the  world.”  These  examples  illustrate  how  the  students  are  figuring   out  the  world  and  illustrate  a  sense-­‐making  goal  in  the  classroom.      

In  addition  to  these  sense-­‐making  goals,  another  critical  aspect  of  engaging   successfully  in  science  and  engineering  practices  is  related  to  classroom  community.   These  practices  create  a  need  for  designing  our  classrooms  as  places  in  which   students  are  working  together  to  share  ideas,  evaluate  competing  ideas,  critique  one   another’s  ideas,  and  reach  consensus  as  a  classroom  community.  This  shift  to   practices  highlights  the  importance  of  working  with  one  another  to  build  and  debate   knowledge,  adding  social  interaction  and  classroom  discourse  to  what  students   need  to  learn  as  they  participate  in  scientific  sense-­‐making.  In  this  way,  the  practices   extend  prior  visions  of  inquiry  to  define  processes  for  building  and  refining   scientific  knowledge  as  a  community.     Exploring  the  Difference  in  Vision  in  Two  Classroom  Cases       One  way  to  begin  to  explore  the  implications  of  a  focus  on  student  sense-­‐making   with  practices  is  to  examine  two  contrasting  learning  environments.  Below  we  will   explore  vignettes  of  two  science  classrooms.  Both  vignettes  involve  middle  school   students  learning  about  the  phases  of  the  Moon,  but  they  differ  in  some  important   respects.    As  you  read  these  vignettes,  look  for  several  critical  parts  of  the  science   learning  environment.  Ask  yourself     • Where  do  the  questions  come  from?     • Who  is  involved  in  figuring  out  how  to  investigate  the  question?   • How  do  students  get  to  an  explanation?     • What  is  the  role  of  agreement,  disagreement,  and  consensus?       Case  1  Moon  phases  in  Ms.  Sheridan’s  class     The  students  come  into  Ms.  Sheridan’s  class  and  find  that  the  topic  for  the  day  is  Moon   phases.  The  day  before  this  class,  students  had  reviewed  the  order  of  the  planets  from   the  Sun.  They  had  also  made  a  chart  about  some  of  the  key  characteristics  of  each   planet.       After  she  introduces  the  topic  of  the  day,  Ms.  Sheridan  asks  the  students  to  raise  their   hands  and  tell  the  class  one  thing  they  know  about  the  Moon.  Students  offer  ideas  such   as,  “I  know  that  we’ve  sent  rockets  to  the  Moon”  and  “Isn’t  the  Moon  involved  in  tides?”       After  three  or  four  students  have  shared,  the  teacher  asks  them  if  they  have  ever   noticed  that  the  Moon  has  different  shapes  at  different  times.  She  explains  that  the   different  shapes  are  called  the  “phases  of  the  Moon”  and  puts  up  a  list  of  eight  Moon   phase  names.    Next  she  explains  that  today  they  are  going  to  learn  why  the  Moon’s   shape  appears  to  change.  She  starts  with  the  main  facts  about  Moon  phases.  The   phases  occur  in  a  cycle.  The  cycle  is  one  revolution  of  the  Moon  around  the  Earth,   about  28  days.  She  explains  that  the  Sun  is  relatively  far  away  from  the  Earth  and  the   Moon.  She  shows  the  class  how  light  from  the  Sun  falls  on  the  Moon,  always  lighting  up   exactly  half  of  the  Moon.  Then  she  explains  that  what  part  of  the  lit  Moon  you  can  see   varies  depending  on  where  the  Moon  is  in  its  orbit  around  the  Earth.  She  shows  the  

class  a  diagram  on  the  smart  board,  and  walks  them  through  the  different  steps  in  the   Moon’s  orbit,  and  describes  the  phase  that  can  be  seen  at  that  point  in  the  orbit,  along   with  telling  students  the  names  of  each  Moon  phase  that  she  expects  them  to  learn.     Ms.  Sheridan  then  tells  the  class  that  now  they  can  try  it  out  for  themselves  to  see  each   of  the  Moon  phases.  She  divides  the  class  into  8  groups  and  gives  each  group  a  small   Styrofoam  ball  to  represent  the  Moon  and  a  larger  blue  one  to  represent  the  Earth.   Each  group  also  gets  a  flashlight  to  represent  the  shining  sun.  She  gives  each  group   one  of  the  8  phases  to  prepare  to  demonstrate.  Each  group  gets  the  name  of  their   phase  and  a  diagram  showing  the  position  of  the  Moon,  Earth,  and  Sun  for  that  phase.   She  gives  each  group  5  minutes  to  match  the  position  of  the  Moon  (the  small   Styrofoam  ball),  the  Sun  (flashlight)  and  the  Earth  (larger  blue  ball)  to  the  diagram   for  their  phase.  She  turns  out  the  classroom  lights  and  students  excitedly  position  the   Moon  and  Sun  to  match  their  diagrams.       Then,  each  group  shows  the  rest  of  the  class  their  Moon  phase  and  the  position  of  the   Sun,  Earth  and  Moon  for  their  phase.  At  the  end  of  the  activity,  she  assigns  students  to   make  8  flashcards  that  evening  with  a  picture  of  the  phase  on  one  side  of  the  card  and   the  name  of  that  phase  on  the  other.  She  lets  them  know  that  they  will  have  a  quiz  the   following  day  on  this  material  and  on  the  planets  from  the  previous  day.       Ms.  Sheridan  has  shared  this  lesson  with  some  of  her  colleagues.  They  all  like  how   hands-­‐on  the  lesson  is.  They  really  like  having  students  demonstrate  the  phases   with  a  flashlight  and  Styrofoam  balls,  and  feel  that  this  activity  helps  make  the  ideas   more  concrete  and  understandable  to  students.  One  of  her  colleagues  wonders   whether  using  a  flashlight  and  Styrofoam  ball  to  represent  the  Sun  and  Moon  is   what  NGSS  means  by  “developing  and  using  models.”     There  is  much  to  like  in  Ms.  Sheridan’s  lesson.  Let’s  come  back  to  it  after  we  look  at   how  Ms.  Lee’s  classroom  works  on  similar  ideas.       Case  2.  Moon  phases  in  Ms.  Lee’s  classroom     The  students  in  Ms.  Lee’s  class  have  been  working  on  near-­‐Earth  astronomy  for  a  few   weeks.  They  have  been  pursuing  an  overarching  question  of  “Why  do  the  Sun,  Moon   and  stars  move  in  our  sky  and  change  in  appearance  over  time?”  Most  recently  they   have  been  investigating  the  appearance  of  the  Moon.  They  wonder  why  it  is  visible  in   the  sky  at  different  times  of  day  and  appears  some  nights  and  not  others.  For  over  a   month  they  have  been  spending  a  few  minutes  each  day  recording  the  appearance  of   the  Moon  on  that  day  in  a  data  table  in  their  notebooks  and  as  it  goes  through  the   cycle  of  the  phases  they  learn  the  technical  names  of  each.    Prior  to  this  lesson,  they   used  Moonrise  time  data  to  figure  out  that  the  Moon  orbits  the  Earth  in  the  same   direction  as  the  Earth  spins  and  it  takes  about  a  month  to  complete  one  orbit.       Ms.  Lee  begins  class  on  this  day  with  a  discussion  to  help  the  class  summarize  what   they  have  figured  out  so  far,  and  what  questions  remain  about  their  observations.  Ms.  

Lee  draws  their  attention  to  a  major  question  about  the  Moon  that  started  them  off  on   their  investigation  –  why  does  the  Moon  change  its  shape  during  the  month?  They  have   collected  data  about  the  Moon’s  appearance  with  the  observations  made  throughout   the  month.  They  know  that  it  takes  the  Moon  28  days  for  a  full  cycle,  as  the  Moon   orbits  the  Earth.  But  they  still  haven’t  figured  out  why  the  shape  changes  during  that   time.       Based  on  what  they  have  figured  out  so  far,  the  class  refines  their  original  question  to   be:  “Why  does  the  appearance  of  the  Moon  change  as  it  orbits  the  Earth?”  The  students   brainstorm  their  initial  ideas  about  why  the  apparent  shape  of  the  Moon  might  change,   using  what  they  have  figured  out  about  the  orbit  of  the  Moon  around  the  Earth  as  a   starting  point.  In  the  discussion,  Ms.  Lee  raises  the  question  of  how  it  is  even  possible  to   see  the  Moon  from  Earth.  Students  draw  on  what  they  know  about  light  sources  and   how  light  allows  us  to  see,  and  generally  agree  that  it  must  be  the  light  from  the  Sun   that  reflects  off  the  Moon  that  is  the  part  of  the  Moon  visible  from  the  Earth  (since  the   Moon  is  not  a  light  source).  But  students  are  not  in  agreement  about  why  this  would   change  as  the  Moon  revolves  around  the  Earth.       Ms.  Lee  suggests  they  try  to  picture  what  is  happening  as  the  Moon  goes  around  the   Earth,  and  suggests  they  use  physical  props  to  see  for  themselves  why  the  shape  might   appear  to  change.  Students  like  the  idea  and  are  eager  to  see  what  would  happen  to   light  from  the  Sun  as  the  Moon  orbits  the  Earth.  As  in  earlier  modeling  activities  in   their  classroom,  Ms.  Lee  has  the  class  agree  on  the  question  the  model  needs  to  explain,   and  then  brainstorm  what  needs  to  be  represented  in  the  model.  In  discussion  students   decide  they  need  to  represent  the  Earth,  the  Moon,  and  the  Sun.  Ms.  Lee  gives  each   group  of  students  a  Styrofoam  ball  and  suggests  that  they  can  use  the  ball  to  represent   the  Moon.  She  suggests  using  a  lamp  she  has  without  the  shade  to  represent  the  Sun   and  places  it  in  the  center  of  the  room  so  all  the  kids  can  use  its  light  in  their   investigation  (she  also  covers  the  windows  so  that  the  “Sun”  is  the  only  light  in  the   room).  Since  the  goal  of  the  activity  is  to  see  what  the  Moon  looks  like  from  the  Earth,   Ms.  Lee  helps  the  students  come  up  with  the  idea  of  using  the  ball  and  their  own  bodies   to  simulate  the  Moon’s  orbit  around  Earth  (recalling  what  they  had  already  figured   out  about  that  from  the  Moonrise  times).  Ms.  Lee  asks  students  to  state  what  they  are   trying  to  figure  out,  and  how  they  will  use  the  props  to  test  their  ideas  before  they   begin.  Students  agree  that  they  need  to  figure  out  what  parts  of  the  Moon  they  can  see   in  each  part  of  the  orbit.         Students  actively  talk  as  they   engage  and  make  notes  about   what  they  can  see  from  each   position.  Once  students  have  been   able  to  collect  all  their  evidence   and  report  they  are  ready  to  try  to   explain  the  phenomenon,  Ms.  Lee  

asks  them  to  discuss  in  their  groups  and  draw  a  representation  on  their  poster  paper   that  shows  why  the  Moon’s  appearance  changes  over  the  course  of  the  month.  Once   each  group  has  finished  she  has  the  class  put  up  their  diagrams  around  the  room.  They   do  a  gallery  walk  so  they  can  all  see  what  the  other  groups  have  created.  Then  they   spend  time  in  their  groups  talking  about  what  they  have  seen,  trying  to  identify  where   they  have  agreed  or  disagreed  with  other  groups  and  what  makes  for  a  good   representation.  As  a  whole  class  they  then  discuss  the  differences  between  the  various   explanations  and  how  they  have  represented  them.  The  teacher  guides  a  discussion  to   help  them  decide  on  a  consensus  explanation  and  a  way  to  represent  it  in  a  diagram.   Ms.  Lee  tells  students  that  their  homework  for  the  day  is  to  write  a  short  paragraph   that  they  could  use  to  explain  to  a  friend  from  a  different  class  why  we  see  phases  of   the  Moon  from  Earth.  The  next  day  in  class  they  apply  their  ideas  by  finding  pictures  in   children’s  books  that  should  be  drawn  differently  based  on  their  knowledge  of  the   Moon  and  it’s  phases.     There  are  some  similarities  between  these  classrooms,  as  well  as  a  number  of   important  differences.  In  both  classrooms,  students  are  engaged  in  hands-­‐on  science   using  physical  materials  and  active  learning  strategies.  In  both  classrooms,  they  are   trying  to  generate  an  explanation  about  why  the  shape  of  the  Moon  appears  to   change.  In  both  classrooms,  students  are  being  challenged  to  think  through  the  ideas.   But  there  is  an  important  difference  in  the  work  the  teachers  and  students  are  doing   in  these  two  classrooms,  and  in  how  that  work  is  divided.       We  could  summarize  Ms.  Sheridan’s  classroom  as  one  in  which  students  are  learning   about  lunar  phases,  while  Ms.  Lees’  classroom  is  one  in  which  students  are  figuring   out  lunar  phases.       In  Ms.  Sheridan’s  class,  her  job  is  to  provide  the  explanation,  and  students’  job  is  to   work  with  the  explanation  and  try  to  learn  it.  The  purpose  of  doing  the  hands-­‐on   activities  was  so  that  students  could  see  and  make  sense  of  the  teacher’s  explanation.   The  students’  role  is  to  follow  the  directions,  do  the  activity,  try  to  apply  the  ideas   provided  by  the  teacher,  and  to  work  on  learning  them  for  assessments.  Although   students  are  working  hard  to  understand  the  explanation  provided  by  Ms.  Sheridan,   it  is  Ms.  Sheridan  who  has  the  authority  and  has  done  the  “heavy  lifting”  of  building   the  knowledge,  not  the  students.  And  while  the  explanation  was  stated  as  the  goal,   the  teachers’  assessment  in  fact  ended  up  focusing  mostly  on  the  names  of  the   phases  and  their  order,  rather  than  assessing  whether  students  could  use  that   explanation  and  reason  with  it.       In  contrast,  Ms.  Lee’s  class  engaged  students  in  the  science  and  engineering   practices  that  are  targeted  in  NGSS.  Students  are  positioned  in  the  class  as  the  ones   who  will  be  doing  the  figuring  out.  They  are  guided  by  the  teacher  and  given   appropriate  props.  If  you  were  to  walk  over  to  a  group  of  students  during  the   activity,  you  could  ask  them  “what  are  you  trying  to  figure  out?”  and  they  would  say   something  like  “we  know  that  the  Moon  is  lit  up  by  the  sun,  but  we  are  trying  to   figure  out  why  the  sunlit  part  of  the  Moon  changes  in  shape  as  it  goes  around  the  

Earth.”  They  are  manipulating  the  objects  or  analyzing  data  not  because  they  are   simply  following  directions,  but  because  they  are  trying  to  figure  out  and  construct   an  explanation  in  response  to  a  question.  It’s  not  that  Ms.  Lee  gave  them  the   challenge  and  let  them  just  “discover,”  “explore,”  or  “do  inquiry.”  She  gave  them  a  lot   of  probing  questions  and  guidance  to  get  them  to  realize  what  needed  to  be   investigated,  and  helped  them  converge  on  a  good  starting  point.  But  throughout   students  were  taking  on  the  responsibility  of  making  sense  of  phenomena,   developing  explanations,  comparing  them  as  a  class,  and  reaching  consensus.  Ms.   Lee’s  students  are  investigating  why  the  Moon  appears  to  change  shape.  They  are   engaged  in  scientific  practices  like  questioning,  analyzing  data,  modeling  to  make   sense  of  the  phenomenon.  The  students  and  their  ideas  are  at  the  center  in  this   classroom.1     To  go  a  bit  deeper  on  the  differences  between  these  two  classrooms,  we  will  ask  a   series  of  questions  about  the  sense-­‐making  process  that  help  us  see  how  the   practices  played  out  in  in  these  classrooms.     Where  do  the  questions  come  from?  In  both  cases,  students  are  working  on  a  similar   idea  -­‐-­‐  the  apparent  shape  of  the  Moon  changes  over  time.  But  in  Ms.  Sheridan’s   class,  the  question  originally  came  from  the  teacher.  The  teacher  told  the  class  they   were  going  to  learn  about  the  phases  of  the  Moon,  introducing  it  as  the  next  topic  in   their  unit.  In  contrast,  in  Ms.  Lee’s  classroom,  the  question  emerged  from  the   students’  own  observations.  As  part  of  a  larger  investigation  of  why  things  change  in   the  sky,  they  noticed  patterns  in  the  shapes  of  the  Moon,  and  uncovered  the  fact  that   these  occur  in  cycles.  In  both  cases,  “lunar  phases”  is  the  name  of  the  scientific  idea   they  are  investigating,  but  in  Ms.  Lee’s  classroom,  the  question  was  identified  and   phrased  in  terms  of  the  phenomena  they  experienced  –  why  does  the  shape  of  the   Moon  change  over  time?  Students  were  involved  in  co-­‐constructing  this  question   with  the  teacher.  In  Ms.  Sheridan’s  class,  a  question  was  not  at  the  center  of  their   work  at  all.  Although  she  made  an  attempt  to  motivate  the  work  on  moon  phases  by   pointing  out  that  the  shape  of  the  Moon  changes  during  the  month,  they  did  not   frame  their  exploration  of  moon  phases  in  terms  of  a  question  at  all.  (You  will  read   more  about  the  importance  of  the  posing  questions  practice  in  Chapter  5.)     Who  is  involved  in  figuring  out  how  to  investigate  the  question?  In  both  cases,   students  were  exploring  how  sunlight  falls  on  the  Moon  and  how  that  influenced   what  could  be  seen  from  Earth.  Notice  however  that  in  Ms.  Sheridan’s  class,  the   design  of  the  investigation  came  directly  from  the  teacher,  and  it  wasn’t  really  an   investigation  at  all.  The  students  were  given  direction  about  how  to  demonstrate   something  they  had  already  been  shown.  In  contrast,  in  Ms.  Lee’s  classroom,  this   was  a  co-­‐construction.  Certainly  the  teacher  played  a  critical  role,  but  the  students   were  also  involved  in  thinking  through  how  they  would  investigate  their  question   and  why  particular  aspects  of  the  set-­‐up  might  relate  to  their  ongoing  work.  The                                                                                                                   1  See  Barton,  2001  and  http://ncisla.wceruw.org/muse/earth-­‐moon-­‐sun/index.html   for  more  information  on  the  instructional  sequence  described  here.  

teacher  proposed  using  physical  props  to  explore  light  falling  on  the  Moon,  but   asked  the  students  what  objects  they  would  need  to  represent.  After  agreeing  on   what  the  model  needed  to  contain,  and  providing  the  objects  they  needed,  she  asked   the  class  how  they  would  manipulate  the  objects  to  explore  the  question.  In  both   cases  the  students  ended  up  doing  something  similar,  interacting  with  balls  and   light  sources.  But  in  Ms.  Lee’s  classroom,  the  students  were  part  of  thinking  through   the  logic  of  what  they  were  doing.    (You  will  read  more  about  the  “planning  and   conducting  investigations”  practice  in  Chapter  7.)     How  do  students  get  to  an  explanation?  This  is  perhaps  the  biggest  contrast  between   our  two  scenarios.  In  Ms.  Sheridan’s  classroom  the  teacher  walked  the  students   through  the  explanation  up  front.  She  then  gave  them  the  opportunity  to  see  the   ideas  in  action,  by  exploring  the  different  lunar  phases.  As  they  were  working  with   the  props,  they  were  attempting  to  match  the  diagram  they  had  been  given.  They   already  knew  that  they  should  be  seeing  different  amounts  of  the  half  of  the   illuminated  Moon’s  surface.  Doing  the  activity  helped  them  visualize  that  idea,  and   see  that  it  worked.  But,  they  had  already  been  given  this  explanation  as  the  correct   answer  before  the  activity.  In  contrast,  in  Ms.  Lee’s  classroom,  the  students  were   more  involved  in  building  and  using  this  explanation.  They  didn’t  start  from  scratch,   of  course.  Ms.  Lee  helped  the  class  converge  on  a  good  starting  point,  so  that  all   students  were  starting  their  investigation  constrained  by  the  ideas  that  what  we  see   is  Sunlight  reflecting  off  the  Moon,  reaching  the  Earth.  But  students  had  to  figure  out   why  different  positions  in  the  orbit  of  the  Moon  around  the  Earth  led  to  the  different   apparent  shapes  and  why  a  host  of  other  possible  explanations  (namely  the  Moon   phases  are  caused  by  a  shadow  from  the  Earth)  would  be  inaccurate.  In  Ms.   Sheridan’s  classroom,  students  had  to  understand  and  replicate  the  teacher’s   explanation;  in  Ms.  Lee’s  classroom,  students  were  involved  in  constructing  that   explanation,  guided  by  the  teacher.  (You  will  read  more  about  the  “developing  and   using  models”  and  “constructing  explanations”  practices  in  Chapters  6  and  10.)     What  is  the  role  of  agreement,  disagreement,  and  consensus?  In  Ms.  Sheridan’s   classroom,  the  teacher  was  the  authority,  providing  the  explanation.  The  students’   job  was  to  try  to  understand  and  use  it.  In  Ms.  Lee’s  classroom,  the  authority  was   more  distributed.  In  their  groups,  students  grappled  with  trying  to  explain  why  the   shape  of  the  Moon  changed.    In  the  gallery  walk,  they  shared  their  group’s  ideas,  and   compared  them  to  other  groups’  ideas.  In  the  discussion,  they  figured  out  where  all   the  groups  agreed,  identified  disagreements,  and  talked  through  the  disagreements   to  figure  out  a  class  consensus  that  everybody  was  convinced  of.    The  decision  for   the  final  model  involved  all  the  students,  guided  by  the  teacher.  (You  will  read  more   about  argumentation  in  Chapter  11.)     Coordinating  Practices  to  Achieve  Sense-­‐making     Now  that  we  have  seen  an  example  of  a  class  engaged  in  using  scientific  practices  to   make  sense  of  scientific  phenomena,  we  turn  towards  thinking  about  how  this  

happens  in  general,  and  how  you  can  think  about  using  practices  to  help  your   students  make  sense  of  the  world.       How  do  we  use  the  science  and  engineering  practices  to  work  toward  sense-­‐ making?  It  may  be  tempting  to  think  of  the  practices  as  a  sequence,  perhaps  like  “the   scientific  method,”  or  as  an  instructional  sequence  like  the  5  E’s.  As  we  will  see  in   Section  II  of  this  volume,  there  are  many  different  paths  we  may  end  up  taking   through  the  practices,  depending  on  the  specific  investigation  or  design  problem.   And  practices  may  need  to  be  brought  in  at  multiple  points  as  we  build  and  refine  an   explanation,  model,  or  design.  But  regardless  of  the  path,  here  are  four  guiding   questions  that  can  help  organize  the  work  of  the  practices  as  part  of  sense-­‐making:       (1)  What  are  we  trying  to  figure  out?       What  is  the  observable  phenomenon,  object,  or  system  we  are  trying  to  figure  out  or   problem  we  are  trying  to  define?  Investigations  and  engineering  problems  are  built   around  phenomena  and  the  questions  connected  with  them.  Engaging  in  practices   means  that  we  are  always  trying  to  figure  something  out  or  solve  some  problem   connected  with  some  phenomenon  in  the  world  –  rather  than  defining  terminology   (such  as  ‘what  is  gravity/energy/an  ecosystem?’).  While  the  word  “phenomenon”  is   not  explicitly  part  of  the  names  of  any  of  the  eight  practices,  phenomena  are  central   to  what  the  practices  are  all  about.  A  goal  of  NGSS  is  connecting  the  science  that   students  are  learning  with  the  application  of  these  ideas  in  the  world.  What  can  we   explain  with  this  idea?  What  problems  do  these  ideas  help  us  understand  and  solve?   When  engaging  in  practices,  there  must  be  something  about  the  world  we  are  trying   to  figure  out.  In  other  words,  there  is  some  event  (like  an  earthquake,  a  storm,   movement  of  objects  in  a  collision,  change  of  materials  in  a  chemical  reaction)  or  a   pattern  (resemblances  of  offspring  to  prior  generations,  changes  in  atmospheric   conditions  before  a  storm,  changing  shapes  of  the  Moon)  that  we  are  trying  to  figure   out,  or  problems  connected  with  events  or  patterns  that  we  are  trying  to  understand   to  design  a  solution  (e.g.,  early  warning  system  for  tsunamis,  less  polluting  sources   of  energy).  In  science,  once  we  recognize  the  phenomenon,  we  need  to  ask  what,   how,  and  why  is  this  phenomenon  happening?  For  engineering,  once  we  have  a   problem,  we  ask  what  are  the  factors  influencing  this  problem  and  how  can  we   intervene  to  alter  these  factors?     The  most  obvious  practice  involved  in  making  sense  of  the  world  around  deciding   what  we  are  trying  to  figure  out  is  asking  questions  and  identifying  problems.  But   it’s  important  to  stress  that  sense-­‐making  is  an  incremental  process.  Questions  will   arise  not  only  in  the  beginning  of  an  investigation  or  design,  but  also  throughout  the   ongoing  process  of  sense-­‐making.  In  our  attempts  to  explain  phenomena,  we  may   uncover  new  questions,  or  realize  that  our  models  work  for  some  parts  of  the   phenomenon  but  not  others.  So  we  may  end  up  in  the  asking  questions  and   identifying  problems  practice  as  we  are  in  the  midst  of  trying  to  design  solutions,   construct  explanations,  or  develop  models,  as  well  as  when  we  are  starting  an   investigation  or  design.  Furthermore,  we  may  need  to  compare  alternative  

questions  or  framing  of  the  design  problem  in  principled  ways,  drawing  on  the   practice  of  argumentation.  So  part  of  refining  what  we  are  trying  to  figure  out  needs   to  build  on  the  next  three  questions.     (2)  How  will  we  figure  it  out?     How  will  we  develop,  explore  and  test  the  model  and  associated  explanation  or   solution?  When  we  have  phenomena  that  have  motivated  questions  to  investigate   and  problems  to  address,  another  collection  of  practices  come  into  play  to  make   progress  on  the  work.  The  most  obvious  practice  relevant  here  is  planning  and   carrying  out  investigations.  Typically  this  may  also  involve  clarifying  what  is  known   to  inform  the  investigation  or  design,  drawing  on  the  practice  of  obtaining  and   evaluating  information.  Again,  because  of  the  incremental  nature  of  sense-­‐making,   constructing  explanations  and  developing  solutions  will  be  ongoing,  perhaps  with   initial  ideas  informing  the  planning  of  subsequent  investigations  or  design   explorations.  Argumentation  may  be  needed  to  make  principled  decisions  between   competing  investigation  plans  or  design  ideas.       (3)  How  do  we  keep  track  of  what  we  are  figuring  out?       Making  sense  of  what  we  are  seeing  goes  hand  in  hand  with  planning  and   conducting  the  investigation  or  design  work.  Key  questions  related  to  this  aspect  of   sense-­‐making  include  –  what  happened?  Is  this  what  we  expected?  What  worked,   what  didn’t  work?  And  most  important,  why  did  this  happen  the  way  that  it  did?   Rather  than  viewing  investigation  or  design  as  a  sequential  process,  proceeding   stepwise  from  questioning  to  planning  to  solution,  it  is  often  more  accurate  and   productive  to  view  sense-­‐making  as  incrementally  building  understanding  or   solutions,  engaging  in  cycles  of  questioning,  gathering  data  through  investigations   or  tests  of  part  of  a  design,  making  sense  by  developing  or  revising  models,   constructing  explanation  or  solutions,  and  then  evaluating  progress  and   determining  where  to  go  next.  In  order  for  this  to  be  effective,  we  have  to  have  a   way  to  keep  track  of  what  we  are  figuring  out  along  the  way.  There  are  many   effective  strategies  for  this  kind  of  work  (See  Windschitl  &  Thompson,  2014  for  a   nice  summary  of  some  tools).  The  practices  central  here  are  analyzing  data,   mathematics  and  computational  thinking,  feeding  into  the  processes  of  developing   or  revising  models,  constructing  explanations  and  designing  solutions  as  well  as   communicating  information.  Argumentation  is  especially  important  in  a  classroom   since  there  are  often  many  different  ongoing  students  ideas,  which  the  class  will   need  to  compare,  evaluate,  and  eventually  reach  consensus  about.     (4)  How  does  it  all  fit  together?  What  does  it  mean?       How  does  what  we  have  figured  out  answer  the  questions  or  solve  the  problems  we   identified?  How  do  we  decide?  Ultimately  our  goal  is  to  develop  deep  understanding   of  the  disciplinary  core  ideas  that  help  us  account  for  how  the  world  works  the  way   it  does.  We  have  to  continually  check  our  developing  ideas  against  the  phenomena  

that  inspired  our  work  to  begin  with.  Two  practices  that  are  central  in  this  process   are  developing  and  using  models  and  constructing  explanations  and  designing   solutions.  In  other  words,  we  have  to  see  if  what  we  have  figured  out  so  far  is  helpful   in  answering  the  questions  that  drove  us  at  the  beginning  of  our  inquiry.  This  cycle   of  wondering,  working  to  figure  out  and  checking  if  our  emerging  ideas  are  actually   useful  to  satisfy  our  initial  wondering  is  at  the  core  of  deep  engagement  in  the   practices.  As  has  been  necessary  throughout  the  process,  we  also  need  to  engage  in   principled  evaluation  of  competing  ideas  through  argumentation  and  reaching   consensus  as  well  as  the  processes  of  communicating  information.       As  this  quick  overview  has  shown,  scientific  and  engineering  practices  need  to  be   used  together  to  work  towards  for  making  sense  of  the  world.  This  idea  is   summarized  in  Figure  1.     Asking Developing Questions & Using & Defining Models Problems Obtaining, Planning Evaluating, & & Carrying Communicating Information Out What are we trying to figure out? Investigations Analyzing & How will we figure this out? Interpreting How can we keep track of ideas? Data Engaging How does it all fit together? In Argument From Evidence Constructing Using Explanations Mathematical & Designing & Computational Solutions Thinking     Figure  1:  The  science  and  engineering  practices  work  together  to  achieve  four  parts  of   sense-­‐making.  This  diagram  illustrates  how  the  practices  always  operate  in   conjunction  with  each  other.    

We  saw  a  number  of  these  connections  in  Scenario  2  from  Ms.  Lee’s  classroom.  In   her  scenario,  we  saw  the  practice  of  asking  questions  as  students  brainstormed   questions  about  why  objects  appear  to  move  in  the  sky.  The  class  conducted  an   investigation  of  rising  and  setting  times  for  the  Moon,  analyzed  data  (their  own  and   a  secondary  data)  and  constructed  an  initial  model  that  included  the  explanatory   idea  that  the  Moon  orbits  the  earth.  This  led  to  another  question  about  why  the   shape  changes  in  patterns,  which  then  led  to  the  main  investigation  using  a  physical   depiction  of  the  Moon,  Sun,  Earth  system.  Students  worked  together  to  construct  an   explanatory  model  of  why  the  Moon  shape  changes,  shared  their  models  and   engaged  in  argumentation  from  evidence  to  reach  consensus  on  a  shared   explanation.       The  idea  that  practices  work  together  for  making  sense  of  the  world  is  one  of  the  most   important  themes  in  this  book.  The  goals  of  the  Framework  and  NGSS  will  not  be  met   if  lessons  are  taught  by  focusing  on  one  practice  at  a  time.  The  practices  work  with   one  another  to  help  us  understand  how  and  why  the  world  works  and  for   engineering  how  to  design  solutions  to  problems.  Focusing  on  the  larger   investigation  context  leads  to  authentic  and  purposeful  reasoning  and  connected   practices.  Looking  back  at  our  questions  informing  the  sense-­‐making  process,  we   can  see  how  thoughtful  work  can  happen  in  many  places  –  during  initial  attempts  to   understand  phenomena,  efforts  to  make  sense  of  patterns  in  empirically-­‐related   phenomena  and  related  scientific  theories,  and  in  applying  the  revised  theories  and   models  to  explain  and  predict  other  phenomena  in  the  world.       The  Structure  of  the  Book     We  compiled  this  book  in  order  to  help  teachers  of  science  grapple  with  three  sets  of   questions  related  to  the  NGSS  Practices,  and  we  suggest  that  you  use  these  to  guide   you  as  you  read.       Section  I:  What  is  the  vision  put  forth  in  the  Framework  and  NGSS  and  how  will   engaging  students  in  science  and  engineering  practices  help  us  improve  our  science   classrooms?       The  chapters  in  Section  I,  including  this  one,  take  a  big  picture  view  and  show  how  a   focus  on  practices  has  the  potential  to  re-­‐shape  our  classrooms.  This  chapter   introduces  the  book  and  explores  the  fundamental  shifts  implied  by  engaging   students  in  science  and  engineering  practices;  Chapter  2,  authored  by  two  of  the   authors  of  the  Framework,  explicates  the  rationale  for  focusing  on  science  and   engineering  practices  and  traces  the  development  of  this  idea  from  earlier  notions   of  inquiry  in  science;  Chapter  3  examines  some  important  questions  about  equity  in   the  science  classroom  and  explores  how  a  focus  on  practices  is  in  line  with  goals  we   have  to  make  science  accessible,  intelligible  and  meaningful  for  all  learners;  Chapter   4  takes  up  the  question  of  our  broader  societal  goals  for  science  literacy  and   explains  the  connections  between  these  goals  and  the  focus  on  engaging  students  in   science  and  engineering  practices.      

  Section  II:  What  are  the  practices,  and  what  does  it  look  like  to  engage  students  in  the   practices  within  the  classroom?       Section  II  of  the  book  addresses  this  question.  Chapters  5  through  13  take  up  each  of   the  practices  and  provide  rich  descriptions  and  rationales  for  each  practice  as  well   as  detailed  accounts  of  what  is  involved  in  engaging  students  in  them.  There  is  a   chapter  dedicated  to  each  of  the  eight  practices  described  in  the  Framework.  Section   II  also  contains  a  chapter  that  discusses  aspects  of  the  eight  practices  that  focus  on   engineering  design.  Although  each  chapter  focuses  on  a  single  practice,  an  important   section  in  each  chapter  highlights  the  connections  between  and  among  the  practices.         Section  III:  How  do  I  get  started?  What  are  some  ways  to  begin?       The  individual  practices  chapters  in  section  II  each  contain  some  ideas  about  how  to   begin  to  involve  your  students  in  the  practices.  In  addition,  section  III  of  the  book   provides  some  guidance  for  planning  NGSS  aligned  instruction,  for  creating   classroom  discourse  environments  that  create  opportunities  for  students  to  engage   in  the  practices  and  the  concluding  chapter  highlights  some  key  ways  to  get  started.       This  book  is  intended  to  help  teachers  of  science  to  develop  a  strong  understanding   of  what  the  Practices  strand  of  the  Framework  and  NGSS  is  all  about.  Engaging   students  in  practices  can  help  motivate  them,  build  their  curiosity,  help  them   understand  how  science  is  connected  with  their  lives,  and  is  more  consistent  with   how  science  and  engineering  is  practiced  in  the  world.       Conclusion     Despite  past  reforms  and  the  best  of  intentions  of  all  of  us  as  educators,  there  are   many  ways  in  which  our  typical  science  instruction  continues  to  send  the  message   to  students  that  all  they  need  to  do  is  know  some  information,  use  the  right   vocabulary,  follow  the  right  procedures  and  they  have  learned  science.  But,  this  is   not  what  science  learning  should  be  about.  People  do  need  to  know  some   information  to  make  sense  of  the  world,  but  simply  learning  a  set  of  facts  will  not   guarantee  that  one  will  come  to  understand  or  make  sense  of  anything.  Likewise,   engaging  students  in  a  hand-­‐on  discovery  with  little  conceptual  guidance  or   grounding  will  not  lead  students  to  robust  and  accurate  understanding  of  the   science  ideas  –  because  the  data  don’t  speak  for  themselves  about  what  is  the  most   scientifically  accurate  and  explanatory  idea.     The  emphasis  on  science  and  engineering  practices  in  NGSS  attempts  to  continue   key  themes  from  past  efforts  to  reform  science  classrooms,  integrating  inquiry  and   the  doing  of  science,  and  pushing  toward  conceptual  understanding  rather  than   focusing  solely  on  facts,  definitions,  and  formulae.  The  sense-­‐making  emphasis  in   NGSS  focuses  on  incrementally  building  and  testing  explanatory  models,  and   working  together  with  argumentation  to  compare  ideas  and  reach  consensus.  The  

practices  reflect  the  different  ways  that  we  need  to  be  able  to  develop  and  use   knowledge  to  make  sense  of  the  natural  and  designed  world  –  explaining   phenomena,  critiquing  arguments,  evaluating  explanations  or  tradeoffs  in  an   engineering  argument.  Three-­‐dimensional  learning  in  Framework  and  NGSS  is   about  using  the  practices  to  develop  and  use  the  disciplinary  and  crosscutting  ideas   of  science.  If  we  define  science  in  this  way,  preparing  literate  adults  requires  that  we   give  students  experience  engaging  in  the  practices  of  science  and  engineering  in   order  to  build  and  gain  facility  with  science  and  engineering  ideas.       In  this  chapter,  we  have  introduced  the  theme  of  the  book  –  the  contrast  between   merely  knowing  about  scientific  ideas  compared  to  figuring  out  how  the  natural  and   designed  world  works  by  formalizing,  sharing,  and  refining  those  idea  within  a   community.  We  call  this  figuring  out  aspect,  sense-­‐making.  NGSS  reflects  the   contrast  between  asking  students  to  know  and  recall  scientific  information  that  has   been  given  to  them,  compared  to  scientific  ideas  and  practices  as  useful  tools  for   reasoning  and  making  sense  of  the  world.  This  is  a  radical  shift  in  framing  what   learning  science  is  about.  This  book  is  intended  to  give  you  ideas  about  how  to   engage  students  in  practices  for  the  purpose  of  making  sense  of  the  world  –  by   sharing  ideas  and  strategies  about  the  practices  and  ideas  about  how  to  get  started   using  the  Framework  and  NGSS  in  your  own  teaching.  We  hope  that  you  are  inspired   by  this  radical  shift,  and  that  this  book  is  helpful  to  you  as  you  learn  ideas  and   strategies  for  moving  this  vision  forward  so  that  all  learners  can  participate  in   science  and  engineering  practices  to  make  sense  of  the  world.       References     Banilower,  E.,  Smith,  P.  S.,  Weiss,  I.  R.,  Malzahn,  K.  A.,  Campbell,  K.  M.,  &  Weiss,  A.  M.   (2013).  Report  of  the  2012  national  survey  of  science  and  mathematics   education.  Chapel  Hill,  NC:  Horizon  Research  Inc,  1-­‐309.     Barton,  A.  M.  (2001).  The  Moon  Also  Rises:  Investigating  Celestial  Motion  Models.   Science  Teacher,  68(6),  34-­‐39.     National  Research  Council.  (2012).  A  framework  for  K-­‐12  science  education:  Practices,   crosscutting  concepts,  and  core  ideas.  Washington,  DC:  National  Academies  Press.     NGSS  Lead  States.  (2013).  Next  Generation  Science  Standards:  For  states,  by  states.   Washington,  DC:  The  National  Academies  Press     Windschitl,  M.,  &  Thompson,  J.  J.  (2013).  The  Modeling  Toolkit:  Making  Student   Thinking  Visible  with  Public  Representations.  The  Science  Teacher,  80(6),  63.      

Commentary pubs.acs.org/jchemeduc Why Ask Why? Melanie M. Cooper* Department of Chemistry, Michigan State University, Lansing, Michigan 48824, United States ABSTRACT: There is a strong case to be made that the goal of science is to develop explanatory theories that help us organize our understanding and make predictions about the natural world. What then, is the goal of science education? What is it that we want students to know and be able to do, and how do we achieve these goals? Here, I argue that one overarching goal is to help students construct causal, mechanistic explanations of phenomena. In chemistry this means we are working to help students use their under- standing of molecular level interactions to explain and predict macroscopic events. Furthermore, while constructing explanations is an important goal in itself, the very act of constructing explanations helps students develop a deeper understanding, and provides the kind of intellectual satisfaction that memorizing facts cannot. I hope to convince you that our current approaches to assessing student learning are, in fact, all too often counterproductive and almost certainly contribute to students’ inability to connect ideas and develop a useful understanding of chemistry and that hat these assessments send the wrong message about what chemistry means (and why it is valuable). I will offer some suggestions how we might design more meaningful approaches to curriculum development and assessment of student understanding. After reading this essay, I hope that I will have convinced you that: (i) if we value something, we must assess it; (ii) we cannot assume students will construct a coherent framework from the fragments we teach; and (iii) we must design assessments that provide us with enough evidence to make an argument that the student understands. KEYWORDS: First-Year Undergraduate/General, Chemical Education Research, Testing/Assessment, Curriculum FEATURE: Award Address ■ INTRODUCTION that it is readily accessible. This contrasts with the often frag- mented jumble of facts and concepts that beginning learners There is general agreement that learning environments that (and, as it turns out, many who are not beginners) frequently support student active engagement are effective in improving work with.2,6 Yet in most of our introductory courses there course grades and retention, particularly for underprepared is a tendency to favor breadth over depth, often trying to students,1 but there is less evidence about what students are provide courses that “cover” everything that might be deemed actually learning in these courses. There is an implicit assump- important for future chemists, despite the fact that most tion that the grades students earn in our courses are a reflection students in introductory courses will never take another chem- of their understanding of the subject matter, and therefore, istry course. There is often a focus on an extensive array of facts increased success and retention rates in a course must mean and skills that are duly tested−but almost never synthesized that more students have learned more deeply. But we know into a deeper, resilient and applicable understanding. Without that even “good” students, who have done all that we asked of an underlying explanatory framework, students are often unable them, often emerge from our courses with profound mis- to put the pieces together themselves; it is no wonder then that conceptions and a fragmented understanding of important students are rarely able to make sense of what we tell them, and concepts, and there is still little evidence that students are able often resort to memorization−not because it is easier, but to transfer what they have learned to a new situation in the because they do not have any alternative. The current model in same course, never mind across courses or disciplines.2,3 The which most introductory courses are a “mile wide and an inch literature is rife with descriptions of what students cannot do,4 deep” is, in essence, providing students with the fragments to but sadly, there are far fewer reports of successful strategies that build a coherent framework without providing the experiences report strong evidence of sustainable improvements in student and context that allow the necessary connections to be made. understanding. While it is easy to blame students (for laziness, By analogy, we are providing students with the building lack of motivation, or general inability to do the work), in fact materials (bricks), and expecting them to build the Taj Mahal the only constant from year to year is our efforts to teach these by themselves as shown in Figure 1. students, and in this, we seem to be failing a large fraction of them. The question then is, what features characterize the curricular structures and instructional activities that can provide the nec- We know that experts in a field have a robust underlying essary scaffolding so that students can build a strong foundation theoretical understanding of the concepts in their discipline.5 on which to build their subsequent knowledge? A “blueprint” Certainly most scientists have a large database of information at their fingertips and, even more importantly, they have this knowl- edge organized into coherent frameworks and contextualized so © XXXX American Chemical Society and A DOI: 10.1021/acs.jchemed.5b00203 Division of Chemical Education, Inc. J. Chem. Educ. XXXX, XXX, XXX−XXX

Journal of Chemical Education Commentary Figure 1. Experts have assembled their fragments of knowledge (the marble blocks) into a coherent, contextualized, and useful framework (the Taj Mahal). for how STEM education might be redesigned based upon the that can be considered as the disaggregated parts of inquiry. best available evidence is outlined in the National Research The Framework states:6 Council (NRC) A Framework for K−12 Science Education: Practices, Crosscutting Concepts, and Core Ideas.7 While the Scientific explanations are accounts that link scientific theory NRC Framework was developed for K−12, the research and with specific observations or phenomenafor example, they learning theories on which it is based are also applicable to at explain observed relationships between variables and describe least the first two years of college (and probably beyond). The the mechanisms that support cause and effect inferences NRC framework approach is designed to support students as about them. they build their own robust underlying knowledge frameworks This approach is particularly appropriate in chemistry, by identifying the core ideas in each discipline on which all where the causal mechanisms lie at the unseen molecular other concepts can be built.8 Students’ knowledge cannot be level. Although we chemists tend to reserve “mechanism” for installed but must be built up over time, a process that enables the curved arrows that represent the flow of electrons in the student to organize and link these core ideas together to organic reactions, it means so much more. A major goal of form a solid foundation for subsequent learning. However, chemistry is to provide molecular level, mechanistic explan- at the college level, most current efforts aimed at improving ations for macroscopic phenomena, which includes for example, student understanding tend to be focused on developing how and why energy is transferred, why particles interact, why student-centered, interactive learning environments that are chemical reactions occur, and why, in a closed system, reactions often layered on top of the existing curriculum.9,10 reach equilibrium. I should also note that the scientific practices of constructing explanations, arguments,12 and models13 are While there are many potential ways to support students as closely linked, and while in this paper I will be focusing mainly they develop a deep understanding, in this paper I will discuss on explanations, much of what I have to say is also relevant to an approach that focuses on helping them develop scientific these other practices. explanations. Over the years, evidence has accumulated indi- The science education literature is clear about the benefits of cating that getting students to articulate not just that something constructing explanations.14−17 For example, “Asking deep happens, but why something happens, is exceedingly valuable explanatory questions” is one of only two instructional practices for a variety of reasons. Researchers from cognitive science, that are considered to have strong evidence for their efficacy, as philosophy, and science education have all weighed in on the reported in the IES practice guide18 (Strong evidence requires value of explanation as an instructional practice, and while each multiple studies, across a wide range of disciplines and age discipline has a somewhat different approach and rationale, ranges that show improvement in learning). A wide range of there is agreement that helping students construct explanations instructional approaches that support students in their develop- is crucial. ment of explanations have been proposed. Some combine the practices of explanation and argumentation, which are closely ■ WHAT IS A SCIENTIFIC EXPLANATION? linked, using a claim-evidence-reasoning framework that originated with Toulmin’s argumentation schemes.19 While Just what we mean by explanation may depend on the context, there has been some discussion of the differences between the but it has been noted that explanations can be characterized two,12,16,20 for our purposes it is not necessary to go beyond by the kinds of thought that are evidenced. The Institute of the idea that having students construct explanations (and/or Education Sciences (IES) states11 arguments) using evidence or scientific principles and reasoning is supported by a large body of evidence that shows improved Shallow knowledge taps basic factual or skill knowledge, student learning. whereas deep knowledge is expressed when learners are able It is the act of constructing explanations that provides the to answer “why” questions and describe causal relationships well-documented beneficial effects. That is, simply reading or between facts or concepts. hearing an explanation does not promote the same kind of and cognitive engagement, nor does summarizing the textbook, or By (deep) explanations, we mean explanations that appeal taking notes. Constructing explanations requires students to to causal mechanisms, planning, well-reasoned arguments, engage in a wide range of cognitive activities and skills; it and logic. requires that students articulate their thoughts and that they Similarly, the NRC Framework for Science Education7 defines connect and reflect on and consider their ideas. For these constructing explanations as one of the eight science practices B DOI: 10.1021/acs.jchemed.5b00203 J. Chem. Educ. XXXX, XXX, XXX−XXX

Journal of Chemical Education Commentary Figure 2. Sequence by which properties can be determined from a molecular structure. reasons, a number of researchers have incorporated construct- An Example from Structure−Property Relationships ing explanations into proposed pedagogical frameworks. For example, Chi has proposed that as student engagement with Much of our work has focused on student understanding of learning materials progresses “from passive to active to con- structure−property relationships, and therefore, I will use this structive to interactive” (ICAP), their learning will improve.21 construct as an example. As might be expected, we have The “constructive and interactive” parts of this framework rely documented a litany of problems including student difficulties on student-generated explanations, predictions, and arguments. with drawing Lewis structures,27 the problems that students Linn’s knowledge integration framework22,23 also supports have with decoding the information contained in such struc- students as they make connections between key concepts to tures,27−29 and the startling finding that the majority of students develop causal explanations. in our studies represent intermolecular forces as interactions within small molecules.30 Indeed, when we interviewed students Philosophers of science have also weighed in on the value about how they use structures to predict properties, we found and nature of explanations. Strevens24 has written that most little evidence that they used much of what they had been taught, contemporary philosophers believe that “to understand a and instead relied on rules and heuristics.2 phenomenon is to grasp how the phenomenon is caused” and that there is “no understanding without explanation.” If we step back and think about just what is involved, it Indeed, if we know how something is caused, then it is much becomes clearer why students have such difficulty. Consider, more likely that we will be able to make testable predictions for example, the sequence of steps that students must follow to about the future when conditions change. Gopnik,25,26 goes predict the properties of a substance from its structure even further and proposes that understanding is an evolutionary (Figure 2). The student must be able to (1) draw the Lewis adaptation. She postulates that in order to survive humans have structure accurately; (2) use the Lewis structure appropriately evolved the need to develop theories about how the world to determine the electron pair geometry and molecular shape; works, and that the pleasure we get when we understand (3) make predictions based on relative atomic electronegativities to something−that is, when we construct a causal explanation−is determine bond polarities and use molecular shape to determine our built in reward system. overall molecular polarity; (4) use the molecular polarity to predict the types and strengths of intermolecular forces, and (5) synthesize So, there is general agreement, helping students construct these factors, together with an understanding of the intermolecular causal, empirically based explanations about phenomena is potential energy changes and the influence of thermal energy, to crucial for a number of reasons: it is an important goal of predicate interaction stabilities and their implications for the science education in itself, it improves learning, and perhaps macroscopic physical and chemical properties of a substance. most intriguingly it can provide an intellectual satisfaction that memorizing facts and performing rote calculations typically The road from structure to properties is complex, and cannot. However, it is a truth (almost) universally acknowl- although experts can and do “chunk” parts of this pathway, edged (at least by students) that the tests and quizzes we use to for beginners we must acknowledge how difficult it is to per- assign grades signify what is actually important. Regardless of form these operations in sequence. Consequently, we dutifully our intent, the activities we use to grade students send an im- assess each step, separately, as if each were the important idea portant, and sadly perhaps the only, message that some (Table 1). For most of us−and particularly those of us who students hear. Even the most well constructed and evidence- teach large enrollment courses−the most common types of based learning activities and curricular materials may not questions (the ones that appear on many of our tests, and that produce observable or meaningful learning gains if we do not are prevalent in publishers test banks), typically target the recall develop assessments that actually assess what is important. In of these fragments of knowledge, the steps in the pathway, fact, it is entirely possible that many reform efforts might have rather than the end goal. failed to show improvements because the assessments used to measure reform were not commensurate with the reform itself. However, as the saying goes, “when everything is assessed, everything becomes important”, and when we emphasize the C DOI: 10.1021/acs.jchemed.5b00203 J. Chem. Educ. XXXX, XXX, XXX−XXX

Journal of Chemical Education Commentary Table 1. Steps in the Structure−Property Pathway and the Ways They Are Commonly Assessed Step Action Common Instructional Strategies Common Assessment Items Identify which is the correct structure, or construct a Lewis structure 1 Draw the Lewis structure Usually rules based, often where completing “octets” is the goal Identify or list the electron pair geometry and/or molecular shape of a particular 2 Determine electron pair molecule geometry and molecular shape Usually rules based (VSEPR) Identify whether a particular molecule is polar 3 Determine bond and molecular Know electronegativity differences, and polarities vector addition of bond dipoles Identify which molecule will exhibit hydrogen bonding, or what intermolecular forces are present in the liquid phase of a particular compound 4 Determine the strengths and Understand the electron density types of intermolecular forces distribution in the molecule Ranking tasks: Identify which compound has the highest boiling point, etc. 5a Determining physical properties Identifying (relative) strengths of Ranking tasks: Identify which compound is most acidic, etc. Predicting interactions between molecules outcomes of reactions 5b Determining chemical properties (e.g., acidity) Look for particular groupings of atoms pieces rather than the whole, it undermines the ultimate goal. compound has, the higher its boiling point, because they thought We ask questions about identifying correct structures, or deter- that it takes energy to break bonds. So, while ranking tasks may mining electron pair geometry, or which of these molecules will seem like an efficient way to test student understanding, exhibit hydrogen bonding, but rarely do we ask students to unless accompanied by extensive student reasoning, even when synthesize what they know to answer the far more important answered correctly, they do not provide evidence that the types of questions, about how and why molecular structure student understands. Clearly, there is a need for other types of predicts macroscopic behavior. assessment that assess deeper learning. No one is disputing that students must have the skills and ■ ASSESSMENT AS A PROCESS FOR ELICITING knowledge to perform each step, but what we have to re- EVIDENCE ABOUT WHAT STUDENTS KNOW AND member is that there is no reason to teach students these CAN DO intermediate steps if we do not help them understand the ultimate purpose for why they are learning them. Most We can never really know what a student knows; all we can do questions do not provide explicit linkages from one step to is make assumptions from the answers that they give on tests another and appear (to the student) to be isolated fragments. and quizzes. As we have seen most tests and quizzes assess facts We should not be surprised then that many students do not or skills, they may test whether a student knows that something know the purpose of Lewis structures:27 without context, they happens, but rarely why it happens. The question then is, how have no meaning. The tenets of meaningful learning31,32 hold do we design tasks that both help students to (for example) that any new knowledge must be solidly connected to students explain how and why a molecular structure can be used to prior knowledge, and that students must also understand the predict properties, and also provide evidence that the student purpose of the new knowledge, so that they can choose to learn has indeed used scientific reasoning to engage with the task? meaningfully rather than in a rote, shallow fashion. While it is clearly easier to assess the individual components, doing While it is beyond the scope of this paper to discuss this sends a message to the students not only about what is assessment design in detail, the NRC report Knowing What important, but also that each task exists in isolation with no Students Know37 is an excellent resource on this topic. The ultimate purpose. report identifies three essential aspects of assessment: cognition (what it is that you want student to know and do), observation Certainly, multiple choice tests are reasonable ways to test (what you will ask students to do and what observations low level knowledge, providing reliable and valid information you will make), and interpretation (how you will interpret the about some aspects of what students know.23 Although multiple observations). There are a number of approaches to the choice questions can be made difficult (a common, but flawed development of assessments that make use of this so-called measure of “rigor”), there is little evidence that they are useful assessment triangle,38−42 including Evidence Centered Design for measuring deep thinking33,34 or the complex constructs that (ECD).41 The central tenet of these approaches is the idea that are involved in structure−property relationships. Even multiple- we must collect evidence that students understand the construct choice questions that are intended to assess students’ ability to being assessed, so that we can make an argument about what it use their knowledge to make predictions may not evoke the is that students know and can do. Clearly, there are many kind of thinking that is intended. For example, as shown in possible ways to elicit such evidence, but it seems clear that a Table 1, a common approach is to ask students to rank, for well-constructed explanation should provide evidence that the example, boiling points (or solubility or acidity or any number student understands a phenomenon. of other properties), the implication being that students who can do so correctly are making predictions based on their Scaffolding Student Constructed Explanations understanding of why the compounds have this particular property. However, several studies2,35,36 have shown that even Asking students to construct explanations is an excellent students who make the correct choices often use strategies that formative assessment strategy and, with appropriate rubrics, can are not scientifically valid. Often students rely on heuristics, be useful in summative assessments. One thing is certain, rules, and test taking strategies. As noted previously,2 we asked however, we cannot expect students to develop the ability to students to use molecular structures to predict relative melting construct explanations without coaching and practice. In our and boiling points. Many students chose the right answer (in experience, asking students to explain how a phenomenon is fact, they mentioned that they had seen similar questions on caused, without rather extensive support and practice, usually tests), while providing faulty reasoning to justify their choices. results in shallow responses that lack the reasoning component For example, some students told us that the more bonds a we are looking for. If the goal is to have students develop the ability to generate coherent causal explanations for complex D DOI: 10.1021/acs.jchemed.5b00203 J. Chem. Educ. XXXX, XXX, XXX−XXX

Journal of Chemical Education Commentary Table 2. An Example of a Question Designed To Elicit Evidence That Students Can Reason about the Link between Structures and Properties Question Rubric Dimethyl ether (CH3OCH3) and ethanol (CH3CH2OH) have the Correct Lewis structures same molecular formula, but one of these compounds is a liquid at room temperature and the other is a gas. Draw Lewis structures for Claim: Ethanol is a liquid at room temperature, while dimethyl ether is not each substance and use them to help you determine which substance is a liquid. Provide a molecular level explanation for your Evidence: Ethanol is capable of hydrogen bonding, while dimethyl ether is not. choice, being sure to include a discussion of the interactions and energy changes involved. Reasoning: The interactions between ethanol molecules are stronger than the interactions between dimethyl ether molecules; Therefore, more energy is needed to overcome the attractions between ethanol molecules compared to dimethyl ether molecules; More energy needed to overcome attractive forces between molecules corresponds to a higher temperature, meaning ethanol will boil at a higher temperature. phenomena, then they must be given ample opportunities respond to the questions. Although this is more time- to practice this kind of activity, while receiving appropriate consuming than grading all multiple choice, or hand grading feedback. calculations, it sends an important message to students; the constructed response sections of the exam are significant and There are a number of studies reporting ways to help must be taken seriously (especially since they are worth 50% of students develop complete explanations,14,15 and most agree the exam grade). Table 2 gives an example of a potential that scaffolding the explanation can produce a more coherent question and grading rubric that emphasizes the reasoning that and rich response,20 for example, by situating the explanation in we are expecting students to use. a phenomenon, or having students draw models and diagrams. We have also been investigating how to encourage students It is also possible to write multiple-choice questions that to construct coherent explanations using our online system, approximate this claim, evidence and reasoning scaffold for beSocratic.43,44 We pose questions that ask students to draw explanation questions. For example, Box 1 shows a question diagrams, molecular-level representations, and write explan- ations in response to prompts. We learned that simply asking Box 1. A Multiple-Choice Question That Uses an students to “explain” did not elicit the kind of rich discussion Explanation Format in the Answer Choices (Claim, that we were hoping for. This is not surprising, since most of Evidence, Reasoning) our students are acculturated to assessment items that typically require them to choose an answer or calculate a number. Which is a stronger base? CH3NH2, or CH3OH One approach involves a scaffolded framework where we A. CH3NH2, because N is more electronegative than O, tell students that an explanation should have (i) a target or and therefore is not as able to donate its lone pair into a claim (what the explanation is about), (ii) the scientific bond with an acid. principle or evidence on which the explanation is based, and (iii) the reasoning that links the two. It is the reasoning part of B. CH3NH2, because N is less electronegative than O, and the explanation that not only provides the most useful insight therefore is better able to donate its lone pair into a into student thinking, but also is almost always missing from bond with an acid. student-generated explanations. Using this scaffold, our goal is to help students develop the connections that lead them back C. CH3OH, because O is more electronegative than N, and to the core ideas that underlie all that they are learning. Other therefore is not as able to donate its lone pair into a approaches involve asking students to draw a molecular level bond with an acid. picture or diagram, or a graph and use it to help them explain a phenomenon. In general, we try to provide “hints” in the D. CH3OH, because O is less electronegative than N, and question about what we are looking for in terms of an explana- therefore is better able to donate its lone pair into a tion. So for example, a question about why one substance has a bond with an acid. higher boiling point than another might include a reminder to discuss the forces and energy changes that are involved when a where students must make a claim, and choose the correct substance boils. explanation. This type of question is easier to grade, and while it may be tempting to return to all multiple choice tests, if this Currently, I teach sections of over 400 students−a less than is the case, we may also be tempted to abandon formative optimal situation that precludes individual feedback for explanation tasks−and then we are back to square one. complex student responses. After every class, students construct their answers to homework questions; they draw and write This brings me to one final (important) point; if we expect using our beSocratic system.44 In the next class, we show students to develop deep understanding, it is highly unlikely to examples of student homework to discuss what factors are happen with the current “mile wide inch deep” curricula that required for a complete answer to the homework. Our are encompassed by traditional textbooks. Unless students have examinations combine both multiple choice and free response the opportunity and time to develop the ability to reason about questions that require students to construct explanations and phenomena, they are unlikely to be able to produce coherent arguments, models, diagrams, and molecular pictures. Student explanations. Focusing on the “big ideas” in the context of responses on the constructed answer section of the exam are scientific practices that put knowledge to use takes time, and graded by graduate students and instructors using rubrics makes it impossible to “cover” 25 or 30 chapters. There is a developed by considering what we deem to be acceptable growing body of evidence that these important ideas should be evidence of student understanding. These rubrics evolve over developed over time in a carefully scaffolded progression.8,45 time as we collect more information about how students One possible approach is exemplified by our general chemistry curriculum, Chemistry, Life, the Universe and Everything (CLUE). CLUE is organized around three interconnected core ideas, structure, properties, and energy that are linked together E DOI: 10.1021/acs.jchemed.5b00203 J. Chem. Educ. XXXX, XXX, XXX−XXX

Journal of Chemical Education Commentary by the idea of forces and interactions.46 Each of these ideas is (3) National Research Council. Education for Life and Work: developed and connected throughout the curriculum starting Developing Transferable Knowledge and Skills in the 21st Century; with the structure, properties and energy changes associated National Academy Press: Washington, DC, 2012. with atoms and progressing to the interconnected networks (4) National Research Council. Discipline-Based Education Research: of chemical reactions that are the basis of simple biological Understanding and Improving Learning in Undergraduate Science and processes. These core ideas of chemistry are developed as Engineering; Singer, S. R, Nielson, N. R., Schweingruber, H. A., Eds.; students use their knowledge to explain and model chemical National Academies Press: Washington, DC, 2012. phenomena, to help them construct for themselves a frame- (5) National Research Council. How People Learn: Brain, Mind, work on which to build for the future.47 We have emerging Experience, and School.; National Academies Press: Washington, DC, evidence from this curriculum that not only are students more 1999. likely than those in traditional courses to understand (6) Bodner, G. M. I Have Found You an Argument: The Conceptual fundamental concepts, but that this improvement is maintained Knowledge of Beginning Chemistry Graduate Students. J. Chem. Educ. throughout organic chemistry.48 1991, 68 (5), 385−388. (7) National Research Council. A Framework for K-12 Science ■ SUMMARY Education: Practices, Crosscutting Concepts, and Core Ideas; National Academies Press: Washington, DC, 2012. I hope that I have convinced you of the importance of asking (8) Krajcik, J. S.; Sutherland, L. M.; Drago, K.; Merritt, J. The students to explain: to themselves, to others, and perhaps most Promise and Value of Learning Progression Research. In Making It importantly (if we want lasting change) on the assessments that Tangible: Learning Outcomes in Science Education; Bernholt, S., we use to assign grades. There is an enormous evidence base Neumann, K., Nentwig, P., Eds.; Waxmann: Münster, 2012; pp for the efficacy of using student-constructed explanations as a 261−284. learning tool, and yet, we rarely use them in our assessment (9) Gafney, L.; Varma-Nelson, P. Peer-Led Team Learning: practices. I hope you will take up the banner, design assess- Evaluation, Dissemination, and Institutionalization of a College ments that provide explicit evidence for the construct you want Level Initiative. In Inovations in Science Education and Technology; to measure, and not be satisfied by merely assessing low level Cohen, K., Ed.; Springer: Weston, MA, 2008. knowledge. In conjunction with this, I hope you will be inspired (10) Process Oriented Guided Inquiry Learning (POGIL); Moog, R. to redesign your curricula so that students can build a founda- S., Spencer, J. N., Eds.; American Chemical Society: Washington, DC, tion of core ideas that can be used as the basis for predicting how 2008. novel systems will behave and for use when needed. If we do not (11) Organizing Instruction and Study to Improve Student Learning: ask students to put together coherent explanations, we cannot be What Works Clearinghouse http://ies.ed.gov/ncee/wwc/ surprised when even our best students do not understand. PracticeGuide.aspx?sid=1 (accessed Feb 2015). (12) Osborne, J. F.; Patterson, A. Scientific Argument and ■ AUTHOR INFORMATION Explanation: A Necessary Distinction? Sci. Educ. 2011, 95 (4), 627− 638. Corresponding Author (13) Clement, J.; Rea-Ramirez, M. A. Model Based Learning and Instruction in Science; Springer: Secaucus, NJ, 2008. *E-mail: [email protected]. (14) McNeill, K. L.; Krajcik, J. S. Supporting Grade 5−8 Students in Constructing Explanations in Science: The Claim, Evidence, and Reasoning Notes Framework for Talk and Writing; Pearson: Boston, MA, 2011. (15) Songer, N. B.; Gotwals, A. W. Guiding Explanation The authors declare no competing financial interest. Construction by Children at the Entry Points of Learning Melanie M. Cooper, Professor of Chemistry and Lappan- Progressions. J. Res. Sci. Teach. 2012, 49 (2), 141−165. Phillips Professor of Science Education at Michigan State (16) Berland, L. K.; Reiser, B. J. Making Sense of Argumentation and University, Lansing, Michigan, received the 2014 American Explanation. Sci. Educ. 2009, 93 (1), 26−55. Chemical Society Award for Achievement in Research for the (17) Chi, M. T.; Bassok, M.; Lewis, M. W.; Reimann, P.; Glaser, R. Teaching and Learning of Chemistry, sponsored by Pearson Self-Explanations: How Students Study and Use Examples in Learning Education, on March 17, 2014, in Dallas, Texas. This paper is To Solve Problems. Cogn. Sci. 1989, 13 (2), 145−182. adapted from her award address. (18) Karpicke, J. D.; Roediger, H. L. The Critical Importance of Retrieval for Learning. Science 2008, 319 (5865), 966−968. ■ ACKNOWLEDGMENTS (19) Toulmin, S. E. The Uses of Argument; Cambridge University Press: Cambridge, U.K., 2003. The author would like to thank Mike Klymkowsky and Sonia (20) Kang, H.; Thompson, J.; Windschitl, M. Creating Opportunities Underwood for helpful suggestions and edits. This work is for Students To Show What They Know: The Role of Scaffolding in supported by the National Science Foundation under DUE Assessment Tasks. Sci. Educ. 2014, 98 (4), 674−704. 0816692, DUE 1043707 (1420005), and DUE 1122472 (21) Chi, M. T.; Wylie, R. The ICAP Framework: Linking Cognitive (1341987). Any opinions, findings, conclusions, or recommen- Engagement to Active Learning Outcomes. Educ. Psychol. 2014, 49 dations expressed here are those of the authors and do not (4), 219−243. necessarily reflect the views of the National Science Foundation. (22) Linn, M. C. The Knowledge Integration Perspective on Learning and Instruction. In The Cambridge Handbook of the Learning ■ REFERENCES Sciences; Sawyer, R. K., Ed.; Cambridge Handbooks in Psychology; Cambridge University Press: Cambridge, U.K., 2005; pp 243−264. (1) Freeman, S.; Eddy, S. L.; McDonough, M.; Smith, M. K.; (23) Lee, H.-S.; Liu, O. L.; Linn, M. C. Validating Measurement of Okoroafor, N.; Jordt, H.; Wenderoth, M. P. Active Learning Increases Knowledge Integration in Science Using Multiple-Choice and Student Performance in Science, Engineering, and Mathematics. Proc. Explanation Items. Appl. Meas. Educ. 2011, 24 (2), 115−136. Natl. Acad. Sci. U.S.A. 2014, 111, 8410−8415. (24) Strevens, M. No Understanding without Explanation. Stud. Hist. (2) Cooper, M. M.; Corley, L. M.; Underwood, S. M. An Philos. Sci., Part A 2013, 44 (3), 510−515. Investigation of College Chemistry Students’ Understanding of (25) Gopnik, A. Explanation as Orgasm and the Drive for Causal Structure−Property Relationships. J. Res. Sci. Teach. 2013, 50, 699− Knowledge: The Function, Evolution, and Phenomenology of the 721. F DOI: 10.1021/acs.jchemed.5b00203 J. Chem. Educ. XXXX, XXX, XXX−XXX

Journal of Chemical Education Commentary Theory Formation System. In Explanation and Cognition; Keil, F. C., (45) Corcoran, T.; Mosher, F. A.; Rogat, A. Learning Progressions in Science: An Evidence Based Approach to Reform; RR-63; Consortium for Wilson, R. A., Eds.; MIT Press: Cambridge, MA, 2000. Policy Research in Education: Philadelphia, PA, 2009. (26) Gopnik, A. Explanation as Orgasm*. Minds Mach. 1998, 8 (1), (46) Cooper, M. M.; Klymkowsky, M. W. Chemistry, Life, the 101−118. Universe and Everything: A New Approach to General Chemistry, and (27) Cooper, M. M.; Grove, N.; Underwood, S. M.; Klymkowsky, M. a Model for Curriculum Reform. J. Chem. Educ. 2013, 90, 1116−1122. (47) Cooper, M. M.; Underwood, S. M.; Hilley, C. Z.; Klymkowsky, W. Lost in Lewis Structures: An Investigation of Student Difficulties in M. W. Development and Assessment of a Molecular Structure and Developing Representational Competence. J. Chem. Educ. 2010, 87, Properties Learning Progression. J. Chem. Educ. 2012, 89, 1351−1357. 869−874. (48) Cooper, M. M.; Williams, L. C.; Underwood, S. M.; (28) Cooper, M. M.; Underwood, S. M.; Hilley, C. Z. Development Klymkowsky, M. W. Are Non-Covalent Interactions an Achilles and Validation of the Implicit Information from Lewis Structures Heel in Chemistry Education? A Comparison of Instructional Approaches. Proceedings of the National Academy of Sciences, Instrument (IILSI): Do Students Connect Structures with Properties? submitted for publication. Chem. Educ. Res. Pract. 2012, 13, 195−200. (29) Underwood, S. M.; Reyes-Gastelum, D.; Cooper, M. M. Answering the Questions of Whether and When Student Learning Occurs: Using Discrete-Time Survival Analysis To Investigate How College Chemistry Students’ Understanding of Structure-Property Relationships Evolves. Sci. Educ., in press. (30) Cooper, M. M.; Williams, L. C.; Underwood, S. M. Student Understanding of Intermolecular Forces: A Multimodal Study. J. Chem. Educ. 2015, DOI: 10.1021/acs.jchemed.5b00169. (31) Novak, J. D. A Theory of Education; Cornell University Press: Ithaca, NY, 1977. (32) Bretz, S. L. Novak’s Theory of Education: Human Constructivism and Meaningful Learning. J. Chem. Educ. 2001, 78, 1107−1117. (33) Smith, C. L.; Wiser, M.; Anderson, C. W.; Krajcik, J. S. Implications of Research on Children’s Learning for Standards and Assessment: A Proposed Learning Progression for Matter and the Atomic-Molecular Theory. Meas. Interdiscip. Res. Perspect. 2006, 4, 1− 98. (34) Stern, L.; Ahlgren, A. Analysis of Students’ Assessments in Middle School Curriculum Materials: Aiming Precisely at Benchmarks and Standards. J. Res. Sci. Teach. 2002, 39 (9), 889−910. (35) Maeyer, J.; Talanquer, V. The Role of Intuitive Heuristics in Students’ Thinking: Ranking Chemical Substances. Sci. Educ. 2010, 94, 963−984. (36) McClary, L.; Talanquer, V. Heuristic Reasoning in Chemistry: Making Decisions about Acid Strength. Int. J. Sci. Educ. 2011, 33 (10), 1433−1454. (37) National Research Council. Knowing What Students Know: The Science and Design of Educational Assessment; Pellegrino, J. W., Chudowsky, N., Glaser, R., Eds.; National Academies Press: Washington, DC, 2001. (38) National Research Council. Developing Assessments for the Next Generation Science Standards; The National Academies Press: Washington, DC, 2014. (39) Wilson, M. Constructing Measures: An Item-Response Modeling Approach; Erlbaum: Mahwah, NJ, 2005. (40) Claesgens, J.; Scalise, K.; Wilson, M.; Stacy, A. Mapping Student Understanding in Chemistry: The Perspectives of Chemists. Sci. Educ. 2009, 93, 56−85. (41) Mislevy, R. J.; Almond, R. G.; Lukas, J. F. A Brief Introduction to Evidence-Centered Design; The National Center for Research on Evaluations, Standards, Student Testing (CRESST), Center for Studies in Education, UCLA: Los Angeles, CA, 2003. (42) Towns, M. H. Guide To Developing High-Quality, Reliable, and Valid Multiple-Choice Assessments. J. Chem. Educ. 2014, 91 (9), 1426−1431. (43) Bryfczynski, S. P. BeSocratic: An Intelligent Tutoring System for the Recognition, Evaluation, and Analysis of Free-Form Student Input. Doctoral Dissertation, Clemson University, 2012. (44) Cooper, M. M.; Underwood, S. M.; Bryfczynski, S. P.; Klymkowsky, M. W. A Short History of the Use of Technology to Model and Analyze Student Data for Teaching and Research. In Tools of Chemistry Education Research; Cole, R., Bunce, D., Eds.; ACS Symposium Series; American Chemical Society, 2014; Vol. 1166, pp 219−239. G DOI: 10.1021/acs.jchemed.5b00203 J. Chem. Educ. XXXX, XXX, XXX−XXX


Like this book? You can publish your book online for free in a few minutes!
Create your own flipbook