MEDIA LITERACY TUTORIAL PROTOTYPE Peer and Instructor Assessments Part if the quality review process included an assessment of the online tutorial by two peers and the instructor of ED6615. Their assessments appear below. I examined their assessments and categorized their suggestions for improvement based on the D.E.C.L. factors as proposed by Mann (2005). PEER #1 Prototype: Media literacy tutorial PSI prototype Goals and Objectives Media Literacy Tutorial PSI prototype, shortened to MLTP, was developed by Cory Antonini for teachers who need to teach Media Study in their subject. It is the best PSI that I had the chance to assess, as it incorporates a lot of elements that have been studied in this course. The main goal of the website is available on the homepage, where it is written that “by learning and applying 5 key concepts of Media Literacy, teachers will be better equipped to integrate Communications Media 10 curriculum in their respective courses” (MLTP, 2015). More particularly, learners will have to create a web page or a blog entry, which will demonstrate their understanding of “media studies and actions plans for integrating media and ICT into classroom instruction” (MLTP, 2015). A list of 6 enabling objectives is provided in the position paper and each enabling objective is also written at the beginning of each module. The website has 3 concepts, followed by a quiz, then 2 more concepts, before finishing with a culminating test. Objectives and goals are well written, following the ABCD format. Moreover, the use of a table to give an overview of all the objectives, sorting them by types and highlighting the cognitive process and the knowledge dimension, gives a clear view to learners about what achievement is expected after each module. Delivery and Environment This prototype is one of the best web APPs designed for a specific school that I have ever seen. The website is well designed and truly includes a media mix that will have “possible Page 50
MEDIA LITERACY TUTORIAL PROTOTYPE effects on students’ cognitive load” (Mann, 2006). The website includes quizzes, video, images with content revealing itself when hovered over by the mouse, and a lot more interactive content. The PSI really uses interactivity of content to teach learners about the topic, without the need to do a lot of reading. Each concept is explained through a video, and then there are interactive exercises that can be completed to practice for understanding. The prototype contains a wash back feature, but does not contain a wash ahead option, which can be annoying when the learner wants to review a concept learned previously, and then comes back to the current concept. In this case, the learner needs to skip all of the exercises and click on many next buttons in order to try to come back to the page that he was previously on, before using the wash back feature. The prototype was designed using azure websites from Microsoft. All of the pages and content of the website are responsive, which makes it an ideal PSI for all electronic devices and computers. Because the website can be accessed from anywhere and the content can be displayed on any device, learners have the opportunity of learning in any environment in which they wish. The rationale also mentions that the website is designed to be used by high school teachers working in a public school, more particularly at F.W. Johnson Collegiate, a school located in Saskatchewan. However, the prototype could easily be used by any teacher who needs to teach media studies and media literacies. Content and Learner The prototype does a very good job of following the SSF model, and uses an animated pedagogical agent with text-to-speech voice to guide the learner on every page. Unlike other prototypes that I analyzed in this course, the web application also includes atmosphere sound, locale sound and POV sound. However, there are no temporal sound cues programmed, other than the agent talking. The agent was programmed to start automatically on each page, which is a strong feature of the website as it attracts the attention of the learner and forces him or her to listen to the content being described. Page 51
MEDIA LITERACY TUTORIAL PROTOTYPE While the avatar is talking, there are pictures and text appearing next to it, which reinforce the content explained by the avatar. The content does not seem to be geared toward a specific limited audience, but is rather built to help all teachers who need help including media literacy in their curriculum. We were asked to design a prototype for a very specific audience. However, this prototype would be very useful for teachers who do not have mastery of integrating multimedia literacy and multimedia content in their subject. The prototype allows the learner to practice the acquired knowledge through three different modules. The website makes good use of blog, discussion board, as well as the possibility to contact the teacher by email or through the blog option, to allow collaboration within the prototype. There are also plenty of opportunities for the learner to rehearse and use information and skills learned in the prototype. Because of the quality of the website in integrating learning with practice and the professional look of the website, learners would be motivated in going back to the website to further their learning. The fact that the website remembers our account details and I.P. address and can save our work, so that we can continue learning at the same place where we have left off, makes this prototype a great tool for learning. Improvement and Recommendations Even though the website is really well designed and looks very professional, there are still a few things that could be improved to make it perfect. Here are the suggestions for improvement: - Add sounds when a button is clicked. - Add temporal sound cues to warn when good answers or wrong answers are given in the tests - Do not lock the website and allow free flow to each module. - Either include a wash ahead button on each page or allow learners to visit any content quickly by unlocking the menu -In the About menu, insert a button to directly access the rationale, as it is difficult to find the position paper hidden in the Answer Key for Prototype Assessors menu. Page 52
MEDIA LITERACY TUTORIAL PROTOTYPE Because of the many strengths of the prototype, a mark of 11.5 out of 15 was assigned by using a checklist of 20 different elements to evaluate, with higher values given on elements with higher importance. PEER #2 I appreciate the layout, content, the chosen modes of content delivery, and the fact that the site can be interacted with at either home or work site. The one thing I would suggest as an addition/change to the C. Antonini’s prototype/tutorial for media literacy would be to have the ‘agent’, Elliot, remind the audience (teachers of media literacy) of ways to use each concept with students. It is evident that much time and consideration went into the creation of CA’s prototype. He touches on all aspects as required by the 6615 course outcomes, thus as a professional having worked through his prototype, I can honestly and whole-heartedly assign a grade of 14/15. The material and the ways in which it was covered were extremely user-friendly, engaging and thought provoking. Exceptional work! Extremely well done!! 14/15. DR. BRUCE MANN (INSTRUCTOR, MEMORIAL UNIVERSITY) Cory Antonini 11.25/15. “Media Studies Key Concepts: Professional Development for Secondary Teachers”. Looks sufficient though the scope may be unwieldy for a PSI for this course. http://medialiteracy.azurewebsites.net/ An interesting mix of media to present a curricular topic. Choice of avatar software and integration onto a dedicated website is very fine indeed. There are however, some educational oversights. The interface is busy for below average user as presented, with much discreet motor activity required (clicking the mouse). Objectives are stated, but in tiny print. The first “activity” is actually a pretest which could be a good idea for personalizing the PSI. At present however full disclosure to the student is missing about the purpose for collecting data at this early stage of the PSI. Animated character has a great delivery but talks too much. Make the character (agent, avatar) cue the users (teachers) to read the details with a hint, or reminder. Let the users (teachers) do the talking. Good peer comments above. Bruce Mann. Page 53
MEDIA LITERACY TUTORIAL PROTOTYPE Peer and Instructor Suggestions for improvement Factors to Evaluate Peer 1 Peer 2 Instructor Delivery -Add a wash ahead features on each -the agent should remind or instruct -increase the font size of the learning objectives page learners how to use the key -the interface is too busy for the average learner (too much - Include ability to complete the tutorial concepts with their students. clicking an navigating in non-linear way -Agent talks too much. Give the agent more of a cuing purpose. -Add button click sounds -more opportunity for teachers to provide the ‘talking’ - Add easier access to the answer key -Activity #1 should personalize the tutorial by explaining the purpose. Environment -wash ahead and nonlinear design none -the scope makes it too large for the teacher PD environment would give teachers (who are busy) opportunity complete this type of PD in under typical school timing demands. Content -Add temporal sounds for quiz -add examples of classroom -move some content to text and less content delivery from the right/wrong responses (like the activity) application of the key concepts agent -the tutorial is too large in scope for a PSI Learner -PSI is more general in audience rather none -the UI does not take into account the below average learner than to the specific school (too busy) -the scope is too large for the type of learner for which a PSI would be designed Page 54
MEDIA LITERACY TUTORIAL PROTOTYPE Analysis of Evaluations The PSI evaluation data and analysis that follow are part of the formative evaluation processes of a Media Literacy Online Tutorial prototype. The formative evaluation data was collected in a two part process: 1) a quality review stage and 2) a pilot test (Mann, 2006). The purpose of this paper is to describe and analyze the formative data for the purpose of planning tutorial revisions to address identified issues. In other words, the purpose here is to ensure that the tutorial “will eventually foster better performance or enhanced schema” for the targeted learners (Mann, 2006b, p.235). Formative Evaluation Process After the initial design, the online tutorial was evaluated by a variety of prototype evaluators: a subject matter expert (SME), a learning specialist (LS), peers and instructor (PA) and a pilot learner (PL). The SME, LS and PA evaluators were provided with evaluation tools, while the peer and instructor evaluated the tutorial based on the D.E.C.L. (Mann, 2005) and S.S.F. models (Mann, 2006b). The evaluation process also included a direct observation of the PL as they completed the online tutorial. Data Analysis Method The peer and instructor assessments were analyzed under the D.E.C.L. (Mann, 2005) and S.S.F. models (Mann, 2006b). The peer and assessment results were compared with the suggestions and feedback from the SME, LS and PL evaluations. The comparisons and analysis revealed the following suggestions and areas of concerns. Delivery: The SME pointed out a few user interface UI problems (lack of direction to click on slider, how to edit images or when to advance to the next section). These navigational issues were noted as well in the PL, LS and peer evaluators. The instructor noted, too, that the UI is too busy and requires too much navigation (clicking of buttons) for the below average learner, and that objectives font size is too small. In regards to navigation, Peer #1 (P1) and the LS felt the tutorial should allow nonlinear completion. P1 suggested a wash ahead option in each section. The instructor noted that the opening activity could be an opportunity to personalize the tutorial if the purpose was better explained. Page 55
MEDIA LITERACY TUTORIAL PROTOTYPE The instructor felt the scope of the tutorial was too large. This feeling was consistent with the PL and LS’s comments that too much time is required for completion as part of a teacher’s busy schedule. Some of the activities were also long, and the SME suggested chunking the deep viewing activity for better sequencing. The instructor also felt the agent speaks too much—this concern is examined further in Content and SSF sections below. The PL felt the social activities were not effective because the collaboration and chat functions were relatively empty. This concern was also noted by the LS, who explained that asynchronous chat has proven ineffective in past Regina Board of Education (RBE) professional development opportunities. The LS indicated that the Saskatchewan Communication Media outcomes should be present alongside the learning objectives; he also suggested that each section include a classroom example that illustrates how the key concept links to both Comm. Media and the subject teacher’s curriculum, a suggestion made by Peer #2 (P2) as well. Environment: The PL felt the tutorial was too long for completion in the typical school day environment, and that it would be better suited to an afternoon professional development day. This is consistent with the instructor, PL and LS’s assessments. The LS also felt that connections to the Saskatchewan curriculum need to be explicit for learner ‘buy-in’, which affirms the P2’s comment that teachers will require more direction regarding how to apply the key concepts with their students; it also explains the PL’s feeling that the tutorial only ‘somewhat’ modeled classroom use of media literacy. Content: The SME pointed out a couple key areas in which the accuracy of the content could be improved, specifically that some terminology (i.e. “deconstruct”) and some concepts (i.e. “broadcast”) are used in ways that are not consistent with current media literacy discourse. SME suggested the inclusion of more modern digital texts in the examples (social media, blogs, micro casting, YouTube). The SME also felt a couple of the activities did not connect sufficiently illustrate the intent of two key concepts, namely Audiences Negotiate meaning and Media and Social and Political implications. The SME also found that some the images were not particularly effective for illustrating key concept #1. The LS indicated that most of the activities are lower and middle order thinking; his suggestion is to add a couple few higher order tasks, such as proposing solutions, which would offer a wider range of challenge. While both the LS and the PL felt the Page 56
MEDIA LITERACY TUTORIAL PROTOTYPE tutorial’s difficulty was appropriate for teacher professional development, they both felt the language was a bit advanced. As well, the PL felt key concept #5 was not fully developed and should have included more examples. Finally, as noted earlier, the PL felt there was too much content for one sitting. Also noted above, the instructor commented that the agent delivers too much of the content. He suggested the agent should cue the learner toward more reading and interaction with the content. The peers did not make specific recommendation regarding the content, other than P2’s suggestion to add classroom examples of key concepts application. SME, LS and PL noted a number of typos. Learners: PL felt some of the vocabulary and concepts may be difficult for her colleagues—a concern echoed by LS. The LS also noted that teacher learners may want fewer tasks but more examples, and that in doing so, one could reduce the time required to complete the tutorial. He also explained that historically RBE teachers have not really engaged with summary activities, and that these types of tasks may be detrimental to engagement. The instructor indicated that the scope of the tutorial and its UI were not particularly suited to the purpose of building a PSI for this target audience. SSF: SME felt the concept #5 agent’s instructions should be split up for better chunking and cueing. The PL reported the agent voice being out of sync with the animation was sometimes distracting. The instructor suggested that the agent speaks too much and should be given more of a cueing function. Also, P1 felt more temporal sounds should be added to button clicks and to the quizzes as a method of directing attention to the feedback. A personal observation is that some of the text in the videos split the attention of the viewer. Discussion Learning experience: There was a general feeling of the tutorial being engaging, but that activities and the large scope of the tutorial creates an increased demand on the learners’ time. While evaluators felt the objectives are mostly met by the activities, SME evaluator noted some of the activities and presentation elements require better connections to the content, and concept #5 needs more instruction and better examples. Page 57
MEDIA LITERACY TUTORIAL PROTOTYPE LS and P2 indicated a need to have clear practical examples of the key concepts may be applied to classroom activities. The feedback on assessments was well received (PL indicated she read all the feedback even when she answered correctly), so future iterations should expand feedback use. The navigation is clear in regards to sequencing, but the UI is busy and/or unclear in some places, for example at points when sections and activities end. As well, a nonlinear completion (wash ahead) option was a strongly suggested. In regards to sound, more temporal convergent cues (including reduction of ‘talk time’ for the agent) would help in focusing attention and in navigation, which could also help clean up the UI. Achievement: The PL consistently met or exceeded the achievement goals in each of the tutorial activities. Achievement goals were 80% or above on quizzes and level 3 ‘meeting’ on written expressions (level 3 is defined as being able to demonstrate understanding of concepts independently, showing evidence of an application or analysis—middle level thinking). Upon completion of the tutorial, the PL felt better prepared to teach media literacy, but she reported that the tutorial only “somewhat” modeled methods to teach media literacy, a result that was predicted by the LS and reflected in P2’s comments that there are not enough examples of classroom application of key concepts. Summary of Assessments At present, the tutorial is too large in scope and improvements to presentation are required, both in regards to UI layout and to temporal sound functions. This includes shifting the agent’s purpose from delivering content to more cueing learners toward content. As well, some content (terms, conceptual descriptions, images, and activities) needs to be updated to reflect current media literacy discourse. While increasing feelings of self-efficacy in regards to media literacy is a main goal of the tutorial, there must be a strategic instructional change in order for learners to make clearer connections to Saskatchewan curriculum and how the key concepts can be applied in classrooms. Page 58
MEDIA LITERACY TUTORIAL PROTOTYPE Proposed Revisions The revisions below are organized by factors and variables of the D.E.C.L. model (Mann, 2005). Delivery Presentation changes: • UI needs navigation improvement and simplification for advancing from activity to activity, section to section. Therefore, at the end of each section or activity, a pop-up will appear to indicate completion and offer options for proceeding. This “end of section” event (pop-up indication of completion) will be accompanied by a unique temporal sound with convergent function for focusing learners toward the options for moving on (‘next’, ‘classroom connections, or ‘learn more’). • Objectives will be in larger font with a “Skip” option to wash ahead to next section. • The temporal right/wrong sounds from the true and false section will be added to the multiple choice quizzes. • The agent scripts will be reduced; instead the agent will provide more cueing to key concept materials for the leaners’ perusal. • Saskatchewan Communication Media Outcomes will be added to each section in “classroom connections” event (explained below). Strategy • Each section will include a “classroom connections” button, which will reveal at least 5 hypothetical classroom applications (varying in subject matter) with corresponding Sask. Comm. Media curricular outcomes. • The opening self-assessment activity will include name submission and a better explanation of the goal of increasing media literacy and learning how to apply media literacy in classrooms. This will also be revisited at the post self-assessment. Page 59
MEDIA LITERACY TUTORIAL PROTOTYPE Scope: • The tutorial will be reduced to the first 3 key concepts, focusing generally toward more classroom application of key concepts. Environment The estimated time required for the tutorial, and for each section, will be added to the left sidebar menu to accommodate for the “in school day” setting of teacher self- directed learning. Classroom applications information will come from examples currently in practice in our division. Content • Key terms will be hyper-linked with ‘tool tip’ pop-up definitions that are activated by hovering over the word. As well, a key terms/glossary button will be added to the top static menu, which will activate a pop-up glossary. • The agent video and scripts for key concepts 2 and 3 will be adjusted to address accuracy issues identified by the SME. Specifically, the examples and activities will reflect a wider mix of modern media texts, such as Twitter, Facebook, Tumblr, Instagram, etc. As well, terms such as “deconstruct” and “good-life” will be removed and replaced with current media literacy discourse (‘analyze’ and ‘mainstream’). • Concepts 4 and 5 will be removed but may be developed later for the “learn more” content. • The asynchronous chat task will be removed and replaced with a ‘click to reveal’ sample readings/interpretations that are followed by higher order questions. • The collaborative board tasks will be rewritten to provide clearer instructions. Learners Many of the planned revisions above will result in a tutorial that is better suited to the targeted learners. Generally, the reduction and change in scope (fewer concepts with more classroom application) will increase teacher “buy-in”, as suggested by the LS. As well, more authentic classroom application examples will better serve the goal of learners Page 60
MEDIA LITERACY TUTORIAL PROTOTYPE transferring their media literacy to the classroom, ultimately increasing media literacy instruction competencies. Finally, cleaning up the UI and adding more temporal convergent cuing will better match the tutorial to the target learners’ computer capacity and competencies. Page 61
MEDIA LITERACY TUTORIAL PROTOTYPE References Anderson, L. W., & Krathwohl, D. R. (2001). A taxonomy for learning, teaching, and assessing: A revision of Bloom's taxonomy of educational objectives. New York: Longman. Excerpt from the world wide web: http://www.celt.iastate.edu/pdfs- docs/teaching/RevisedBloomsHandout.pdf Beyerbach, B., Walsh, C., & Vannatta, R. (2001). From teaching technology to using technology to enhance student learning: Preservice teachers' changing perceptions of technology infusion. Journal of Technology and Teacher Education, 9(1), 105-27. Chase, J. A., & Houmanfar, R. (2009). The differential effects of elaborate feedback and basic feedback on student performance in a modified, personalized system of instruction course. Journal of Behavioral Education, 18(3), 245-265 Ertmer, P. A., Gopalakrishnan, S., & Ross, E. M. (2001). Technology-using teachers: Comparing perceptions of exemplary technology use to best practice. Journal of Research on Technology in Education, 33(5), 1-26. Gagné, R.M., Briggs, L. & Wager, W. (1992). Principles of Instructional Design (4th Ed.). Fort Worth, TX: HBJ College Publishers Krathwohl, D. R. (2002). Revising Bloom’s Taxonomy. Theory Into Practice, 41(4), 212-264, from the World Wide Web: http://www.celt.iastate.edu/teaching-resources/effective- practice/revised-blooms-taxonomy/ Livingstone, S. (2004) Media Literacy and the Challenge of New Information and Communication Technologies, The Communication Review, 7(1), 3-14, DOI: 10.1080/10714420490280152 Mann, B.L. (2005). Making your own educational materials part I. International Journal of Instructional Technology & Distance Learning 10(2), from the World Wide Web: http://itdl.org/Journal/Dec_05/article02.htm Mann, B.L. (2006a). Conducting formative evaluations of online instructional material. In Bruce L. Mann (Ed.). Selected styles in web-based educational research (pp. 232-242). Hershey, PA: Information Science Publishing. Mann, B.L. (2006b). Making your own educational materials, part II: Multimedia design for learning. International Journal of Instructional Technology & Distance Learning 3(10). Page 62
MEDIA LITERACY TUTORIAL PROTOTYPE Mann (2009). Computer-aided instruction. In Benjamin W. Wah (Ed.). Wiley Encyclopedia of Computer Science & Engineering. Volume 1 (pp. 583-592). Hoboken, NJ: John Wiley & Sons, Inc. Ontario Ministry of Education (1989). Media Literacy Resource Guide. Queen’s Printer for Ontario. Phelps, R., & Maddison, C. (2008). ICT in the secondary visual arts classroom: A study of teachers' values, attitudes and beliefs. Australasian Journal of Educational Technology, 24(1), 1-14. Puentedura, R. R. (2006). Transformation, Technology, and Education. Retrieved February 5, 2014, from the World Wide Web: http://www.hippasus.com/rrpweblog/archives/2012/08/23/SAMR_BackgroundExemplars .pdf Reeves, T. C. & Jonassen, D. H. (1996). Learning with technology: Using computers as cognitive tools. In Jonassen, D. H. (Ed.), Handbook of Research for Educational Communications and Technology, 693-719. New York, NY: Simon & Schuster Macmillan. Saskatchewan Ministry of Education (2011). English language arts 10. Retrieved from the world wide web: http://www.curriculum.gov.sk.ca/ Saskatchewan Ministry of Education (2010). Communications media 10. Retrieved from the world wide web: http://www.curriculum.gov.sk.ca/ Page 63
MEDIA LITERACY TUTORIAL PROTOTYPE CONTACT INFORMATION Cory Antonini [email protected] Tel 306 216 8566 Page 64
Search
