ED565799

Chapter 24 Ministry of Education, Culture & Sports. (2013). Ley Orgánica 8/2013, de 9 de diciembre, para la mejora de la calidad educative (p. 97858-97921). Madrid, Spain: Boletín Oficial del Estado. Retrieved from https://www.boe.es/boe/dias/2013/12/10/pdfs/ BOE-A-2013-12886.pdf Pimsleur, P. (1975). Criterion vs. norm-referenced testing. Language Association Bulletin, 27(1), 21-24. Schaffhauser, D. (2012). You may now open your test tablets. T.H.E.Journal, 39(7), 16-23. Shohamy, E. (1992). Beyond performance testing: a diagnostic feedback testing model for assessing foreign language learning. The Modern Language Journal, 76(4), 513-521. Retrieved from http://dx.doi.org/10.1111/j.1540-4781.1992.tb05402.x Siozos, P., Palaigeorgiou, G., Triantafyllakos, G., & Despotakis, T. (2009). Computer based testing using \"digital ink\": Participatory design of a tablet PC based assessment application for secondary education. Computers & Education, 52(4), 811-819. Retrieved from http:// dx.doi.org/10.1016/j.compedu.2008.12.006 Spolsky, B. (1994). Conditions for second language learning in Israel. English Teacher’s Journal, 47(May), 45-54. Sullivan, R. M. (2013). The tablet inscribed: inclusive writing instruction with the iPad. College Teaching, 61(1), 1-2. Retrieved from http://dx.doi.org/10.1080/87567555.2012. 700339 Van Oostveen, R., Muirhead, W., & Goodman, W. M. (2011). Tablet PCs and reconceptualizing learning with technology: a case study in higher education. Interactive Technology and Smart Education, 8(2), 78-93. Retrieved from http://dx.doi.org/10.1108/17415651111141803 Wall, D. (2000). The impact of high-stakes testing on teaching and learning: can this be predicted or controlled? System, 28(4), 499-509. Retrieved from http://dx.doi.org/10.1016/ S0346-251X(00)00035-X Wall, D. (2005). The impact of high-stakes examinations on classroom teaching: a case study using insights from testing and innovation theory. Cambridge: Cambridge University Press. Waters, J. K. (2010). Enter the iPad (or not?). T.H.E. Journal, 37(6), 38-40. Weir. C. J. (2005). Language testing and validation: an evidence-based approach. New York, NY: MacMillan. Whittaker, R. (2006). Review of Estudios y Criterios para una Selectividad de calidad en el examen de inglés. Estudios Ingleses de la Universidad Complutense, 14, 198-206. 292

25The implications of business English mock exams on language progress at higher education Rocío González Romero1 Abstract Language learning has been increasingly influenced by technology over the last decades thanks to its positive effects on language acquisition. It is thanks to the technology’s supportive role towards language learning that an increasing number of online foreign language courses have appeared. Besides, foreign language courses are more and more specialised covering a wide range of topics, from nursing to agricultural studies. However, this study is exclusively concerned with a well-known Language for Specific Purposes (LSP) subject: Business English. The objective of this research is twofold: on the one hand, to describe the implications of mock exams on foreign language learning; on the other hand, it aims at contributing to the field of computerised language testing by properly analysing the effects of these kinds of exams on learners’ foreign language progress. Previous studies have focused on the development of specific language skills (Dunkel, 1991; Larson, 2000), or have reported the improvement of computer adaptive testing on official language exams (Alderson, 2000), or have simply described the advantages and disadvantages of computer- based tests (Alderson, 2000; Brown, 1997; Dunkel, 1999). However, few studies have considered the role of mock exams as scaffolding activities for language learning. The present study involves adult participants at the higher education context undertaking online Business English as a compulsory subject of their degree on Economics. The paper discusses the importance of scaffolding activities such as mock exams and self- 1. Centro de Educación Infantil y Primaria La Moraña, Arévalo, Ávila, Spain; [email protected] How to cite this chapter: González Romero, R. (2016). The implications of business English mock exams on language progress at higher education. In A. Pareja-Lora, C. Calle-Martínez, & P. Rodríguez-Arancón (Eds), New perspectives on teaching and working with languages in the digital era (pp. 293-302). Dublin: Research-publishing.net. http://dx.doi. org/10.14705/rpnet.2016.tislid2014.442 © 2016 Rocío González Romero (CC BY-NC-ND 4.0) 293

Chapter 25 assessment activities in order to ensure learners’ language progress and makes reference to supporting articles in the field at the same time that it presents some materials illustrating these developments. Keywords: information and communication technologies, ICT, language for specific purposes, LSP, online learning, assessment, higher education. 1. Introduction Integrating the use of technology into education requires the adaptation of good teaching materials into digital format in the simplest and most cost-effective way. This is the case with assessment since it is an essential part of any course because it checks students’ understanding of the course’s content. Tests and examinations are widely used as assessment tools for being objective indicators of a student’s performance. In the current teaching context, the increasing use of e-learning platforms has triggered the need of automatic tests to check students’ progress. Given the growing variety of Information and Communication Technology (ICT) tools involved in education, it is becoming a challenge to monitor students’ progress. That is why this article attempts to justify the positive effects of mock exams on language learning and gives evidence of this practice as scaffolding activities. 2. The relationship between online learning and LSP Online learning has become an established way of teaching and learning over the last decade. The particularity of this type of education is the use of technology by both the tutor and the learners with the objective of designing digital learning content, offering interaction among participants, and fostering the process of learning. Until recently, online learning was just an excellent way of enlarging the target audience of a course; however, it now requires 294

Rocío González Romero getting closer to each student by offering different interactive possibilities, such as counselling or individual feedback. Turning to the issue of LSP, the same circumstances have arisen regarding the new virtual challenges that tutors face. Not only do the tutors need to master the course’s content, but they also have to be computer literate. According to Arnó-Macià (2011), “in the context of […] LSP, technology also becomes a gateway to specialised discipline knowledge and to students’ relevant discourse communities” (p. 24). Hence, the tutor must collect the latest language resources within a specialised field. 2.1. The potential benefits of online learning Online learning contributes to LSP teaching and learning by providing a great variety of resources to improve grammar and the four main language skills: reading, listening, writing and speaking. In addition to these skills, there are several ways in which online learning is beneficial: it caters for students’ specific needs; it provides access to the digital resources at any time; it fosters students’ autonomy; it develops awareness and learning strategies; and it is based on students’ own responsibility. All in all, online education requires the use of technology to personalise a course’s content so that students get the most out of it, given their particular characteristics. 2.2. Online learning and assessment When designing an online course, one needs to consider whether the course would mainly be exam-based or assignment-based. This means asking yourself which are the most important elements within the course. If it is the exam, the course will likely prioritise the acquisition of knowledge. However, if the course is assignment-based, students will be probably expected to put knowledge into practice over the length of the course. This is the view of James and Fleming (2005), who think that students tend to perform in a better way at coursework 295

Chapter 25 assignments rather than at examinations. What lies beneath is the importance of providing enough tools and opportunities for students to check their knowledge through the course. In fact, most of the courses currently created tend to include both types of assessment. 2.3. The importance of feedback On numerous occasions, students within an online learning course require feedback in order to properly obtain the course’s objectives. This feedback could come from different areas, like answering doubts about grammar, providing the accurate answers for exercises, correcting a written composition, or specifying the course’s assessment criteria for beginners, among others. Some authors, like Hattie (1987), argue that feedback is the main interactive component in many forms of online learning. For example, Gibbs and Simpson (2004) claim that “[students] can cope without much, or even any, face-to-face teaching, but they cannot cope without regular feedback on assignments. […] Regular assignments and comprehensive feedback is understood to be central to distance education” (p. 9). In this case, online tutors should provide comprehensive feedback on regular assignments as frequently as possible. 2.4. Automatic assessment tools Marking may become a tedious task if tutors find they have a large amount of work to correct. Within online education, automatic assessment tools are increasing in order to save the tutors’ precious time and give students immediate feedback. Among these automatic assessment tools, this article will briefly describe the mock exams and the self-assessment tests. Regarding mock exams, one can consider them as a kind of examination under similar conditions as the final exam, but usually shorter. The objective of mock exams is to provide feedback to the student without having to wait until very late in the course, or even after the final exam. That is the reason why they can be used as scaffolding activities supporting students’ learning. Mock exams have indeed many advantages, they: 296

Rocío González Romero • support learning by checking what students have learnt; • give learners the opportunity to practice and consolidate what they think they know; • provide instant knowledge of results and feedback; • help students to monitor their own progress; • develop learners’ self-evaluation skills; • guide the choice of further learning resources to increase mastery; • give learners a sense of accomplishment; • provide tutors with the necessary materials to monitor and evaluate students’ progress. Concerning the self-assessment tests, it is highly recommended to use them when the tutor wants to know students’ perceptions and beliefs. The need for self- assessment fosters the reflection on one’s own learning, which not only helps to know the areas one needs to study more, but also facilitates the development of critical thinking. It is a common belief among tutors that a percentage of the final mark has to be devoted to assessment activities, such as mock exams and self-assessment tests, in order to encourage students to do them. Evidence on the field, like the research performed by Forbes and Spence (1991), show that without the input of motivation given by marks students did not solve practically any activity, and as a consequence, their final grade was noticeably low compared to students who were used to assessing their peers’ work. Gibbs and Simpson (2004) state that “coursework does not have to be marked to generate the necessary learning. […] The trick when designing assessment regimes is to generate engagement with learning tasks without generating piles of marking” (p. 8). Thus, the use of 297

Chapter 25 automatic mock exams and self-assessment tests helps students improve their learning while the teacher saves marking time. 3. Methodology and results 3.1. Setting and participants The present research was carried out to study the relationship between mock exams and the students’ final exam achievement. To do so, the work reported here has been carried out at the Universidad Católica Santa Teresa de Jesús de Ávila, Spain, where participants were adult learners enrolled in an online official degree on Economics. As part of their compulsory curriculum, students took Business English. This subject’s level is pre-intermediate to intermediate and it is worth six credits, which is equivalent to a hundred and fifty hours. Participants who were involved in the research were twenty in total and were all Spanish. They had a basic knowledge of English but had never studied any LSP class before. This is why all of them were considered novel learners in this respect. 3.2. Procedure Before beginning the study, the mock exams were designed according to the course’s content. Each mock exam included five multiple-choice questions learnt in the lesson: two of the items requested specific business vocabulary, another one checked a linguistic expression, and the last two items were about grammatical features. There were a total of twenty-four mock exams, which corresponded to the total number of lessons. After having created the mock exams, they were uploaded into the course’s virtual platform. Students did not have a fixed deadline to take the mock exams, nor did they have a time limit to complete them. On the very first day of the course, students were highly recommended to take each mock exam after studying the lesson, but they were not obliged to do so. Hence, two groups were accordingly created: the experimental and the control ones. Once data was collected, it was analysed by calculating the average marks of the students who took the mock exams 298

Rocío González Romero and the ones who did not, and later on, they were compared to the final exam’s grade of all students. The last step consisted in creating a graph to show the results. 3.3. Results and discussion Comparing participants’ answers of both students who took the mock exams and the ones who did not, it was realised that the fact of taking the mock exams had significant correlation with students’ academic achievement. Those who took the mock exams had higher final grades than those who decided not to take them, as can be seen in Figure 1. Figure 1. Influence of mock exams on final exam’s grade The preceding graph shows an interesting finding too. Considering the final exam to be worth from one to ten points, where five is the passing grade, the difference between students who did not take the mock exams and the ones who did so is almost one point. Since this difference is notably large, it seems that taking mock exams as scaffolding activities improves learning. An appealing insight into the findings of this research is the fact that many participants took all the mock exams, even though they were not marked (as 299

Chapter 25 mentioned above). The number of these participants is quite high, especially with respect to the low number of participants that either did not take any mock exam or did not complete all the mock exams. There are several possible reasons that could explain this situation. One could be that students may have been highly motivated to acquire as much knowledge as they could in this subject, given that they were all novel LSP learners; hence, they took all the mock exams because they might want to check their understanding before the final exam. Another feasible explanation could be that those students who completed all the mock exams considered it important (1) to check their progress, so that they knew in advance those aspects that they needed to study harder for the final exam, or (2) to build up the content they thought they knew. Yet, there might be students who did not entirely understand (or were not fully confident) that they would not receive any mark for taking the mock exams; and, for this reason, they completed all the mock exams. Unfortunately, it is not possible to know the reason(s) that moved students to complete all the mock exams since no survey was conducted to check students’ inner motivations. This issue could become an interesting starting point for a future research. All in all, whether it is for one reason or another, what matters is that those participants who completed all the mock exams greatly surpassed the ones who did not take them, proving that this assessment tool can support students’ learning and progress. Another consideration that can be done in light of the previously stated results is that mock exams were the only learning resource tested in this research. This means that the present study is limited by the number of different learning strategies applied. However, mock exams can be combined with many other resources, such as self-assessment tests, individual or collaborative projects, or even educational activities or games like quizzes, mazes, crosswords, guessing the object, and so on. To sum up, the information presented before provides some support for the view that not only do students who take mock exams achieve a brilliant final grade in a given subject, but they also outperform those students who decided 300

Rocío González Romero not to take the mock exams. Moreover, these results might endorse Gibbs and Simpson’s (2004) opinion about not assessing coursework, but rather engaging students in doing the course tasks since it reports large benefits for their learning process even though the reasons for their motivation could be imprecise. Indeed, a promising line of study regarding the field of online LSP learning and language progress could be taking into consideration sociolinguistic variables like age, gender, language level, and so on, affecting the performance of mock exams. Another attainable research could focus on the importance of motivation for language learning, but this seems to be a well-established area of research (Dörnyei, 1998; Fernández Orío, 2013; Pourhosein Gilakjani, Leong, & Banou Sabouri, 2012; Ushida, 2005). In addition, further investigations could feasibly tackle the issue on other LSP disciplines. 4. Conclusions Online learning is now an extended everyday practice in which many disciplines are taught, including the one this article is about: Business English. In online higher education, feedback is what makes a difference on students’ achievement; however, it is difficult to provide detailed feedback when the tutor has countless students. It is thanks to technology that automatic assessment tools, such as mock exams, can be used to give immediate and helpful feedback to students. This article has outlined and verified the benefits of mock exams as scaffolding activities to foster language learning. Results indicate that these types of activities promote outstanding final grades as well as prove to be an effective way of engaging students in learning tasks. References Alderson, J. C. (2000). Technology in testing: the present and the future. System, 28(4), 593- 603. Retrieved from http://dx.doi.org/10.1016/S0346-251X(00)00040-3 301

Chapter 25 Arnó-Macià, E. (2011). Approaches to information technology from an LSP perspective: challenges and opportunities in the new European context. In N. Talaván Zanón, E. Martín Monje & F. Palazón Romero (Eds.), Technological innovation in the teaching and processing of LSPs: Proceedings of TISLID’10 (pp. 23-40). Madrid: Universidad Nacional de Educación a Distancia. Brown, J. D. (1997). Computers in language testing: present research and some future directions. Language Learning and Technology, 1(1), 44-59. Retrieved from http://llt. msu.edu/vol1num1/brown/default.html Dörnyei, Z. (1998). Motivation in second and foreign language learning. Language Teaching, 31(3), 117-135. Retrieved from http://dx.doi.org/10.1017/S026144480001315X Dunkel, P. (1991). Computerized testing of nonparticipatory L2 Listening comprehension proficiency: an ESL prototype development effort. Modern Language Journal, 75(1), 64- 73. Retrieved from http://dx.doi.org/10.1111/j.1540-4781.1991.tb01084.x Dunkel, P. (1999). Considerations in developing or using second/foreign language proficiency computer-adaptive tests. Language Learning and Technology, 2(2), 77-93. Fernández Orío, S. (2013). Motivation and second language acquisition. Retrieved from Universidad de La Rioja, Servicio de Publicaciones. Retrieved from http://biblioteca. unirioja.es/tfe_e/TFE000342.pdf Forbes, D., & Spence, J. (1991). An experiment in assessment for a large class. In R. Smith (Ed.), Innovations in Engineering Education. London: Ellis Horwood. Gibbs, G., & Simpson, C. (2004). Conditions under which assessment supports students’ learning. Learning and Teaching in Higher Education, 1, 3-31. Hattie, J. A. (1987). Identifying the salient facets of a model of student learning: a synthesis of meta-analyses. International Journal of Educational Research, 11, 187-212. James, D., & Fleming, S. (2005). Agreement in student performance in assessment. Learning and Teaching in Higher Education, 1, 32-50. Larson, J. W. (2000). Testing oral language skills via the Computer. CALICO Journal, 18(1), 53-66. Pourhosein Gilakjani, A., Leong, L. M., & Banou Sabouri, N. (2012). A Study on the role of motivation in foreign language learning and teaching. International Journal of Modern Education and Computer Science, 4(7), 9-16. Retrieved from http://dx.doi.org/10.5815/ ijmecs.2012.07.02 Ushida, E. (2005). The role of students’ attitudes and motivation in second language learning in online language courses. CALICO Journal, 23(1), 49-78. 302

26Assessing pragmatics: DCTs and retrospective verbal reports Vicente Beltrán-Palanques1 Abstract Assessing pragmatic knowledge in the instructed setting is seen as a complex but necessary task, which requires the design of appropriate research methodologies to examine pragmatic performance. This study discusses the use of two different research methodologies, namely those of Discourse Completion Tests/Tasks (DCTs) and verbal reports. Research has shown that the use of DCTs in combination with verbal reports can increase the trustworthiness of the results (Félix-Brasdefer, 2010). Hence, taking into account the potential of verbal reports, the present study aims to investigate the cognitive processes undertaken by a group of English language learners as regards their pragmatic performance. Findings regarding the value of retrospective verbal reports are discussed together with practical recommendations for the use of DCTs and verbal reports to assess speech act performance in the instructed setting. Keywords: pragmatics, speech acts, assessment, DCTs, retrospective verbal reports. 1. Introduction Assessment of Second Language/Foreign Language (SL/FL) pragmatics is a growing area in the field of Interlanguage Pragmatics (ILP). Several researchers have drawn their attention to this particular aspect, especially in recent years (see Beltrán-Palanques, 2013, 2014; Félix-Brasdefer, 2010; Roever, 2010, 2011; Ross 1. Universitat Jaume I, Castellón de la Plana, Spain; [email protected] How to cite this chapter: Beltrán-Palanques, V. (2016). Assessing pragmatics: DCTs and retrospective verbal reports. In A. Pareja-Lora, C. Calle-Martínez, & P. Rodríguez-Arancón (Eds), New perspectives on teaching and working with languages in the digital era (pp. 303-312). Dublin: Research-publishing.net. http://dx.doi.org/10.14705/rpnet.2016.tislid2014.443 © 2016 Vicente Beltrán-Palanques (CC BY-NC-ND 4.0) 303

Chapter 26 & Kasper, 2013; Usó-Juan & Martínez-Flor, 2014). Pragmatics, which is one of the main components of the communicative competence model (Bachman, 1990; Usó-Juan & Martínez-Flor, 2006), should be appropriately introduced in the instructed setting in order to teach and assess this competence successfully. In this paper, I will focus on the issue of assessing pragmatics in the online instructed setting combining two different research methodologies: Discourse Completion Tests/Tasks (DCTs) and retrospective verbal reports. In the first part of the paper, I will provide a theoretical framework which focuses on the use of the two aforementioned research methodologies in the field of ILP. In the second part I will explain how the study was developed and the results derived from it. Finally, I will briefly discuss practical recommendations for the use of DCTs and verbal reports to assess speech act performance in the instructed setting. 2. Theoretical framework Verbal reports have been used in the field of ILP in combination with other research instruments, particularly those of role-plays (Beltrán-Palanques, 2013; Cohen & Olshtain, 1993; Félix-Brasdefer, 2008a, 2008b; Widjaja, 1997; Woodfield, 2012) and DCTs (Beltrán-Palanques, 2013; Robinson, 1992; Woodfield, 2008, 2010). However, for reasons of space, this paper is restricted to focus on the studies in which verbal reports were employed in combination with DCTs (see Beltrán-Palanques, 2013, 2014; Félix-Brasdefer, 2010). One of the pioneering studies which used verbal reports in combination with written DCTs was conducted by Robinson (1992). The author combined concurrent (single-subject think-aloud) and retrospective verbal reports (i.e. interviews). The data obtained by means of verbal reports provided specific information about the planning process of refusal semantic formulae, evaluation of different utterances, pragmatic and linguistic difficulties, and knowledge sources. Woodfield (2008) employed paired concurrent verbal reports and retrospective verbal reports with three pairs of native speakers of English to provide insights concerning issues of validity noticed during the reconstruction of requests in 18 written DCTs. Woodfield (2010) explored the role of paired 304

Vicente Beltrán-Palanques concurrent and retrospective verbal reports to examine the cognitive processes of advanced learners of English as a SL on written DCTs which elicited status- unequal requests. The data obtained from concurrent verbal reports revealed that the social context of the discourse situation affected the pragmalinguistic and sociopragmatic choices and language-related episodes showed participants’ negotiation of lexical and grammatical elements when planning the request strategies. Regarding retrospective reports, the author reported that they offered information about participants’ language of thought and the difficulties that participants experienced with the research methodology employed. More recently, Beltrán-Palanques (2013) conducted a study employing retrospective verbal reports in combination with both open role-plays and interactive written DCTs. In this study, the speech act under investigation was that of apologies. Results revealed that retrospective verbal reports appeared to be instrumental in gathering information regarding participants’ pragmatic production. In short, studies using verbal reports, either concurrently or retrospectively, in combination with DCTs, have shown the positive effects of this particular research methodology. Verbal reports seem to be instrumental in providing information as regards the participants’ cognitive process, perceptions of speech act performance, validation of research instruments, sociocultural and sociolinguistic knowledge, as well as politeness issues. Hence, taking into account the literature review sketched above, the present study aims to contribute to this specific field of research by examining the potential of using interactive written DCTs in combination with retrospective verbal reports in an online instructed setting. 3. The study 3.1. Participants This small explanatory study included 30 adult learners (12 male and 18 female) of an online English as a Foreign Language (EFL) course (mean age: 28.3). Participants were first asked to complete the UCLES Quick Placement Test 305

Chapter 26 (Oxford University Press) to test their proficiency level. Results revealed that their proficiency level was B1, according to the Common European Framework of Reference for Languages (CEFR – http://www.coe.int/t/dg4/linguistic/source/ framework_en.pdf). This particular result was expected, as participants were taking an online B1 course. Nevertheless, it was necessary to administer this placement test to appropriately identify participants’ proficiency level. In addition to this, a background questionnaire was also administered in order to gather information regarding participants’ personal information (e.g. age, gender, mother tongue) and FL learning experience (adapted from Beltrán-Palanques, 2013). Data gathered from the questionnaire revealed that participants were bilingual (i.e. Catalan and Spanish) and had studied English at school, secondary school and university. 3.2. The speech act under investigation The pragmatic aspect under investigation in this study is the speech act of apologies. According to Searle (1979), apologies fall into the category of expressives, since they “express the psychological state specified in the sincerity condition about a state of affairs specified in the propositional content” (p. 15). Apologies are here defined as a “compensatory action to an offense in the doing of which S (the speaker) was causally involved and which is costly to H (the hearer)” (Bergman & Kasper, 1993, p. 82). Then, apologies can be used as remedial exchanges to restore harmony between speakers after a given offense (Goffman, 1971). People often take part in remedial actions in which they attempt to save face (Brown & Levinson, 1987) and restore the social harmony of the speech community (Goffman, 1971). Following Brown and Levinson (1987), an apology is typically viewed as negative politeness whose main goal is that of providing a redressive action. From the domain of politeness, an apology is seen as a communicative event in which the speaker (i.e. the apologiser) should take into account the other interlocutor’s face (i.e. the apologisee) in order to restore the situation (Brown & Levinson, 1987). Hence, apologising, as mentioned by Bataineh and Bataineh (2006), is a face-saving act for the hearer and a face- threatening act for the speaker. 306

Vicente Beltrán-Palanques 3.3. Instruments and procedure The research methodology employed in this study consisted of interactive written DCTs (Beltrán-Palanques, 2013). This consists of eight different situations containing the following variables: social status (i.e. equal and hearer-dominant), social distance (i.e. stranger and acquaintance), and level of offense (high and low). However, in the present study, only four of the eight situations used in the aforementioned study were selected, specifically those involving the following variables: social status (i.e. equal), social distance (i.e. stranger and acquaintance), and level of offense (high and low). Table 1 shows the different situations. Table 1. Situations Variables Context of the situation Social status Sit. 1 Bookshop Social distance Level of offense Sit. 2 University Equal Sit. 3 Student’s flat Equal Stranger Low Sit. 4 University Equal Acquaintance High Equal Acquaintance Low Stranger High Participants were grouped in pairs and they were asked to complete the interactive written DCTs in pairs. Immediately after the completion of each written cognitive task, participants took part in the retrospective verbal report. Tasks were performed using Skype. However, due to the interactive nature of the written DCTs, only one participant of each pair elicited the speech act under investigation (i.e. apologies). Participants could read back their written production before engaging them in the verbal probes. In so doing, participants were exposed to their own production, in order to make them aware of what they produced. Participants were allowed to use Catalan, Spanish and/or English during the verbal probes, since in this case, the major goal was to examine participants’ thoughts while performing the tasks, rather than exploring their spoken competence in English. Furthermore, it was believed that allowing them to use their L1 would facilitate the verbal reporting. Verbal reports were recorded in order to further examine participants’ contributions. The retrospective verbal reports were transcribed following Jefferson’s (2004) transcript notation. 307

Chapter 26 4. Results and discussion This section provides an overview of the results of this study. Table 2 displays the results derived from the retrospective verbal reports. Table 2. Situations Verbal reports Situations Situation 1 Situation 2 Situation 3 Situation 4 6 7 9 Grammar and lexicon 7 4 5 8 5 5 9 Social status 7 12 10 15 Social distance 8 Level of offense 9 Each participant was asked to take part in the verbal report, which focused on aspects related to grammar and lexicon, and pragmatic knowledge. As mentioned above, only one of the participants in each pair took part in the apology production. Therefore, only the data derived from the participants who produced apology sequences are shown here. Regarding grammar and lexicon, retrospective verbal reports revealed that, in general, participants focused on aspects related to grammar and lexicon when planning their pragmatic production, especially in the fourth situation (i.e. nine participants out of 15). This could be related to the fact that this situation in particular could be more demanding for participants since it involved the following context: two equal participants who were strangers and whose level of offense was high. As a matter of fact, most participants indicated that this situation was very offensive for the other interlocutor since it involved damaging students’ class notes. It is worth mentioning that participants were students, so perhaps they perceived this as very offensive. Concerning social status, participants indicated that in situations 1 and 4 this particular variable seemed to have affected the way they addressed their interlocutors. In this case, all the participants shared the same social status, and they revealed that the fact that they had the same status affected their production. 308

Vicente Beltrán-Palanques According to them, having a different social status could involve apologising in a different manner, thus, using more strategies to restore the situation. As regards this specific variable, participants did not provide much information, probably due to the fact that they all shared the same social status. Social distance, according to the results obtained in the verbal reporting, appeared not to have a great impact on participants’ pragmatic production. As a matter of fact, eight and nine participants mentioned this specific variable in the situations 1 and 4, in which social distance was that of stranger. According to the verbal reporting, in the other remaining two situations (i.e. 2 and 3), containing acquaintance social distance, a lower number of participants seemed to have paid attention to this issues, particularly five participants in each one. Finally, retrospective verbal reports revealed that the severity of offence played a paramount role. As a matter of fact, those situations whose level of offense was classified as high seemed to have received the attention of the majority of the participants. Particularly, situations 2 and 4, in which the severity of offense was classified as high, called participants’ attention as they involved situations which violated social norms. The results of this study are in line with previous works in the field, in which verbal reports were also instrumental in revealing participants’ information regarding their speech act performance in DCTs (Beltrán-Palanques, 2013; Robinson, 1992; Woodfield, 2008, 2010). Robinson (1992) indicated that verbal reports were useful to obtain information about attended aspects, and indications of linguistic and pragmatic difficulties, among others. Woodfield (2008) found that verbal reports were instrumental in identifying participants’ attention while working on the tasks. Similarly, Woodfield’s (2010) study revealed that retrospective verbal reports provided information regarding participants’ cognitive processes while on task. Beltrán-Palanques (2013) found that verbal reports were instrumental in providing information regarding participants’ pragmatic knowledge. In this explanatory study, retrospective verbal reports were also useful to obtain information related to participants’ attention to grammar and lexicon features as well as pragmatic knowledge. 309

Chapter 26 5. Conclusions The aim of this explanatory study was to contribute to the growing body of literature that employs verbal reports in combination with other research instruments, particularly DCTs. Results from this study are consistent with previous research in the field, since retrospective verbal reports were instrumental in providing further information concerning participants’ speech act production. Several studies have concluded that verbal reports, in their various forms (i.e. concurrent and retrospective) can be of paramount interest, given that researchers can obtain further information related to learners’ pragmatic performance. This, in turn, can benefit language teachers in their tasks of developing instructional approaches aim to (i) integrate speech acts; and (ii) better understand how learners at different levels process and perform pragmatic utterances in contextualised situations. Empirical studies whose goal is to obtain participants’ thoughts while performing cognitive tasks, such as DCTs, should employ verbal reports, since this tool allows researchers to better understand participants’ pragmatic behaviour. Moreover, the use of DCTs and verbal reports can also be of paramount interest for instruction, as a tool to obtain information about learners’ pragmatic performance, and to improve instructional approaches as well as design them drawing on empirical findings. 6. Acknowledgements The research conducted in this paper is part of the Education and Innovation Research Project “Proyecto de Innovación Educativa Universitat Jaume I 2779/13, Parámetros de aproximación a la evaluación de las destrezas orales en lengua inglesa: tipología, diseño de tests y criterios de validación”. References Bachman, L. F. (1990). Fundamental considerations in language testing. Oxford: Oxford University Press. 310

Vicente Beltrán-Palanques Bataineh, R. F., & Bataineh, R. F. (2006). Apology strategies of Jordanian EFL university students. Journal of Pragmatics, 38(11), 1901-1927. Beltrán-Palanques, V. (2013). Exploring research methods in interlanguage pragmatics. A study based on apologies. Saarbrücken: Lambert Academic Publishing. Beltrán-Palanques, V. (2014). Methodological issues in interlanguage pragmatics: some food for thought. In G. Alcaraz-Mármol & M. M. Jiménez-Cervantes Arnao (Eds.), Studies in philology. Linguistics, literature and cultural studies in modern languages. Cambridge: Cambridge Scholars Publishing. Bergman, M. L., & Kasper, G. (1993). Perception and performance in native and nonnative apology. In G. Kasper & S. Blum-Kulka (Eds.), Interlanguage pragmatics (pp. 82-107). Oxford: Oxford University Press. Brown, P., & Levinson, S. C. (1987). Politeness. Some universals in language usage. Cambridge: Cambridge University Press. Cohen, A. D., & Olshtain, E. (1993). The production of speech acts by EFL learners. TESOL Quarterly, 27(1), 33-56. Retrieved from http://dx.doi.org/10.2307/3586950 Félix-Brasdefer, J. C. (2008a). Perception of refusals to invitations: exploring the minds of foreign language learners. Language Awareness, 17(3), 195-211. Félix-Brasdefer, J. C. (2008b). Politeness in Mexico and the United States: a contrastive study of the realization and perception of refusals. Amsterdam: John Benjamins Publishing Company. Retrieved from http://dx.doi.org/10.1075/pbns.171 Félix-Brasdefer, J. C. (2010). Data collection methods in speech act performance: DCTs, role- plays, and verbal reports. InA. Martínez-Flor & E. Usó-Juan (Eds.), Speech act performance. Theoretical, empirical and methodological issues (pp. 41-56). Amsterdam: John Benjamins Publishing Company. Retrieved from http://dx.doi.org/10.1075/lllt.26.03fel Goffman, E. (1971). Relations in public. Microstudies of the public order. Harmondsworth: Penguin. Jefferson, G. (2004). Glossary of transcript symbols with an introduction. In G. Lerner (Ed.), Conversation analysis: studies from the first generation (pp. 13-31). Amsterdam: John Benjamins Publishing Company. Retrieved from http://dx.doi.org/10.1075/ pbns.125.02jef Robinson, M. A. (1992). Introspective methodology in interlanguage pragmatics research. In G. Kasper (Ed.), Pragmatics of Japanese as a native and target language [Second Language Teaching and Curriculum Center Technical Report No. 3] (pp. 27-82). Honolulu HI: University of Hawai’i Press. 311

Chapter 26 Roever, C. (2010). Researching pragmatics. In B. Paltridge & A. Phakiti (Eds.), Continuum companion to research methods in applied linguistics (pp. 240-255). London: Continuum International Publishing Group. Roever, C. (2011). Testing of second language pragmatics: past and future. Language Testing, 28(4), 463-481. Retrieved from http://dx.doi.org/10.1177/0265532210394633 Ross, S., & Kasper, G. (Eds.). (2013). Assessing second language pragmatics. Basingstoke: Palgrave Macmillan. Searle, J. (1979). Expression and meaning: studies in the theory of speech acts. Cambridge: Cambridge University Press. Retrieved from http://dx.doi.org/10.1017/ CBO9780511609213 Usó-Juan, E., & Martínez-Flor, A. (2006). Approaches to language learning and teaching: towards acquiring communicative competence through the four skills. In E. Usó-Juan & A. Martínez-Flor (Eds.), Current trends in the development and teaching of the four language skills (pp. 3-25). Berlin: Mouton de Gruyter. Retrieved from http://dx.doi. org/10.1515/9783110197778.1.3 Usó-Juan, E., & Martínez-Flor, A. (2014). Reorienting the assessment of the conventional expressions of complaining and apologising: from single-response to interactive DCTs. Iranian Journal of Language Testing, 4(1), 113-136. Widjaja, C. (1997). A study of date refusal: Taiwanese females vs. American females. University of Hawai’i Working Papers in ESL, 15(2), 1-43. Woodfield, H. (2008). Problematising discourse completion task: voices from verbal report. Evaluation and Research Education, 21(1), 43-69. Retrieved from http://dx.doi. org/10.2167/eri413.0 Woodfield, H. (2010). What lies beneath?: Verbal report in interlanguage requests in English. Multilingua, 29(1), 1-27. Retrieved from http://dx.doi.org/10.1515/mult.2010.001 Woodfield, H. (2012). Pragmatic variation in learner perception: the role of retrospective verbal report in L2 speech act research. In J. C. Félix-Brasdefer & D. A. Koike (Eds.), Pragmatic variation in first and second language contexts. Methodological issues (pp. 209-238). Amsterdam: John Benjamins Publishing Company. 312

Section 3. Applying computational linguistics and language resources to language teaching and learning 313

314

27An updated account of the WISELAV project: a visual construction of the English verb system Andrés Palacios Pablos1 Abstract This article presents the state of the art in WISELAV, an on-going research project based on the metaphor Languages Are (like) Visuals (LAV) and its mapping Words-In-Shapes Exchange (WISE). First, the cognitive premises that motivate the proposal are recalled: the power of images, students’ increasingly visual cognitive learning style, and the importance of grammar in L2 learning. Then an updated report follows on WISE’s analysis of the English verb system, an interpretation in terms of a transfer of morphological-functional information, represented through a series of fitting shapes. These are purposely assigned, as certain basic iconicity principles are applied to associate verb grammar meanings and graphic forms. The shapes so described appear with the steps taken to develop a pilot computer programme and to both highlight the visual aspects of the system and eventually enable its use online. Keywords: English verb, applied linguistics, construction grammar, cognitive learning, verbo-graphic metaphor. 1. Universidad de Burgos, Burgos, Spain; [email protected] How to cite this chapter: Palacios Pablos, A. (2016). An updated account of the WISELAV project: a visual construction of the English verb system. In A. Pareja-Lora, C. Calle-Martínez, & P. Rodríguez-Arancón (Eds), New perspectives on teaching and working with languages in the digital era (pp. 315-326). Dublin: Research-publishing.net. http://dx.doi. org/10.14705/rpnet.2016.tislid2014.444 © 2016 Andrés Palacios Pablos (CC BY-NC-ND 4.0) 315

Chapter 27 1. Introductory foundations of WISELAV WISELAV is a research project whose purpose is to study the implications of the proposed metaphor (LAV/WISE), and to develop a set of suitable supports for assessing its potential pedagogic value. As an on-going project, it aims to explore how its proposed visual framework can account for analysing and teaching English verb phrases, rather than becoming a grammar of the English verb on its own or a comparative study of other existing analyses. Because of this and space restrictions, this paper includes hardly any references to the huge number of existing works dealing with verbs and tense in English. For a better understanding of the project and its scope, it seems convenient to first acknowledge the cognitive premises that set it off. The first one is the power of images. Through multiple arrangements, they have always proved to be useful to illustrate and explain the often-complex processes of science; and now, with the new technologies, hypertext and the digital era, images have also taken over outside the academic world in our everyday lives. Thus, this expansion of image usage is bringing about some changes for our cognitive learning styles. This phenomenon, of particularly decisive pedagogic consequences for the younger generation, has not been unnoticed by researchers: Myers (2003), for example, suggested the incorporation and explanation of non-verbal elements within visual texts; Littlemore (2004) verified students’ increasingly visual cognitive learning style; and Jiang and Grabe (2007) explored the linguistic usage of some graphic organisers to represent text structure. Another premise of WISE was to acknowledge that nowadays grammar teaching – although considered an essential component in L2 learning – is frequently neglected, a situation often aggravated in brief English for Specific Purposes (ESP) courses that focus on specific content and hardly cover students’ grammatical needs. However, as Dudley-Evans and St. John (1998, pp. 74-80) make quite clear, teaching Languages for Specific Purposes (LSP) should not give carte blanche to overlook knowledge gaps in grammar. In an attempt to improve this state of affairs, WISE merges these two referred 316

Andrés Palacios Pablos premises, searching for a way to take advantage of the illustrative power of images as a support for language grammar teaching. While metaphor theory can provide some theoretical rationale to explain the cognitive extent to which visuals are mappable onto languages, it suffices here to outline the following notion: if graphic supports help to understand complex concepts (in our case, grammatical), it is because some common conceptual metaphors like ‘understanding is seeing’ can ease the mapping of our commonplace knowledge about visuals onto students’ own knowledge of grammar (cf. Lakoff, 1987, p. 222). Then, recognising the power visuals have, we can map them onto our understanding of language and deploy proposals that, like WISE, can draw on students’ visual knowledge and help them improve their frequently blurred idea of grammar. If these were the premises that somehow made the conceptualisation of WISE possible, the stimulus that initiated its conception and ultimately defined its goal however, was the frequent verb-related mistakes made by low-level English as a Second Language (ESL) and ESP students alike. Many of them are false beginners with a rudimentary grammar base. As failing to use verbs properly can undermine communication seriously, it seemed worthwhile to pay verbs due attention and take into account any new insights of their performance to relieve or upgrade students’ learning. That was exactly the project’s final target. 2. From WISE analysis to the WISELAV web application WISE (Palacios, 2009) is a verbo-graphic analogy developed to describe the English verb by means of a system of fitting shapes. It exemplifies verbs as they combine to produce different verb phrases or groups (henceforth VGs). This construal of verb structure through fitting shapes, in turn, involved a thorough analysis of verb rules at work so that their functional and morphological performance could be conveyed onto the parallel, purposely-created visual 317

Chapter 27 system. As a result, it works rather well illustrating many operational aspects of verbs such as the different degree of inflection and periphrasis VGs have in order to produce grammatical forms and meanings, the subject-verb agreement, modals’ different operation and the distinct telescoping arrangement of English verb structure. All these features often lead to the verb mistakes referred to before and so, in principle, WISE could meet a good deal of students’ verb grammar needs and increase their linguistic awareness. Table 1. Principles and contrast between the English verb system and WISE Principles and contrast English verb system Graphic WISE scheme Typical paradigm of information flow Each unit (word/shape) has 2 codes (transmitter and receptor). Reversion Some code is passed forwards to the next unit. of transfer system Morphological codes: Graphic codes: Functional value of units (Transmitting stems + (Receptive backs + receptive endings). transmitting fronts). Irregular morphologies Values of the first verb Inverted or non-continuous: Direct-contact or (operator, in compound structures) Stem+(…)→ flat transfer: (…)+Receptive ending Back0+Front1 Stem1+ending0 → Back1+Front2 … → Stem2+ending1 … There are several WISE units represent functional verb categories. forms of functional status. Some auxiliary verbs can Those verbs have different vary their functions and WISE shapes depending thus their categories. on their function. Compound structures Whether they are have the functional explicit or not, WISE value given by the always represents their combination of units. functional value. Even if shortened or omitted, they are disambiguated by context. Some forms are Yet their representation metamorphosed into with WISE is irregular morphologies or always regular. have receptive code Ø. Subject-verb agreement Both variables (rarely displayed in displayed in the active English) and mode/ core of first shape. tense disjunction. 318

Andrés Palacios Pablos The process to develop a computer program has brought about some improvements to WISE’s first interpretation of the English verb. These improvements, together with the on-going software development and some preliminary trials, account for WISELAV’s current state of the art. Before referring this, it is highly convenient to recover some basic principles of its graphic analysis, here briefly related to their analysed target. WISE’s graphic approach allows a higher level of representational meaning than linguistic instances: rather than precise VG examples alone, WISE also produces the patterns of verb structure to which any discrete example belongs. This can easily be explained: the last link of the VG chain (unless there is an ellipsis or omission) must always be a lexical verb, and that lexical verb, even if it is a single concrete one, is seen as a category with no further distinction. An important improvement has been the reduction of WISE verb assignments provided by Palacios (2009, p. 106). The initial system of six possible backs and six possible fronts has been simplified to just five fronts and five backs (Figure 1). Whereas the backs are related to the morphological receptors of verb endings (except for the subject-verb agreement, which appears in the shape cores), the fronts are associated with the six functional verb categories (modals and auxiliary ‘do’ share the same front and hence merge in just one front, F1). These ten graphic contour codes can be considered the WISE basic constituent parts that make up single-shaped verb units together with their structural extended patterns. To increase distinction, different colours have been given to the six verb categories. Apart from the contour assignments resulting in five backs and five fronts, WISE makes use of a few other useful distinctions in order to produce its graphic shapes. First, to distinguish between unchanging and variable assignments and thus resemble verbs’ mentioned functional variability at times, some contours can be represented with continuous or broken lines, depending on whether they represent fixed or variable verb functional categories. Following these criteria, for example, the front of lexical verbs has a continuous line, which shows this category is always depicted this way (Figure 1, F5). At sentence level, we could take into account transitivity and so make some graphic 319

Chapter 27 distinction for such a feature. However, to focus on the verb phrase, WISELAV neglects that variable. Figure 1. WISE’s upgraded assignments of backs and fronts Another WISE distinction is the one made for cores. Finite verbs, apart from their intrinsic grammatical meanings (in Figure 1), carry the specific values of finiteness: subject-verb agreement (although not so much in English) and conveying either tense or modality. These features can only be expressed by the verbs appearing in VG first position within sentences. In order to depict these meanings, WISE distinguishes finite verbs with an active core (Figure 2a) that can symbolically differentiate among all the mutually exclusive values each of these two variables can take. On the other hand, verbs become non-finite when occurring in whatever other circumstances, therefore carrying neither subject agreement nor any tensed or modal load. This condition is shown through a black non-active core (Figure 2b). Actually, in a strict sense, it is the non-finite backs (Figure 1, B2-4) that can be considered the only three possible receptive shapes in an extended VG, the other two backs being finite receptors (Figure 1, B1 & 5) and therefore carriers of subject agreement. This fact can prove of interest when teaching the system. Figure 2. Patterns of finiteness and telescopic arrangement 320

Andrés Palacios Pablos These distinctive criteria (front/back shape, colour, line type and core) work together to provide an accurate description of verbs. In the case of modals, for instance, as they can only perform as finite modal auxiliaries, that quality always makes them finite-core shapes with both back and front of continuous round contour. Similarly, we can check the shapes of the three non-finite verb simple forms (Figure 1, B2-4) and easily visualise it in other possible extended structures (see the schematic WISE telescopic verb arrangement in Figure 2c). In this way, when a verb is conjugated, it will necessarily follow the sequence of first finite then non-finite cores. It must also be reminded that the ten graphic assignments were not given capriciously but, up to what was feasible, by considering the time semantic value each adds to the VG and trying to convey some mnemonic strategy. In order to ease the visual proposal and make the most of its cognitive possibilities, everything was carefully looked into, deliberately pursuing some principles of iconicity (such as conciseness, generalisation, autonomy, and structure). Their correct assembly allows us to project at a glance the five basic verb combinations (cf. Downing & Locke, 2006; Quirk, Greenbaum, Leech, & Svartvik, 1985) while mirroring their restrictions (Figure 3). Figure 3. The five basic verb combinations Thus, by following grammar rules and applying the explained distinctions of continuous-broken line and core type, we have what can be regarded as the nine 321

Chapter 27 WISE basic verb shapes (Figure 4). As they need proper understanding and learning to deal easily with the different verb patterns, it is worth recollecting their mnemonic support: 1) Infinitive: back of forward semicircle, neutral semantic value. 2) Past participle: back of backward arrow, value of completed action. 3) –ing form: back of forward arrow, value of action in progress. 4) Auxiliary ‘do’: Shape that recalls its capital D. 5) Auxiliary ‘have’: backward-arrow front linkable to the perfective aspect. 6) Progressive ‘be’: forward-arrow front linkable to the progressive aspect. 7) Modal: Invariable shape that may remind us of a circle. 8) Passive ‘be’: zigzag front that may recall the passive role drift to the subject as verb action addressee. 9) Lexical verbs: vertical front that usually closes the verb chain. Figure 4. The nine WISE basic verb forms 322

Andrés Palacios Pablos The continuous-broken line distinction is useful for synthesis and research purposes when studying the different patterns of structure. However, it is not so when describing concrete VGs whose components have concrete functions. Therefore, broken lines are not generated with the WISELAV application, intended to create plain shapes of particular examples with fixed contours. The first version (http://www3.ubu.es/wiselav) already highlights the visual aspects of the system but has just partly developed some of the necessary subsystems: an introduction; the example builder, meant to upload an assortment of examples designed to practise and validate WISE pedagogical interest; a user’s application, to allow interaction with the system and perform the designed learning tasks; and a teacher’s management application, to allow teachers to control the different possibilities the platform offers. Underlying the example builder and the user’s application is the WISELAV Figures Generator (henceforth WFG, Figure 5), a subsystem that holds all the concepts defined in WISE as interactive icons to be used and form the different shapes. Figure 5. WFG At the sides, the backs and fronts appear to generate verb shapes. Clicking on the subject displays the subject-dummy and activates the core, conceived to 323

Chapter 27 mark subject agreement and tense-modal value of the first verb by choosing among the different options of the core’s left and right tabs. The contraction icon allows to show both the negative contractions (clicking on contraction + oval + NOT), and the operator contractions (clicking on subject + contraction + front). Finally, between subject and contraction appear the non-verbal elements that can intervene and interact with verbs. The oval displays those elements which do not alter the verb’s morphological transfer (inverted subject, negation, intervening adjuncts and, occasionally, the object), as in the role of double dominoes. Rather than ovals, they actually become strips replicating the preceding front. The fork reproduces the effect that certain coordinators (AND, OR, BUT, the comma, etc.) have in inter- verbal position. Its two variable sides allow for graphic adjustments in order to integrate the morphological changes these coordinators bring about in the VG. Prepositions have the shape of a forward arrow triangle, which fits in the back of –ing forms, thus imitating their linguistic behaviour. The TO particle is represented by a forward semicircle that fits the back of infinitives. The lexical expander is a rectangular band, which is narrower than verb shapes and which can represent both intervening non-verbal elements and other components external to the VG (thus allowing for sentence level analysis). When a lexical verb front is formed, the system is programmed to pop up the interactive addition icon. On the left appears the explicit addition, meant to represent examples like ‘to be able to’; on the right appears the inherent addition, which can be employed for cases whose internal morphological transfer is not explicit, cases such as ‘to stop + –ing’. If both addition icons remain unclicked, the next shape is not attached, as in ‘before coming’. All in all, being provided with these interactive icons, the WFG can represent both the mainstream English verb conjugation and other off-mainstream verb arrangements, like lexical auxiliaries, catenative verbs and phrasal verbs. What is most revealing is that the WFG can do this without betraying WISE principles, and hence, show English verb grammar at work. 324

Andrés Palacios Pablos However, in order to offer a complementary teaching resource, WISELAV also includes a smaller analysis of tense functions through a series of diagrams correlated to each particular tense and deliberately simple (based on the communicative power of arrows, brief labels and examples). These slides can emerge along with examples and thus offer a quick functional hint for students. Figure 6 charts the diagrams corresponding to the present progressive. Figure 6. Uses of present progressive 3. WISELAV’s progress and future implementation The software system has been designed to detect and show mistakes and, by doing so, it is expected to increase users’ linguistic awareness and cut down on their mistakes. In any case, WISELAV objectives must not be misunderstood. The project is not meant to produce an alternative, complete grammar, or to replace conventional teaching methodology. Rather, it is meant to provide some auxiliary support and increase the learning of some grammatical issues. In a second phase (http://www3.ubu.es/wiselav2), the program has been redesigned and upgraded to allow for a better management of exercises and results. For instance, an exercise builder can now upload specific tasks associated to selected examples (avoiding random example occurrence), manual sound recording of examples is added, the colour distinction of verb categories has been implemented, and a score-keeping application registers exercise results as 325

Chapter 27 accessible data for subsequent analysis. However, important advances remain to be implemented: the future software should allow us to record sound and manage sentence segments better and, overall, it should be more user-friendly. Once these features are integrated into a smoothly-run application, the different shapes will be close to having ‘their own life’. This will enable an appropriate use online, providing a valuable learning tool. The results obtained in some preliminary trials seem to support this prospect. References Downing, A., & Locke, P. (2006). English grammar. A university course. New York: Routledge. Dudley-Evans, T., & St. John, M. J. (1998). Developments in English for specific purposes. Cambridge: Cambridge University Press. Jiang, X., & Grabe, W. (2007). Graphic organizers in reading instruction: research findings and issues. Reading in a Foreign Language, 19, 34-55. Lakoff, G. (1987). Image metaphors. Metaphor and Symbolic Activity, 2(3), 219-222. Retrieved from http://dx.doi.org/10.1207/s15327868ms0203_4 Littlemore, J. (2004). Item-based and cognitive-style-based variation in students’ abilities to use metaphoric extension strategies. Ibérica, 7, 5-31. Myers, G. (2003). Words, pictures and facts in academic discourse. Ibérica, 6, 3-13. Palacios, A. (2009). Languages are (like) visuals: linguistic considerations and potential usage. Ibérica, 17, 99-118. Quirk, R., Greenbaum, S., Leech, G., & Svartvik, J. (1985). A comprehensive grammar of the English language. London: Longman. 326

28Generating a Spanish affective dictionary with supervised learning techniques Daniel Bermudez-Gonzalez1, Sabino Miranda- Jiménez2, Raúl-Ulises García-Moreno3, and Dora Calderón-Nepamuceno4 Abstract Nowadays, machine learning techniques are being used in several Natural Language Processing (NLP) tasks such as Opinion Mining (OM). OM is used to analyse and determine the affective orientation of texts. Usually, OM approaches use affective dictionaries in order to conduct sentiment analysis. These lexicons are labeled manually with affective orientation (polarity) of words such as positive or negative. There are few dictionaries of affective orientation for Spanish; also, the size of these dictionaries is small. Thus, we propose a method for building a large affective Spanish dictionary for subjectivity and sentiment analysis. Supervised learning techniques are used to classify the entries from a lexical dictionary according to their affective orientations based on their definitions. We combine three classifiers (decision trees, naive Bayes, and a support vector machine) to determine the final polarity of each entry, that is, positive or negative. Keywords: opinion mining, subjectivity and sentiment analysis, affective orientation, polarity detection. 1. Prospectiva en Tecnología e Integradora de Sistemas SA de CV, Ciudad de México, Mexico; [email protected] 2. INFOTEC- Centro de Investigación e Innovación en Tecnologías de la Información y Comunicación, Aguascalientes, México; [email protected] 3. Services & Processes Solutions, Ciudad de México, México; [email protected] 4. UAEM UAP Nezahualcóyotl, Estado de México, México; [email protected] How to cite this chapter: Bermudez-Gonzalez, D., Miranda-Jiménez, S., García-Moreno, R.-U., & Calderón-Nepamuceno, D. (2016). Generating a Spanish affective dictionary with supervised learning techniques. In A. Pareja-Lora, C. Calle- Martínez, & P. Rodríguez-Arancón (Eds), New perspectives on teaching and working with languages in the digital era (pp. 327-338). Dublin: Research-publishing.net. http://dx.doi.org/10.14705/rpnet.2016.tislid2014.445 © 2016 Daniel Bermudez-Gonzalez et al. (CC BY-NC-ND 4.0) 327

Chapter 28 1. Introduction In recent years, the automatic processing of opinions has increased because of its potential applications. One of them is sentiment analysis (Pang & Lee, 2008) in social networks. Most people write their opinions in forums, review sites, and microblogging (Twitter, Facebook, among others). This information is useful for companies, governments, and individuals who want to obtain global feedback for their activities or products. Machine learning techniques have been used to face sentiment analysis problems, namely, to determine the affectivity of texts: their positive or negative orientation. As stated by Banea, Mihalcea, and Wiebe (2011), “[m]uch of the research work […] on sentiment and subjectivity analysis has been applied to English, but work on other languages is [a] growing [need]” (p. 1). In this paper, we propose a new method to build a subjectivity and sentiment dictionary for Spanish based on the definitions from an explanatory dictionary and three classifiers, which will be employed to perform sentence level sentiment classification. 2. Related work Lexicons have been used for subjectivity and sentiment analysis because they can be applied to identify opinions or emotions by means of rule-based opinion classifiers. For example, there are several popular lexicons for subjectivity and sentiment analysis for English, such as the OpinionFinder lexicon (Wiebe & Riloff, 2005), which contains 6,856 unique entries associated with a polarity label (positive, negative, neutral). SentiWordNet (Esuli & Sebastiani, 2006) is another popular lexicon, which is based on WordNet (Miller, 1995) and encompasses more than 100,000 words. It was automatically generated, starting with a small set of manually labeled synsets. A synset represents a group of cognitive synonyms (nouns, verbs, adjectives, or adverbs) that express a distinct concept. 328

Daniel Bermudez-Gonzalez, Sabino Miranda-Jiménez et al. In the case of the Spanish language, there are few dictionaries of affective orientation. One of them, the Spanish Emotion Lexicon (SEL), has 2,038 words (Díaz-Rangel, Sidorov, & Suárez-Guerra, 2014; Sidorov et al., 2012). This dictionary was manually classified into 6 affective categories (joy, anger, fear, sadness, surprise, and disgust). The Polarity Lexicon (PL) presented in Saralegi and San Vicente (2013) has 4,738 words classified as positive or negative. Another lexicon, the Spanish Sentiment Lexicon (Pérez-Rosas, Banea, & Mihalcea, 2012) uses a cross-language expansion approach based on WordNet to determine the polarity. This lexicon has 3,843 words, classified as positive or negative. Our approach is different: we used the entry definition of an explanatory dictionary to determine the polarity of the entry itself. We used two affective dictionaries manually labeled (SEL and PL) to train the classifiers in order to classify the entries from a large explanatory dictionary. 3. Building the affective dictionary We were interested in discovering the positivity or negativity of dictionary entries in order to use them in opinion mining tasks. Thus, our objective was to automatically build an affective dictionary for Spanish with two categories: positive and negative. However, other works have used more categories to determine semantic orientation of messages or documents. The number of classes used depends on the particular purposes and the domains of these works. For instance, Pang, Lee, and Vaithyanathan (2002) used two classes (positive and negative) for movie reviews, and Pérez-Rosas et al. (2012) used these two classes for generating sentiment lexicons in a target language using annotated English resources. Three classes (positive, negative, and neutral) were the base to predict contextual polarity of subjectivity phrases in a sentence in Agarwal, Biadsy, and Mckeown (2009); four classes (positive, negative, neutral, and informative) to determine the semantic orientation on Twitter data (Sidorov et al., 2012); and six classes helped determine a fine-grained affective 329

Chapter 28 orientation in sentences (joy, anger, fear, sadness, surprise, and disgust) in Díaz- Rangel et al. (2014). In our approach, we used three dictionaries in order to obtain the resulting affective dictionary: an explanatory dictionary, which has the words (entries) to be classified, and two lexicons, labeled by hand with different affective categories. 3.1. The affective lexicons Two labeled affective dictionaries were used to train the classifiers: the SEL lexicon (Díaz-Rangel et al., 2014; Sidorov et al., 2012) and the PL lexicon (Saralegi & San Vicente, 2013). In our approach, we used only two categories; thus, we mapped SEL’s categories to a positive or negative category, that is, joy and surprise to positive, anger, fear, sadness, and disgust to negative. 3.2. Preprocessing of the explanatory dictionary We used the words from Anaya explanatory dictionary of Spanish as input data to be classified (30,228 entries). Also, we used the entry definitions in two different ways: first, the definitions of words from the affective dictionaries were used to train the classifiers, and second, the definitions of the remaining entries were used to classify the entry itself. In order to prepare the dictionary entries for classification, we removed from the entries all phrases and words with no alphabetic symbols, suffixes, and prefixes (such as mountain bike, modus vivendi, ‘ido, ida’, -a, or neumo-). Also, we removed stop words such as articles, prepositions, and conjunctions. We just used content words, that is, single words such as abeja, abrumar, rata, etc. To process the definition of each dictionary entry, we applied some rules. For example, if the entry definition had a text such as ‘véase CONCEPT’ (see CONCEPT), then the definition of the CONCEPT was searched in the explanatory dictionary and was used instead of ‘véase CONCEPT’. In the 330

Daniel Bermudez-Gonzalez, Sabino Miranda-Jiménez et al. following example, the definition of alcohómetro is replaced by the definition of alcoholímetro. Applying this rule the substitution is as follows: ENTRY DEFINITION alcohómetro alcohómetro véase alcoholímetro Dispositivo para medir la cantidad de alcohol presente en el aire expirado por una persona. We also removed from definitions numbers, suffixes, prefixes, phrases with abbreviations, and abbreviations such as ‘del lat.’, ‘del ár.’, FAM., vulg., among others. For example, this sort of particle is removed in the following definitions: ENTRY DEFINITION ruborizar sonrojar, adquirir o producir rubor en el rostro. FAM. ruborizado. palmar del lat. palmare, golpear; o del caló palmar, acabar. Finally, in this step, if the words had multiple definitions, we used the most frequent ones, that is, we selected a percentage of the definitions, because different definitions of a same word have different affective orientations for the same word. In the following example, the word perro (dog) has multiple definitions and each definition has a different affective orientation; the definitions 2 and 3 have clearly negative meanings, and the definitions 5, 7, and 8 have positive meanings. ENTRY DEFINITION perro 1) Nombre común de cierto mamífero carnívoro, doméstico, del que hay infinidad de razas muy distintas entre sí por la forma, el tamaño y el pelaje 2) muy malo [lleva una vida de perro] 3) Dícese de la persona vil, traidora y astuta [no te fíes de él, es muy perro] 4) Persona que siempre va pegada a otra. 5) Persona que acompaña tenazmente a otra para protegerla de supuestos peligros. 6) Perro faldero. Perro pequeño que siempre acompaña a su amo. 7) Perro guardián. Perro que guarda una propiedad. 331

Chapter 28 8) Atar los perros con longaniza. Frase con que se da a entender la abundancia o riqueza. Because the linguistic resource that we have created aims at supporting other practical applications for opining mining, we classified the most frequent meanings of the explanatory dictionary in order to avoid too many semantic orientations of a same word. That is, we selected the first definitions up to a predetermined percentage. The explanatory dictionary lists its entries from the most frequent meanings to the least frequent meanings. The percentage was defined using the following rules. If a word had 1-3 senses (definitions), all senses were used; if a word had 4-6 senses, 80% of the senses were used; if a word had 7-10 senses, 60% are used; if a word had more than ten senses, 40% are used. For instance, five senses were selected for the word perro mentioned above. We used these percentages because the distribution of number of senses per word is substantially reduced after three senses in the Anaya dictionary. For example, words that had from one to three senses were 26,064; four senses, 1,792; and five senses, 774 (Gelbukh, Sidorov, & Ledo-Mezquita, 2003). 3.3. Preprocessing of the training data In order to train the classifiers, we used the word definitions of SEL and PL. We used only the content words of the definitions, as we mentioned. In order to reduce their dimensionality, the Porter (2006) stemming algorithm for Spanish was applied to the definitions. For example, after applying the preprocessing to the original text (1), we obtained a transformed text (2) which is used as a unigram model to train the classifiers, that is, we use single words (stems). alegre dícese de la persona, gesto, etc., que tiene o original text (1) denota alegría transformed text (2) alegr dices person gest tien denot alegr 332

Daniel Bermudez-Gonzalez, Sabino Miranda-Jiménez et al. 3.4. Selected classifiers Our method uses three machine learning classifiers. The selected machine learning classifiers were Naïve Bayes (NB), Decision Trees (DTs) and Support Vector Machines (SVMs). We used the Waikato Environment for Knowledge Analysis (WEKA) software (Hall et al., 2009) that implements the machine learning algorithms mentioned above, and we implemented our version of the NB algorithm. WEKA implements SVM as a Sequential Minimal Optimisation (SMO), and DT with J48 algorithms. Our input data is a vector. Each entry (stem) in the vector corresponds to a feature. For SVM and DT, a Term Frequency-Inverse Document Frequency (TF-IDF) weighting approach was used (Salton & Buckley, 1988), that is, we used not only the presence of each feature, but also its global importance in the explanatory dictionary. We used a training set that consists of 5,222 words, which came from the two affective lexicons (SEL and PL). The training set has 1,924 positive words and 3,298 negative words. Additionally, the test set consists of 3,000 words that were randomly selected from the explanatory dictionary in order to be labeled manually to assess our method. The resulting test set has 2,316 positive words and 684 negative words. 4. Experiments and results As mentioned above, our method uses three machine learning classifiers and consists of three steps. First, we generated the model for each classifier considering the training data. Second, we classified the entries from the explanatory dictionary using each model; thus, three affective dictionaries were generated, that is, one for each 333

Chapter 28 classifier. Third, we combined the results of the three classifiers in one affective dictionary using a voting scheme. For example, the word hurgar (delve) was labeled as positive (p) by NB, negative (n) by SVM, and negative (n) by DT; thus, the global orientation was negative because there were two votes for negative orientation. This strategy was applied for all entries from the Anaya dictionary. We applied standard measures used in many NLP tasks. The precision (P) of a system is computed as the percentage of correct answers given by the automatic system. Recall (R) is defined as the number of correct answers given by the automatic system over the total number of answers to be given. F-measure is the harmonic mean of precision and recall. The coverage is 100%, thus precision and recall are equal (Navigli, 2009). The results obtained for each classifier are shown in Table 1. As shown in this table, we generally obtained better results when we combined the results of the classifiers. The precision obtained was 67%. The precision for the positive category was 71%, and for the negative category was 53.3%. Table 1. Evaluation of the classifiers Classifier Category Precision Recall F-measure SVM - 49.4% 49.4% 49.4% Positive 42.0% 42.0% 42.0% DT Negative 74.2% 74.2% 74.2% - 30.3% 30.3% 30.3% NB Positive 12.7% 12.7% 12.7% Negative 89.7% 89.7% 89.7% Voting Scheme - 59.6% 59.6% 59.6% Positive 60.5% 60.5% 60.5% Negative 56.7% 56.7% 56.7% - 67.0% 67.0% 67.0% Positive 71.0% 71.0% 71.0% Negative 53.3% 53.3% 53.3% 334

Daniel Bermudez-Gonzalez, Sabino Miranda-Jiménez et al. The results show that the performance for the negative category is reduced. Each classifier alone is better than the voting scheme because of discrepancies among classifiers. For example, if two classifiers vote for a positive polarity, the word will be classified as positive, even if the SVM classifies it correctly as negative. The resulting lexicon has 30,773 affective words, including different meanings for each word. For example, the number accompanying the word rosa (3) indicates that the third meaning was used to determine its polarity. In the resulting dictionary, we included the polarity, the word, and the gloss that describes the sense of the word. For example, in Table 2, we show some results of the dictionary. The first column indicates the polarity as positive (p) or negative (n), the second column indicates the classified word (hurgar / delve), and the last column indicates the meaning of the word. Table 2. Excerpt from the affective Spanish dictionary Results Word Gloss Polarity hurgar remover una cosa, escarbar. n chorrear caer un líquido a chorro. n n roer raspar con los dientes una cosa, generalmente un alimento, arrancando parte de ella. n rosa3 mancha rosácea que sale en el cuerpo. n aberrar andar errante, equivocarse, aberración, aberrante. p comer2 tomar la comida principal del día [en mi casa comemos a las dos]. p contribuir pagar las contribuciones o impuestos. With respect to analysis of errors, we identified some errors when analysing the classified words. For example, if the word sense is related to a common animal (error type 1), the classified word has no positive or negative polarity at all; human annotators also hesitate how to classify these sorts of words. In the second error type, the definition is very short at word level; it is difficult that classifiers assign the correct label due to lack of context. 335

Chapter 28 Table 3. Errors in the classification process Error Example 1. common animal p|perro|nombre común, mamífero carnívoro, doméstico (p|dog| common animal, carnivorous mammal, domestic) 2. short definition n|rivera|arroyo (n|stream|creek) n|revuelto|de revolver (n|mess-up|to mess up) 5. Conclusions In this paper, we have presented a method that generates a large affective lexicon using supervised learning techniques for Spanish. We evaluated the results obtained using a test set with 3,000 words that were selected randomly and labeled by hand. The training set used consisted of 5,222 words from two affective lexicons (SEL and PL). The resulting lexicon has 30,773 words, classified as positive or negative words, including different meanings for each word. The precision obtained was 67.0%. It shows that the quality of our lexicon outperforms that of lexicons whose entries are classified using just one classifier. In the future, we will not use stems to train the classifiers. In order to improve their performance, we will use lemmatisers, part of speech taggers, or syntactic n-grams (Sidorov et al., 2014) for example, since they have already been used for this purpose with good results. References Agarwal, A., Biadsy, F., & Mckeown, K. (2009). Contextual phrase-level polarity analysis using lexical affect scoring and syntactic n-grams. In Proceedings of EACL 2009 (pp. 24- 32). Greece: ACL. Retrieved from http://dx.doi.org/10.3115/1609067.1609069 Banea, C., Mihalcea, R., & Wiebe, J. (2011). Multilingual sentiment and subjectivity analysis. In D. Bikel & I. Zitouni (Eds.), Multilingual natural language processing applications: from theory to practice. Boston: IBM Press. Retrieved from http://people.cs.pitt. edu/~wiebe/pubs/papers/multilingualSubjBookChap2011.pdf? 336

Daniel Bermudez-Gonzalez, Sabino Miranda-Jiménez et al. Díaz-Rangel, I., Sidorov, G., & Suárez-Guerra, S. (2014). Creación y evaluación de un diccionario marcado con emociones y ponderado para el español. Onomazein, 29, 1-23. Retrieved from http://dx.doi.org/10.7764/onomazein.29.5 Esuli, A., & Sebastiani, F. (2006). SentiWordNet: a publicly available lexical resource for opinion mining. In Proceedings of LREC 2006, (pp. 417-422). Italy. Gelbukh, A., Sidorov, G., & Ledo-Mezquita, Y. (2003). On similarity of word senses in explanatory dictionaries. International Journal of Translation, 15(2), 51-60. Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P., & Witten, I. H. (2009). The WEKA data mining software: an update. ACM SIGKDD Explorations Newsletter, 11(1), 10-18. Retrieved from http://dx.doi.org/10.1145/1656274.1656278 Miller, G. A. (1995). WordNet: a lexical database for English. Communications of the ACM, 38(11), 39-41. Retrieved from http://dx.doi.org/10.1145/219717.219748 Navigli, R. (2009). Word sense disambiguation: a survey. ACM Computing Surveys (CSUR), 41(2). Retrieved from http://dx.doi.org/10.1145/1459352.1459355 Pang, B., Lee, L., & Vaithyanathan, S. (2002). Thumbs up?: sentiment classification using machine learning techniques. In Proceedings of the ACL-02 conference on Empirical Methods in Natural Language Processing (pp. 79-86). USA: ACL. Retrieved from http:// dx.doi.org/10.3115/1118693.1118704 Pang, B., & Lee, L. (2008). Opinion mining and sentiment analysis. Foundations and Trends in Information Retrieval, 2(1-2), 1-135. Retrieved from http://dx.doi.org/10.1561/1500000011 Pérez-Rosas, V., Banea, C., & Mihalcea, R. (2012). Learning sentiment lexicons in Spanish. In Proceedings of LREC 2012, (pp. 3077-3081). Turkey: ELRA. Porter, M. F. (2006). Spanish stemming algorithm. Retrieved from http://snowball.tartarus. org/algorithms/spanish/stemmer.html Salton, G., & Buckley, C. (1988). Term-weighting approaches in automatic text retrieval. Information Processing and Management, 24(5), 513-523. Retrieved from http://dx.doi. org/10.1016/0306-4573(88)90021-0 Saralegi, X., & San Vicente, I. (2013). Elhuyar at TASS 2013. In Proceedings of XXIX Congreso de la Sociedad Española de Procesamiento de lenguaje natural. Workshop on Sentiment Analysis at SEPLN (TASS2013) (pp. 143-150). Madrid: SEPLN. Sidorov, G., Miranda-Jiménez, S., Viveros-Jiménez, F., Gelbukh, A., Castro-Sánchez, N., Velásquez, F., Díaz-Rangel, I., Suárez-Guerra, S., Treviño, A., & Gordon, J. (2012). Empirical study of machine learning based approach for opinion mining in tweets. In Advances in Artificial Intelligence (MICAI 2012) (pp. 1-14). Mexico: Springer. 337

Chapter 28 Sidorov, G., Velasquez, F., Stamatatos, E., Gelbukh, A., & Chanona-Hernández, L. (2014). Syntactic N-grams as machine learning features for natural language processing. Expert Systems with Applications, 41(3), 853-860. Retrieved from http://dx.doi.org/10.1016/j. eswa.2013.08.015 Wiebe, J., & Riloff, E. (2005). Creating subjective and objective sentence classifiers from unannotated texts. In Proceeding of CICLing-05, International Conference on Intelligent Text Processing and Computational (pp. 486-497). Mexico: Springer. Retrieved from http://dx.doi.org/10.1007/978-3-540-30586-6_53 338

29Transcription and annotation of a Japanese accented spoken corpus of L2 Spanish for the development of CAPT applications Mario Carranza1 Abstract This paper addresses the process of transcribing and annotating spontaneous non-native speech with the aim of compiling a training corpus for the development of Computer Assisted Pronunciation Training (CAPT) applications, enhanced with Automatic Speech Recognition (ASR) technology. To better adapt ASR technology to CAPT tools, the recognition systems must be trained with non-native corpora transcribed and annotated at several linguistic levels. This allows the automatic generation of pronunciation variants, new L2 phoneme units, and statistical data about the most frequent mispronunciations by L2 learners. We present a longitudinal non-native spoken corpus of L2 Spanish by Japanese speakers, specifically designed for the development of CAPT tools, fully transcribed at both phonological and phonetic levels and annotated at the error level. We report the results of the influence of oral proficiency, speaking style and L2 exposition in pronunciation accuracy, obtained from the statistical analysis of the corpus. Keywords: non-native spoken corpora, spontaneous speech transcription, L1 Japanese, L2 Spanish, standards for transcription and annotation. 1. Universitat Autònoma de Barcelona, Barcelona, Spain; [email protected] How to cite this chapter: Carranza, M. (2016). Transcription and annotation of a Japanese accented spoken corpus of L2 Spanish for the development of CAPT applications. In A. Pareja-Lora, C. Calle-Martínez, & P. Rodríguez-Arancón (Eds), New perspectives on teaching and working with languages in the digital era (pp. 339-349). Dublin: Research- publishing.net. http://dx.doi.org/10.14705/rpnet.2016.tislid2014.446 © 2016 Mario Carranza (CC BY-NC-ND 4.0) 339

Chapter 29 1. Introduction Several studies have pointed out the possibility of efficiently adapting ASR systems to pronunciation assessment of non-native speech (Neri, Cucchiarini, & Strik, 2003) if technology limitations are compensated with a good design of language learning activities and feedback, and the inclusion of repair strategies to safeguard against recognition errors. An ASR system can be adapted as an automatic pronunciation error detection system by training it with non-native speech data that generates “new acoustic models for the non-native realizations of L2 [phones], and by the systematization of L1-based typical errors by means of rules […]. In order to do so, phonetically transcribed non-native spoken corpora are needed; however, manual transcription of non-native speech is a time-consuming costly task, and current automatic transcription systems are not accurate enough to carry out a narrow phonetic transcription” (Carranza, 2013, p. 168). In this paper we will introduce a corpus of non-native Spanish by Japanese speakers that contains spontaneous, semi-spontaneous and read speech. The corpus is transcribed at the orthographic, phonological and phonetic levels, and annotated with an error-encoding system that specifies the error type and its phonological context of appearance. This database was compiled and annotated considering its future adaptation as a training corpus for developing ASR-based CAPT tools and applications for the teaching of Spanish pronunciation to Japanese speakers. In section 1, we will present the general features of the corpus. Section 2 deals with the levels of transcription, the annotation standards, and the phone inventory used in the transcriptions. Finally, the results of the statistical analysis of errors are presented in section 3, followed by a discussion concerning our findings. 340

Mario Carranza 2. Corpus data description The corpus features 8.9h of non-native speech, divided into semi-spontaneous speech (91’), spontaneous speech (214’), read speech (9’) and conversational speech (201’). Spontaneous speech represents more than 80% of the recordings. The data was obtained from 10 male and 10 female Japanese students of L2 Spanish at the Spanish Department of the Tokyo University of Foreign Studies. They were selected according to their dialectal area (Kanto dialect) and none of them had previous academic contact with Spanish. The corpus contains the oral tests of the 20 informants throughout their two first academic years of Spanish study (from 1/4/2010 to 31/3/2012), which corresponds to the A1 and A2 levels in the Common European Framework of Reference for Language Learning (Council of Europe, 2001). Oral tests took place every six months, and consisted of different types of tasks that involved spontaneous, semi-spontaneous, and read speech. Semi-spontaneous speech was obtained from oral presentations prepared before-hand (in the 1st and 2nd semesters) and spontaneous speech was gathered from conversations between the student and the examiner and role- plays with no previous preparation (in all semesters). Oral proficiency was also taken into account by computing the mean of all the oral-test scores of each informant. Three proficiency levels were established according to this score: low (N=6), intermediate (N=8) and high (N=6). The recordings were made with portable recorders and were segmented into individual audio files. The audio files were converted into WAV format and labelled with information regarding the student, the task type and the period of learning (semester). This allows the automatic computation of error rates according to proficiency level, learning stage and speaking style after the transcription and annotation of the corpus. 3. Levels of transcription Transcription of non-native spontaneous speech is a complex activity due to its high degree of variability and the interference of the L1 and constant 341