Research in English Language Instruction

Terminology Explanation 3. cluster sampling C. The entire group of people or objects to 4. quota which the researcher wishes to generalize the 5. purposive sampling study findings D. The selected elements (people or objects) 6. sample chosen for participation in a study; people are 7. snow ball referred to as subjects or participants 8. systematic random E. A complete set of elements (persons or 9. random sampling objects) that possess some common 10. convenient sampling characteristic defined by the sampling criteria established by the researcher 11. stratified random F. the process of selecting a group of people, 12. sampling frame events, behaviors, or other elements with which to conduct a study 94 G. Sample must be as much like the population in as many ways as possible H. Everyone in the population has equal opportunity for selection as a subject I. Population is divided into subgroups, called strata, according to some variable or variables in importance to the study J. A random sampling process in which every kth (e.g., every 5th element) or member of the population is selected for the sample after a random start is determined K. Network sampling L. Researcher uses personal judgement to select subjects that are considered to be representative of the population

Terminology Explanation 13. sampling M. Selection of sample to reflect certain characteristics of the population 14. representativeness N. Selection of the most readily available people or objects for a study 15. accessible population O. A random sampling process that involves stages of sampling Source: https://www.umsl.edu/~lindquists/sample.html Activity 6: Case studies for claiming on representativeness of the population in survey research Case study 1: Perceptions of Using social media as an ELT Tool among EFL Teachers in the Saudi Context Method Research Design Underpinned by a positivistic paradigmatic assumption, the main theoretical approach of this study is manifested in the descriptive and inferential statistical methods. This approach has been undertaken so as to target the quantitative data. This is due to the fact that providing description of a specific social or human phenomenon is more easily applicable than applying the experimental approach on humanities. The main tool used in the research has been through the use of a survey due to its efficiency in terms of cost, time and effort in studying the entire population (Ornstein, 2013). Participants For the sake of having an unbiased and a representative sample of the population, the researcher has selected opted to rely on a probability simple random sample 95

(Blair, Czaja, & Blair, 2013) of 75 individuals (n = 75). There were 35 males and 40 female participants. All the participants work at two different Saudi universities in two different cities. (Source: Allami, M. & T. Elyas. (2016). Perceptions of Using social media as an ELT Tool among EFL Teachers in the Saudi Context. English Language Teaching, 9 (7), 1-9.) Case study 2: An Investigation of the Difficulties Faced by EFL Undergraduates in Speaking Skills Participants The whole population of this study is composed of years 1 to 4 EFL students at public universities in different parts of Jordan. Stratified random sampling technique was adopted in the current research. This type is considered less complicated and less expensive (Cohen, Manion, & Morrison, 2007). The researcher has opted for the probability for selecting students at Jordanian Universities. There was clear intention that these Universities should be representative of the English Language departments at public Universities in Jordan and within each participating English Language Department; the students should be representative of the broader university population from which they were drawn. The following Table displays the numbers withdrawn from each university as amounting up to 566 students (247 males and 319 females) who voluntarily cooperated in the present study for the survey questionnaire. Sixty-four students from all groups participated in the current study accepted to have face-to-face interview sessions. The interviews were conducted in order to obtain information about the speaking abilities of the two groups of students. To analyze the data, transcripts were made of the interviews; notes were made from the recorded material on the speaking performance. These data were confirmed with those from the survey questionnaire. 96

(Source: Al-Jamal, D.A. & Al-Jamal G.A. (2014). An Investigation of the Difficulties Faced by EFL Undergraduates in Speaking Skills, English Language Teaching, 7(1), 19- 27. Case study 3: Web-based surveys in educational research Sampling and Representativeness In order to generalize the data to the whole population, the issue of sampling comes to the fore. Foreman (1991) defines the term sampling as “an example of inductive logic by which conclusions are inferred on the basis of a limited number of instances” (p. 1). If the right methods of sampling are followed, the data can easily be generalized to the whole population. In this respect, every survey research entails a proper sampling method to find out a body of respondents that may be people or institutions. Without a proper sampling method, the respondents chosen may not be representative of the target population. According to Bethelem (2009), “to select a proper sample, a sampling design must be defined, and the sample size must be determined such that the required accuracy of the results can be obtained” (p. 3). Bethelem further cites a powerful example, in which scientists and statisticians came to realize how important sampling was with the help of Gallup, an American researcher, and Literary Digest Magazine. Both of these attempted to make a good guess in the presidential elections of 1936 in the US. The former of these correctly guessed that the new president would be Roosevelt, while the latter made a false guess, saying that Alf Landon would win the elections. Although Literary Digest Magazine surveyed an unbelievably high number of people (2 million in contrast to Gallup’s 3 thousand respondents), it failed to come up with a correct guess. The obvious reason for its failure was that the sampling method chosen was incorrect because the researchers surveyed people chosen through telephone directories and car registration lists, which were both typical of middle and upper class, whereas Gallup used quota sampling, surveying people of diverse characteristics within a 97

quota for each type (Bethelem, 2009, pp. 9-10). It is apparent from this example that the sampling method followed is rather more important than the size of the sample. Traditionally three types of sampling methods are cited: snowball sampling, quota sampling and random sampling. Snowball sampling, in Dörnyei’s (2003) words, refers to “a ‘chain reaction’ whereby the researcher identifies a few people who meet the criteria of the particular study and then uses these participants to identify further members of the population” (p. 72), and quota sampling involves the selection of respondents by considering the proportion of the population that belongs to a previously determined subgroup. As for random sampling, as the name suggests, the respondents are chosen randomly as in the case of drawing lots. Random sampling means that all potential respondents have equal chance of being selected. Such a sampling method aims to eliminate the influence of potential variables that might harm the results of the study (Dörnyei, 2003, pp. 72-73). Web survey administrators can carry out random sampling if they have a list of e- mail addresses at hand because it is not possible to generate e-mail addresses unlike the process of choosing random telephone numbers for a telephone survey. When carrying out sampling procedures, the three sampling methods mentioned above can be followed or as Jansen et al., (2007) suggest visitors who visit the survey’s web site by chance while they are normally browsing the web respond to the survey if they want to. The second case is called self-selection. In other words, sampled respondents are intentionally directed to the web site where the survey is, while self-selection involves respondents who themselves choose to visit your site without being notified. A distinction is made between selection and recruitment of the respondents of a survey. Selection involves the method of selecting the respondents, whereas recruitment refers to how the respondents are requested to take 98

Fig. 2. Interaction of Selection and Recruitment Processes in Sampling Beidernikl and Kerschbaumer (2007a) suggest asking three basic questions regarding how representative the sample is: (1) Is the target group of the survey clearly defined, and has every individual access to the Internet? (2) Is it possible to invite these people individually (selectively and actively), based on a specified selection response sampling procedure to the survey? (3) Is this sampling procedure in case of the underlying population suitable to collect representative data? Coming to the first question, if the researcher uses selective and active invitation processes mentioned above to do a research targeting a specific group (e.g., EFL teacher working in high schools in Turkey) and chooses respondents who are representative of the whole population, he/she faces no problems regarding the first question mentioned above. Concerning the second, the researcher has to find out contact information (e-mail addresses) of the respondents. In order to reach reliable data, the researcher in our example should choose teachers of different educational and socio-economic backgrounds, geographical locations (the number selected respondents in a specific geographical location should be proportionate to the total number of EFL teachers in that area and others), ages and teaching experience. Otherwise, some sampling errors and the problem of representativeness would emerge. Some respondents might deliberately or accidentally respond to the questionnaire more than once. To avoid such an undesirable situation, repeated responses should be blocked by cookies or IP addresses. There is still the risk that the respondent might complete the survey from different computers repeatedly, 99

but such a case is extremely rare. Using passwords or access codes is another way to eliminate such a problem. Apart from repeated responses, we have the problem of not knowing who the actual respondents are. When we send the questionnaire link to a group of respondents, do we know that it is these people who really respond? This lack of knowledge is equally problematic for most type of surveys either electronic or paper-and-pencil surveys. However, in the case of active and selective sampling, since e-mail addresses are used, it is assumed that the respondents are the owner of the e-mail addresses. Whether you follow snowball sampling, quota sampling or random sampling, the sample must be representative of the whole population. In web-based surveys the issue of representativeness is a bit problematic. Discussing this issue, Beidernikl and Kerschbaumer (2007a) mention three levels of representativeness: (1) Representativeness for the whole population, (2) Representativeness for the Internet population and (3) Representativeness for a certain population. The most problematic level is representativeness for the whole population. This is because nearly half of the people even in developed countries do not have access to the Internet. For example, 52.8 % of the households in Turkey, as of April 2012 did not have access to the Internet (Tüik, 2012, On-line) or those with an Internet connection might be different from those who do not in terms of educational and socio-economic backgrounds. Therefore, active and selective invitation procedures should be followed to guarantee high representativeness for a certain population, not for the whole population or the Internet population. To put it another way, respondents from a specific population should be invited to take part in your survey on an individual basis to reach the highest level of representativeness. Some software provides researchers with alternative ways to ensure that only certain people are allowed to take the survey. For example, Datacurious (2008), web-based survey design software, assigns passwords to individual users to make sure that each person responds to the questionnaire once. Although some limits are mentioned to generalize a set of data gained from a small population to a larger one, generalizibility of this data is not directly connected with 100

the number of respondents in the sample. It is the sampling method chosen that is so important in terms of data generalizibility. Imagine that you set out to design a survey on a specific issue that concerns both sexes, but you exclude one of the sexes from the sample, but survey a large number of people of the other gender. Can we expect your data to be generalizable to human beings in general? The answer should be a clear-cut “No.” As it is apparent in the case of Reader’s Digest, no matter how large the sample is, the results are likely to be incorrect if the sample is not representative of the whole population. (Source: Bakla, A., Cekic, A & Koksai, P.O. (2013). Web-based surveys in educational research, International Journal of Academic Research, 5 (1), 4-13.) Case The technique the researcher use to justify the sample for study representativeness of the population 1 2 3 Activity 7: Read the case studies below in each case, identify what type of sampling the study recruited the research participants or respondents. What justification each study has made, can you explain? Case Study 1: A Multistage Random Sampling Technique Employed in a Survey Research: A Study of Thai EFL teachers’ use of media in reading instruction (Nutsareeya Sangpan, 2009) The population for this study was Thai EFL teachers who taught at a secondary school levels in Trang. These teachers volunteered to participate in this study because Thai EFL teachers at the secondary school level tended to use technology and media materials in class more than any other levels. The students at this level also knew how to use technology and media from various sources. 101

The researcher used the multi-stage random sampling to select the sample size in this study. The total numbers of samples in this study were 127 Thai EFL teachers who taught at the small, medium and large secondary school in Trang and 398 Thai EFL students who studied at the secondary school levels in Trang. The number of the participants at each level was calculated by using Yamane (1967)’s formula with a 95% confidence level and a 5% of precision level sampling selected. In this study, there were 52 teachers at the large school, 67 teachers at the medium schools and 8 teachers at the small school as shown in Table 3.2 and 3.3 Table 3.2 Number of sample Teachers: School Size Total numbers of Thai EFL Number of Sample teachers - Small School 13 8 - Medium School 104 67 - Large School 71 52 Total 188 127 Number of Sample Table 3.3 Number of sample Students: School Size Total numbers of Thai EFL students - Small School 2,442 31 - Medium School 14,163 182 - Large School 14,508 185 Total 31,113 39 102

Source: Nutsareeya Sangpan. (2009). A Study of Thai EFL teachers’ use of media in reading instruction. Unpublished thesis. Master of Education Program in Teaching English as a Foreign Language, Department of Curriculum and Instruction, Faculty of Education, Chulalongkorn University. Pp.25-29. Case Study 2: A Delphi Sampling Technique Employed in a Survey Research: Proposed guidelines for developing primary English teacher training programs for provincial teacher training colleges, Kingdom of Cambodia. Unpublished thesis (Pum Chea, 2013). Panel selection is critical when using the Delphi technique. The success of a Delphi study rests upon selecting appropriate experts qualified in the subject area. The effective selection of the panel not only maximizes the quality of responses but also gives the results of the study credibility (Lang, 1998); therefore, in this study the list of prospective panelists was purposively selected from two primary sources in Cambodia. The first group was the educators who were working as English curriculum designers/developers in related offices and organizations such as the Department of Curriculum Development and the Department of Teacher Training of the Cambodian Ministry of Education, Youth and Sports. The second group included English teacher trainers from National Institute of Education (NIE), Regional Teacher Training Colleges (RTTCs) and Provincial Teacher Training Colleges (PTTCs). To be considered as a panelist in this study, each expert needed to have at least two of the following three qualifications: 1. Had a minimum of five-year experiences working as an English teacher educator. 2. Had a minimum of five-year experiences in English curriculum or course development. 103

3. Had a minimum of five-year experiences working as an English teacher trainer at NIE, or RTTC, or PTTC. The final list generated from the first group was 8 participants and the second group was 19 participants, resulted in a total of 27 potential participants. Delphi Panel Profile This section presents the profile of the Delphi panel. Round 1 of this study included a section soliciting participants’ profiles in terms of primary disciplinary expertise, years of professional experience, highest educational degree, and profession. These profile questions were included to understand the expertise of the panel and also explore any subgroup differences in the responses. A total of 27 participants completed responses in Round 1. For Round 2, all 27 respondents of Round 1 survey were invited to participate and 20 completed responses were received, resulting in a response rate of 74%. For Round 3, all 20 respondents of Round 2 survey were invited to participate and 17 completed responses were received, resulting in a response rate of 85%. It is natural in a Delphi study that some participants will drop out in later rounds for several reasons (Franklin & Hart, 2007). In this study, there was a notice of a drop out of participants across the three rounds, 26% (7 participants) from Round 1 to Round 2 and 15% (3 participants) from Round 2 to Round 3, resulting in an overall drop out of 37% (10 participants) from Round 1 to Round 3. However, the overall number of respondents remained acceptable and in line with other research studies such as Clark (2005), Kantz (2004), and Senyshyn (2002). Source: Pum Chea. (2013). Proposed guidelines for developing primary English teacher training programs for provincial teacher training colleges, Kingdom of Cambodia. Unpublished thesis. Master of Education Program in Teaching English as a Foreign Language, Department of Curriculum and Instruction, Faculty of Education, Chulalongkorn University. Pp.99-102. 104

Case Study 3: A Purposive Sampling Technique Employed in a Survey Research: Intercultural Communication Challenges and Strategies for the Thai Undergraduate Hotel Interns (Anyada Jhaiyanuntana and Nomnian, 2020) This current study was conducted at a hotel belonging to an international college located at a leading university near Bangkok after receiving approval from the hotel manager who was also the lecturer responsible for the three-month-long in-house internship. There were twenty-six participants in this study. Thirteen of them were third-year students and the rest were fourth-year students. They were enrolled in a Bachelor of Business Administration (International Program) in Tourism and Hospitality Management at the in-house hotel training center. The interns had to work in the following departments: food and beverage, front office, kitchen, and housekeeping. Used as a medium of instruction, English has become a lingua franca in Thai higher education due to the increasing demand from ASEAN students to study in international programs in Thailand (Nomnian, 2014c). This internship course was aimed at equipping the interns with experience and the skills needed to work in the hotel’s intercultural context, where communicating with foreign guests, and experiencing intercultural communicative encounters was the norm. Such encounters were most likely to take place at the front office, and in the food and beverage departments. (Source: Anyada Jhaiyanuntana and Nomnian. (2020). Intercultural Communication Challenges and Strategies for the Thai Undergraduate Hotel Interns. PASAA, 59, 204- 235, January - June 2020.) 105

Case Study 4: A Snowball Sampling Technique Employed in a Survey Research: Motivational strategies for English classrooms in Thailand: Perspectives of English teachers (Lihua Wang & Jutarat Vibulphol, 2014) Methodology The present study is a survey study, which aims to investigate English teachers’ perspectives on the importance and the use of motivational strategies in English classrooms at the basic level of education. Population and Participants The population in this study was Thai nationality teachers who were teaching English in basic education English classrooms from grades 1 to 12 in Thailand. The participants were obtained by using a ‘snowball’ sampling strategy (Cheng & Dörnyei, 2007). Eleven English teachers from different basic education schools agreed to be the key informants to find five to six participants in order to obtain the total of 100 participants for the study. The eleven key informants were divided into 2 groups, one group had 6 teachers while the other group had 5. Each key informant then distributed the questionnaires to other English teachers that they knew either from their school or others. The two groups were informants of different topics under investigation—the importance and the frequency of the use of motivational strategies. In total, 74 English teachers from schools in various areas of the country participated in the study. Thirty-eight of them (51%) answered the ‘importance’ questionnaire and the other thirty-six (49%) answered the ‘frequency’ questionnaire. The 74 participants had different teaching experience and background, fourteen (19%) were male teachers, fifty-six (76%) were female teachers (four participants did not identify their gender on the questionnaire). The number of teaching years ranged from 1 month to 35 years. In terms of the experiences in other countries, twelve (16%) of them had studied abroad. Concerning teaching area, most of the participants were from Bangkok (62%), the rest were from the Central region (23%), North (8%), Northeast (4%) and West (1%) regions of Thailand. 106

(Source: Lihua Wang & Jutarat Vibulphol. (2014). Motivational strategies for English classrooms in Thailand: Perspectives of English teachers. An Online Journal of Education,10 (1). 2015, 429-443.) Case Study 5: A Stratified Sampling Technique Employed in a Survey Research: How good are we at knowing what learners like? (Spratt, M., 1999). The study involved 997 students studying `service English' and 50 English Department teachers. Details of the subjects are as follows: Details of the student sample The percentage of each category of student was representative of those studying service English (Table 1). At the time of the study, students within the service program were following English courses which differed from department to department in terms of course length, assessment procedures, degree of course specificity and specific syllabus objectives. Table 1 Student sample Level Year Total % of whole student population for each level Higher Diploma 1 236 17 Undergraduate 1 450 21 Undergraduate 2 221 16 Undergraduate 3 90 25 However, within this varying framework, all first-year students were studying `English for Academic Purposes', all second-year students `English in the Workplace', and third year students both of these at a more advanced level. The student sample included 553 females and 379 males (65 participants did not provide information on 107

gender). The students came from 17 different departments in Hong Kong Polytechnic University, and the number of students from each department was representative of their participation in the English service program. Details of teacher sample All service English teachers in the English Department were invited to participate in the study. Fifty of them (50%) did so. These teachers were from Hong Kong, China, Australia, Britain and the USA. They were all trained as language teachers to at least RSA/UCLES Diploma level or equivalent, while varying in the particular amount and kind of training they had received. Their age range was between 25 and 60 and their experience of teaching English in the service English program at Hong Kong Polytechnic University ranged from 2 to 20 years, averaging 6 years. The teachers taught English to a range of departments, i.e., they did not specialize in anyone. (Source: Spratt, M. (1999).How good are we at knowing what learners like? System, 27 (1999) 141-155.) Case Study 6: A Cluster sampling Techniques Employed in a Survey Research: Teachers’ preference toward and needs of ICT use in (Nurdin Noni, Riny Jefri & Nasrullah, 2017) RESEARCH METHOD This research used a descriptive method. There are four aspects to describe; namely, the profile of ICT tools and senior secondary school teachers of English; teachers’ preferences for using ICT in ELT; implementation of ICT in ELT; and teachers’ needs in ICT use to facilitate their teaching. The sample for the research consisted of English teachers at public senior secondary school in South Sulawesi. The sample taken using cluster sampling technique consisted of 46 teachers from six districts. 108

The instrument used was a questionnaire divided into several parts; namely, questions or statements about the profile of ICT tools at schools and English; teachers’ preferences on ICT use in ELT; implementation of ICT use in supporting ELT delivery; and needs in relation to the ICT facilities. The second, third, and fourth parts of the questionnaire used a Likert scale, which provided five options for each question or statement. The first part used varied scales. The data collection was conducted by distributing the questionnaire to the targeted sample of respondents by visiting all sample schools in the six districts. In this data collection, the researcher was assisted by other research team members. To anticipate misunderstanding by the respondents about the questions or statements of the questionnaire, the team were requested to be available at the sites. This research used descriptive analysis. The respondents’ answers were firstly scored according to a Likert scale. These raw scores were then converted into the range of 20-100, which was classified into six categories; namely, very high, high, fair, slightly low, low and very low. The converted scores were further analyzed using frequency, percentage and mean analysis. The results of the analysis were either tabulated or presented in a diagram. (Source: Nurdin Noni, Riny Jefri & Nasrullah (2017). Teachers’ preference toward and needs of ICT use in ELT. Global Journal of Engineering Education, 19 (1), 88-92. Case Study 7: Perceptions of and Experiences with Vocational College English Majors’ Out-of-class English Learning in Taiwan (Ching-Mei Cheng, 2015) Abstract—The purpose of this study was to investigate vocational college EFL students’ experiences with and perceptions of out-of-class English learning. The research involved a survey, comprising three sets of questionnaires concerning learners’ perceptions of and experiences with out-of-class English learning. Four intact classes of 164 EFL vocational college students participated in the study. Eight 109

semi-structured interviews were conducted with eight successful EFL learners. The intentions were to highlight the perspectives of the eight successful English learners with in-depth understandings. The results showed that the successful EFL learners more aggressively and voluntarily participated in their out-of-class English learning because they were highly motivated. To conclude, this study may be of importance for explaining the dynamic relationship between learning outcomes and learners’ autonomy as well as in providing English teachers with a better understanding of how students’ out-of-class learning outcomes relates to their motivation and autonomy. (Source: Ching-Mei Cheng. (2015). Perceptions of and Experiences with Vocational College English Majors’ Out-of-class English Learning in Taiwan. Journal of Language Teaching and Research, 6 (4), 737-748) Case Study Type of sampling the study Justification 1 2 3 4 5 6 7 110

Activity 8: Consider the initial interview questions and how the author report the findings from the articles below • Flowerdew, J. (1999). Problem in writing for scholarly publication in English: The case of Hong Kong. Journal of Second Language Writing 8 (3), 243-264. • Spratt, M. (1999). How good are we at knowing what learners like? System 27, 141- 144. • Flowerdew, J. (1999). Problem in writing for scholarly publication in English: The case of Hong Kong, Journal of Second Language Writing, 8 (3), 243-264. How did the researchers post the questions and how appropriate the questions were prepared to elicit self-report data? Activity 9 Interview task Directions: Below are some of the structured interview questions the students plan to ask the first year M Ed TEFL students about their opinion towards the program. Please help the students edit the questions and design the IOC form to give to the experts to validate the questions. Remember the purpose of this interview is to find out what the students like or dislike about the M Ed TEFL program and the suggestions that the program may need to improve. Structured Interview About Curriculum 1. What are your expectations from the M Ed TEFL program? 2. Do you feel that the required courses will provide you with enough of a foundation to be an educator? If not? What should be included or excluded? 3. Are all four required courses provided practical? 4. Do you think that the teaching practicum should be included in required course? Why? 5. What do you think of the comprehensive exam? What should be included or excluded? 111

6. Do you feel that the selective courses provided in each semester vary and serve your interests? Are there any particular field of interest that you want to suggest to be an elective course? 7. What will you expect from the program expedition? 8. Do you have any difficulties enrolling? About Facilities 1. What do you think about the facilities which are available for you at the present? (e.g., parking, toilet, security, break etc.)  NOT Enough Information. Can you provide more detail?  Enough Information 2. How the classroom can be improved?  NOT Enough Information. What is the difficulty you found?  Enough Information 3. What information should the program provide you more?  Some problems. What seem to be the problems?  Nothing 4. If you were an administrator, what service would you offer to the program? Why?  NOT Enough Information. Could you clarify more?  Enough Information 5. What is your attitude toward the current service?  NOT Enough Information. Could you clarify more?  Enough Information 6. Do you have any other suggestion about the program’s facilities?  NOT Enough Information. Could you clarify more?  Enough Information 112

About Individual Concerns 1. How do you feel toward the personnel team of M Ed TEFL program? 2. Can you clarify a bit when you say that you like/dislike the personnel team? 3. What do you think about the faculty of M Ed TEFL program? 4. In your opinion, are they easy to be approached? Why/why not? 5. Are the staff and the administrators willing to assist you when you ask for help? In what way? 6. Do you think senior friends could help improve your learning experience at M Ed TEFL program? How? 7. What activities do you think could develop the relationship between you and your senior friends? 8. What do you think about your classmates? 9. Can you say something more about your relationships with your classmates? 10. How can the personnel team be improved in order to improve your learning experience at EIL program? Activity 10 Read the following research articles and answer the questions listed below: Article 1: Rosukhon Swatevacharkul & Nida Boonma. (2020). Learner autonomy: Attitudes of graduate students in English language teaching program in Thailand. Language Education and Acquisition Research Network Journal (LEARN), 13(2), 176- 193. Article 2: Tan Bee Tin. (2013). Exploring the development of ‘interest’ learning English as a foreign/second Language. RELC Journal, 44(2), 129-146. 113

Article 3: Karansuda Wararodphaksakul & Sumalee Chinokul. (2013). A study of Thai EFL Teachers’ beliefs, perceptions. and practice of multiple assessments in classroom. An Online Journal of Education (OJED), 8(2), 191-203. Article 4: Watson Todd, R., Pansa, D., Jaturapitakkul, N., Chanchula, N., Pojanapunya, P., Tepsuriwong, S., Towns, S. G., Trakulkasemsuk, W. (2021). Assessment in Thai ELT: What do teachers do, why, and how can practices be improved? LEARN Journal: Language Education and Acquisition Research Network, 14(2), 627-649. Focusing on survey research design • What are research questions in each case? • Where was the survey conducted? • What was the population surveyed? What sampling procedures were used? • How were the data collected? • Were the questions used in the survey well-designed? • How were the data analyzed? If open-ended questions were used, does the researcher describe how they were analyzed and coded? • How were the data presented? If the study included numerical data, how were they reported—by frequency, percentage, and/or mean? If the mean is reported, is the standard deviation reported? • What conclusion were drawn? Are they justified in light of the data presented? • What are the pedagogical implication of the study? Are they warranted in light of the findings of the study? 114

Review Questions Guidelines for using questionnaire • Read Chapter 3 Griffee, D.T. (2012). An Introduction to second research methods design and data • Read the guideline from Colorado State University Survey Research. Writing@CSU. Colorado State University. Available at http://writing.colostate.edu/guides/guide.cfm?guideid=68 and discuss the main issues raised in the book. • Read Brown, J.D. (2001).Using surveys in language programs • Read Dornyei, Z. (2003). Questionnaire in second language research • Read Spada, N.; K. Barkaoui; C. Peters; M. So; A. Valeo. (2009). Developing a questionnaire to investigate second language learners’ preferences for two types of form-focused instruction System, 37,70–81. • Read Block, D. (1998). Exploring interpretations of questionnaire items, System 26, 403-425. • Read references for survey research in Appendix A. Chapter summary Survey research Definition: Uses various data collection procedures to enable the researcher to investigate a construct by asking questions of either fact or opinion (explanatory) from a sample of a population for the purpose of generalizing to the population. Strengths: Descriptive summary Generalized statements based on large databases 115

Providing information for curriculum development and researching certain topics. Weaknesses: Directly measuring learning that has or has not taken place. Providing detailed explanation. Wrong questions can lead to wrong answers. Components: A construct A population of interest A sample A data collection instrument Data elicitation techniques and interpretation: Data from self-report Keys issues to consider when designing a questionnaire 1. construct identified 2. construct sampled adequately 3. cross sectional or longitudinal 4. closed items and open items 5. response format 6. length of questionnaire 7. piloting 8. return rate 9. reliability and validity 10. respondents described 11. translation of L2 to L1 in the questionnaire 12 truthfulness of respondents Keys issues to consider when designing interview questions 116

References Al-Jamal, D.A., & Al-Jamal, G.A. (2014). An Investigation of the Difficulties Faced by EFL Undergraduates in Speaking Skills. English Language Teaching, 7(1), 19- 27. Allami, M., & Elyas, T. (2016). Perceptions of Using Social Media as an ELT Tool among EFL Teachers in the Saudi Context. English Language Teaching, 9(7), 1-9. Anyada, J., & Nomnian, S. (2020). Intercultural Communication Challenges and Strategies for the Thai Undergraduate Hotel Interns. PASAA, 59, 204-235. Bakla, A., Cekic, A., & Koksai, P.O. (2013). Web-based surveys in educational research. International Journal of Academic Research, 5(1), 4-13. Beidernikl, G., & Kerschbaumer, A. (2007). Sampling in Online Surveys. In Rayvolds, R., A., Woods, R., & Baker, J., D. (Eds.), Handbook of research on electronic surveys and measurements (pp. 90-96). London: Idea Group. Bethelem, J.G. (2009). Applied Survey Methods: A Statistical Perspective. Hoboken, NJ: John Wiley & Sons. Block, D. (1998). Exploring interpretations of questionnaire items, System, 26, 403-425. Brian, D. (2020, June 9). How to pick the right survey tool for your COVID-19 context. Research for Evidence. Retrieved August 31, 2021, from https://researchforevidence.fhi360.org/how-to-pick-the-right-survey-tool-for- your-covid-19-context Broca, N. (2015). Questionnaires on L2 Learning and Teaching Practices: Rating Responses on Frequency and Opinions. TESOL Quarterly, 49(2), 429–440. https://doi.org/10.1002/tesq.228 Brown, J. D. (2001). Using surveys in language programs. Cambridge, England: Cambridge University Press. Cheng, H. & Dörnyei, Z. (2007). The use of motivational strategies in language instruction: The case of EFL teaching in Taiwan. Innovation in Language Learning and Teaching, 1(1), 153-174. 117

Ching-Mei, C. (2015). Perceptions of and Experiences with Vocational College English Majors’ Out-of-class English Learning in Taiwan. Journal of Language Teaching and Research, 6(4), 737-748 Clark, A. (2005). Listening to and involving young children: A review of research and practice. Early Child Development and Care, 175(6), 489–505. https://doi.org/10.1080/03004430500131288 Cohen, L., Manion, L., & Morrison, K. (2007). Research Method in Education (6th ed.) [E-book]. Routledge. https://gtu.ge/Agro- Lib/RESEARCH%20METHOD%20COHEN%20ok.pdf Colorado State University. Writing Guides. Writing@CSU. Retrieved August 31, 2021, from http://writing.colostate.edu/guides/guide.cfm?guideid=68 Datacurious. (2008). Datacurious [Computer Software]. Retrieved August 31, 2021, from http://www.datacurious.com/ Dornyei, Z. (2003). Questionnaire in second language research: Construction, Administration, and Processing. Routledge. Dörnyei, Z. (2003). Questionnaires in Second Language Research: Construction, Administration, and Processing. Mahwah N.J.: Lawrence Erlbaum Associates. Edward, A., G., & Lori, W., C. (2004). Sample Questionnaire: Self Esteem Study. Introduction to SPSS for Windows. Arizona State University College of Nursing. Retrieved August 31, 2021, from http://www.public.asu.edu/~eagle/spssintro/esteem.htm Flowerdew, J. (1999). Problem in writing for scholarly publication in English: The case of Hong Kong. Journal of Second Language Writing, 8(3), 243-264. Foreman, E., K. (1991). Survey Sampling Principles. New York, NY: Marcel Dekker. Griffee, D.T. (2012). An Introduction to second research methods design and data [E- book]. TESL-EJ Publications. http://www.tesl- ej.org/pdf/ej60/sl_research_methods.pdf 118

Jansen, K., Corley, K., G., & Jansen, B., J. (2007). E-survey methodology. In Rayvolds, R., A., Woods, R., & Baker, J., D. (Eds.), Handbook of research on electronic surveys and measurements (pp. 1-8). London: Idea Group. Johnson, K., E. (1992). The relationship between teachers’ beliefs and practices during literacy instruction for non-native speakers of English. Journal of Reading Behavior, 24, 83-108. Karansuda, W., & Sumalee, C. (2013). A study of Thai EFL Teachers’ beliefs, perceptions. and practice of multiple assessments in classroom. An Online Journal of Education (OJED), 8(2), 191-203. Levy, P., & Lemeshow, S. (1991). Sampling of Populations: Methods and Applications. New York, Chichester: Wiley J. Lihua, W. & Jutarat, V. (2014). Motivational strategies for English classrooms in Thailand: Perspectives of English teachers. An Online Journal of Education, 10(1). 2015, 429-443. Nomnian, S. (2014). English as the ASEAN lingua franca in Thai higher education. Veridian E-Journal (International Issue), 7(4), 75-84. Nurdin, N., Riny, J., & Nasrullah. (2017). Teachers’ preference toward and needs of ICT use in ELT. Global Journal of Engineering Education, 19(1), 88-92. Nutsareeya, S. (2009). A Study of Thai EFL teachers’ use of media in reading instruction. Unpublished thesis. Master of Education Program in Teaching English as a Foreign Language, Department of Curriculum and Instruction, Faculty of Education, Chulalongkorn University, 25-29. Ornstein, M., D. (2003). A Companion to Survey Research. SAGE Publications Ltd. https://www.doi.org/10.4135/9781473913943 Paul, C., P., Rajiv, J., I-Chant, A., C., Dana, C., L., & Carrie, C. (2014). Overview of Survey Research. Research Methods in Psychology (3th American ed.). Pressbooks. https://opentext.wsu.edu/carriecuttler/chapter/7-1-overview-of-survey- research/ 119

Prof. Nikki Hozack. (2014, September 17). 01 Sampling What is Sampling [Video]. YouTube. https://www.youtube.com/watch?v=Gs-gLeYuDZw Pum, C. (2013). Proposed guidelines for developing primary English teacher training programs for provincial teacher training colleges, Kingdom of Cambodia. Unpublished thesis. Master of Education Program in Teaching English as a Foreign Language, Department of Curriculum and Instruction, Faculty of Education, Chulalongkorn University, 99-102. Rice University. (2021). Data, Sampling, and Variation in Data and Sampling. Introductory Business Statistics. Openstax. Retrieved August 31, 2021, from https://openstax.org/books/introductory-business-statistics/pages/1-2-data- sampling-and-variation-in-data-and-sampling Rosukhon, S. & Nida, B. (2020). Learner autonomy: Attitudes of graduate students in English language teaching program in Thailand. Language Education and Acquisition Research Network Journal (LEARN), 13(2), 176-193. Simple Learning Pro. (2015, November 26). Types of Sampling Methods (4.1) [Video]. YouTube. https://www.youtube.com/watch?v=pTuj57uXWlk Spada, N., Barkaoui, K., Peters, C., So, M., Valeo. A. (2009). Developing a questionnaire to investigate second language learners’ preferences for two types of form- focused instruction System, 37, 70–81. Spratt, M. (1999). How good are we at knowing what learners like? System, 27, 141- 155. Tan, B., T. (2013). Exploring the development of ‘interest’ learning English as a foreign/second Language. RELC Journal, 44(2), 129-146. University of Missouri–St. Louis. (n.d.). Population and Samplings. Retrieved August 31, 2021, from https://www.umsl.edu/~lindquists/sample.html 120

Watson, R., T., Pansa, D., Jaturapitakkul, N., Chanchula, N., Pojanapunya, P., Tepsuriwong, S., Towns, S., G., Trakulkasemsuk, W. (2021). Assessment in Thai ELT: What do teachers do, why, and how can practices be improved? LEARN Journal: Language Education and Acquisition Research Network, 14(2), 627- 649. Yamane, Taro. (1967). Statistics: An Introductory Analysis (2nd ed.). New York: Harper 121

Chapter 4: Experimental Research Introduction This chapter introduces the experimental research. One of the popular types of research being conducted in schools as teachers want to study about the effects of their designed instruction, often called intervention. The readers will learn and read case studies of various types of experimental research design employed in EFL context and data elicitation techniques often used to collect data from the treatment group to claim the results of their learning. 1) What is experimental research? The experimental design is a blueprint of research procedure that the researcher controls all factors that may influence the outcome of an experiment by attempting to determine or predict what may occur. Typical empirical research design would try to claim the cause-effect dilemma by doing something special to the samples and measure their progress, compare the results with data obtained from another group or what they could make before the treatment begins. ▪ The question of using two groups treatment and controlled group: Are they comparable? ▪ The question of using a single group: How to claim that they make progress at a significant level? Characteristics of experimental research • It allows the researcher to control the situation where the researchers can answer the question, “What causes something to occur?” 122

• It permits the researcher to identify cause and effect relationships between variables and to distinguish placebo effects from treatment effects. • It supports the ability to limit alternative explanations and to infer direct causal relationships in the study. • The approach provides the highest level of evidence for single studies. Challenges of experimental research • The design is artificial, and results may not generalize well to the real world. • The artificial settings of experiments may alter the behaviors or responses of participants. • It can be costly if special equipment or facilities are needed. • Some research problems cannot be studied using an experiment because of ethical or technical reasons. • It is difficult to apply ethnographic and other qualitative methods to experimentally designed studies. Experimental design and non-experimental design Experimental design Non-experimental design 1. True experimental = random 1. Descriptive = provide a summary of assignment of subjects to different groups an existing phenomenon by using • Every subject has an equal numbers to characterize individuals or chance of being in each group. a group. 2. Comparative = take descriptive • It enables the researcher to conclude that the results are not studies a step further. It investigates if there are differences (or similarities) due to differences in characteristics of the subjects. between groups (e.g., whether language aptitude differed by gender). 123

Experimental design Non-experimental design 2. Quasi-experimental design 3. Correlational = examine approximates the true experimental relationships between two or more type. phenomenon. • The purpose is the same • A positive correlation = high determining cause/effect plus values of variable A are manipulating conditions. associated with high values of variable B. • However, there is NO random assignment of subjects. • A negative correlation = high values of variable A are 3. Single-subject = similar to quasi- associated with low values of experimental research, there is direct manipulation but no random variable B. assignment. 4. Survey = select a sample of subjects • In many situations, it is and administers a questionnaire or impossible or inconvenient (time conducts interviews to collect data. + budget) to study entire groups • Surveys are used frequently in of subjects. educational research to describe • Furthermore, the researcher may various types of information. be interested in one or two subjects, not large groups of • The research is designed so that subjects. information about a large number of people (= population) can be inferred from the responses from a smaller group of subjects (= sample) 5. Ex Post facto = explore possible causal relationships among variables that cannot be manipulated by the researcher. (Because the possible causes are studied AFTER they have occurred; rather than manipulate what WILL HAPPEN to subjects, as in experimental design.) 124

1. The research design and the design notation Research design can be seen as a structure of research that connects all of the elements in a research project together. A design is often described using a concise notation that enables us to summarize a complex design structure efficiently. The elements of a design include: ➢ Observations or Measures These are symbolized by an ‘O’ in design notation. ➢ Treatments or Programs These are symbolized with an ‘X’ in design notations. ➢ Groups The major types of assignment are: ▪ R = random assignment ▪ N = nonequivalent groups ▪ C = assignment by cutoff ➢ Time Time moves from left to right. Elements that are listed on the left occur before elements that are listed on the right. Common research design and research notation Research Design Research Notation Randomized treatment Pretest Treatment Posttest and control groups with pretest and posttest R O1 X O2 R O3 O4 R = Randomized, that is, subjects are randomly assigned to the treatment group O = Observation or testing X = Treatment 125

Research Design Research Notation Randomized treatment Treatment Posttest and control groups R X O2 posttest only R O4 Solomon four-group Pretest Treatment Posttest design R O1 X O2 R O3 O4 R X O6 R O8 Randomized immediate Pretest Treatment Immediate 3 and delayed posttest Posttest months posttest Two experimental treatment groups and R O1 X O2 one control group with randomization and R O3 O4 posttesting only R X O6 R O8 Treatment Posttest R X1 O2 R X2 O4 R O6 126

Experimental and control Pretest Treatment Posttest pretest and posttest O1 X1 O2 design O3 O4 Research Design Research Notation Experimental and control Treatment Posttest posttest design X O2 O4 One group pretest posttest Pretest Treatment Posttest design O1 X O2 One group posttest only Treatment Posttest design X O2 2. Population, sample and representativeness (Dornyei, 2007) Population ➢ The group of people the study is about. Sample ➢ The group of participants whom the researcher actually examines in an empirical investigation Representativeness ➢ The sample is a subset of the population that is representative of the whole population. 127

➢ The representativeness is crucial as the strength of the conclusions we can draw from the results obtained from a selected small group depends on how accurately the particular sample represents the larger population. Sampling procedures Probability sampling ▪ Random sampling ▪ Stratified random sampling ▪ Systematic sampling ▪ Cluster sampling Non-probability sampling ▪ Quota sampling and dimensional sampling ▪ Snowball sampling ▪ Convenience or opportunity sampling ------------------------------------------------------------------------------------------------------------------------ Scan the following QR codes to study more on research design: Types of Experimental Designs Common Research Designs Exploring Pretest Designs Quasi Experimental Designs 128

3. Guidelines for reading quantitative research (McMillan & Schumacher, 2005) ➢ Research problem or purpose 1. How clearly and sufficiently is the problem or purpose stated? 2. Is it difficultly delimited to be amenable to investigation? Does it have sufficient practical or theoretical value to warrant study? 3. Does it have a rationale? Has the problem been studied before? If so, should this problem be studiesd again? Is the study likely to provide additional knowledge? 4. Will the finding give rise to further hypotheses, thereby increasing the probability of adding to existing knowledge? ➢ Review of literature 1. How adequately has the literature been surveyed? 2. Does the review critically evaluated previous findings and studies, or is it only a summary of what is known? 3. Does the review support the need for studying the problem? 4. Does the review establish a theoretical framework for the problem? 5. Does the review relate previous studies to the research problem? ➢ Hypotheses or Questions 1. Are any assumptions advanced with respect to the hypotheses or questions? 2. Are hypotheses consistent with theory and known facts? 3. Are they testable? 4. Do they provide an expected result? ➢ Results 1. Were statistical techniques needed to analyze the data? If so, were the most appropriate and meaningful statistical techniques employed? 2. Have the results been adequately and clearly presented? 3. Is there reference to practical as well as statistical significance? ➢ Discussion, Implications, Conclusions 129

1. Are the conclusions and generalizations consistent with the findings? What are the implications of the findings? Has the researcher overgeneralized the findings? 2. Does the researcher discuss the limitations of the study? 3. Are there any extraneous factors that might have affected the findings? Have they been considered by the researchers? 4. Are the conclusions presented consistent with theory or known facts? 5. Have the conclusions been presented adequately and discussed? 4. Data elicitation for ESL/EFL research ➢ Data obtained from external sources ➢ Prompted production and responses to obtain data for cognitive processes, capacities and strategies: ▪ The strategy inventory for language learning (SILL), stimulated recall, think-aloud protocols, immediate recalls, aptitude test, working memory tests ➢ Prompted production and responses to obtain data for linguistics-based research: ▪ Elicited imitation, picture description, structured elicitation, story- telling and sentence combining ➢ Prompted production to obtain data for interaction-based research: ▪ Picture-description tasks, spot-the-difference tasks, story completion tasks, map tasks, consensus tasks, consciousness-raising tasks ➢ Prompted production and responses for sociolinguistics and pragmatics-based research: ▪ Narrative such as interviews, silent films, and film strips with minimal sound, role play, discourse completion test (DCT), video playback for interpretation, match guise. 130

Sample of data obtained from external sources (A language test): Sample of tests used in research 1. Explicit negative feedback: Metalinguistic feedback Metalinguistic error correction was operationalized as researcher’s grammatical explanations without providing the correct answer. The following was an example for giving explicit negative feedback: An item taken during giving explicit feedback: Mary________for me when I arrived. a. waited b. is waiting b. was waiting d. waits Learners: Mary waited for me when I arrived. Researcher: You need past continuous here. Learners: Mary was waiting for me when I arrived. 2.Implcit negative feedback: Recast Recast was operationalized as researcher’s reformulation of learners’ error by changing components (subject, verb or object.) The following was a example for giving implicit negative feedback: An item taken during giving implicit feedback: Stephanie __________ German very well a. speak b. speaks b. is speaking d. has spoken Leaners: Stephanie speak German very well. Researcher: Stephanie speaks German very well. Leaners: Stephanie speaks German very well. (Source: Thongthangsai & Wiriyakarun. (2012). 131

Sample of prompted production and responses to obtain data for cognitive processes, capacities and strategies: The strategy inventory for language learning (SILL) Sample of short-term memory test Digit Span Subtest Part 1 Digit Span Forward (DSF) Direction: You are going to hear a group of numbers. After you hear the numbers, you will hear a signal . . . . (a beep signal). When you hear the signal, pick up your pen and write the numbers you heard. Then put down your pen. For example: if you hear the numbers 7-2-4, after the signal write the digits in order 7-2-4. Then put down your pen and wait for the next group of numbers. series 1: 5-8-2 series 2: 6-9-4 series 3: 6-4-3-9 series 4: 7-2-8-6 series 5: 4-2-7-3-1 series 6: 7-5-8-3-6 series 7: 6-1-9-4-7-3 series 8: 3-9-2-4-8-7 series 9: 5-9-1-7-4-2-8 series 10: 4-1-7-9-3-8-6 series 11: 5-8-1-9-2-6-4-7 series 12: 3-8-2-9-5-1-7-4 series 13: 2-7-5-8-6-2-5-8-4 series 14: 7-1-3-9-4-2-5-6-8 132

Part 2 Digit Span Backward (DSB) Direction : Now you will hear some more numbers. But this time after the signal, you will write the numbers backwards or in reverse order. For example: if you hear 7-2-4, after the signal . . . (a beep signal) you will write 4- 2-7 and put down your pen. series 1 : 2-4 series 2: 5-8 series 3: 6-2-9 series 4: 4-1-5 series 5: 3-2-7-9 series 6: 4-9-6-8 series 7: 1-5-2-8-6 series 8: 6-1-8-4-3 series 9: 5-3-9-4-1-8 series 10: 7-2-4-8-5-6 series 11: 8-1-2-9-3-6-5 series 12: 4-7-3-9-1-2-8 series 13: 9-4-3-7-6-2-5-8 series 14: 7-2-8-1-9-6-5-3 End of the task. Thank you. 133

Sample of prompted production to obtain data for interaction-based research: Describing a picture: A family scene Source of picture: freepik How can you describe an ordinary picture in an interesting way? In these activities, you’ll practice saying what you see as well as make up a story about the picture to bring your description alive. Tasks Task 1 - vocabulary from the photograph Task 2 - making the description more interesting Task 3 - summary of the description Task 4 - your turn Map tasks: The two maps below show an island, before and after the construction of some tourists facilities. Summarise the information by selecting and reporting the main features, and make comparisons where relevant. Write at least 150 words. Source: https://www.ieltsadvantage.com/2015/03/24/ielts-writing-task-1-maps-lesson/ 134

Sample of prompted production and responses to obtain data for linguistics- based research: Video playback for interpretation Baby Cinderella - Diapers commercial ad (Molfix) Scan QR code to wath the video 135

Discourse completion test (DCT) Six types of Discourse Completion Task (DCT) (Parvaresh & Tavakoli, 2009) 1. Written Discourse Completion Task (WDCT) • Requires students to read a written description of a situation and then asks them to write what they would say in that situation. This form can be a purely open one (OWDCT) or one made based, for example, on a piece of dramatic research, in which some parts have been removed. 2. Multiple-choice Discourse Completion Task (MDCT) • Requires students to read a written description of a situation and select what would be best to say in that situation. 3. Oral Discourse Completion Task (ODCT) • Requires students to listen to a description of a situation and to say aloud what they would say in that situation. 4. Discourse Role-play Task (DRPT) • Provides a description of a situation and asks students to play a particular role with another person in that situation. 5. Discourse Self-assessment Task (DSAT) • Provides a written description of situation and asks the students to rate their own ability to perform the necessary speech act in that situation 6. Role-play Self-assessment (RPSA) • Combines the DRPT with the DSAT by requiring students to rate their own pragmatics performance in a previously performed role-play that has been recorded on a video recorder (Blum-Kulka, 1982; Brown, 2001) 136

5. Activities Activity 1: Study case studies in ELT and discuss on the research design and justification for using the research design. Case study 1: A Time-series Design with One Pre-experiment and Two Post-experiments: The Effects of an Experimental Training Program for Teachers of Vocational English Using Concentrated Language Encounter Instructional Processes and Reciprocal Peer Teaching Strategies (Rattanavich, 2017). Research Questions 1. Does the tendency, shown by the line graph, indicate better overall use of the teacher’s English language after the experiment? 2. Does the tendency, shown by the line graph, indicate better overall use of the teacher’s English speaking through the use of scaffolding techniques after the experiment? 3. Does the teacher of each vocational English class express mostly positive comments/opinions regarding the CLE learning process using reciprocal peer teaching strategies, as well as students’ learning development and attitudes? 4. Does the tendency, shown by the line graph, indicate overall better student listening and speaking performance in each class after the experiment? 5. Do the students in each experimental class improve their reading and writing performance after the experiment? Methodology Research Design and Sampling A quasi-experimental time-series design was used in the study. T1 X T2 X T3 X = Experimental group treatment T1= Pre-test T2= Post-test 1 T3= Post-test 2 137

Three second-year vocational English teachers in three vocational classes (i.e., sales, print media, and secretarial work) at SIBA College (Bangkok, Thailand) were purposively sampled during the second semester of the 2015 academic year (November, 2015–March, 2016). Learners received 3 teaching hours per unit per week over 4 months. Teachers were pre-tested during the first week of the semester and given one week orientation and training to the experimental treatment. Teachers then had 12 weeks of actual teaching (i.e. 36 teaching hours with three teaching units/class/week). The teachers taught six teaching units before having their first post-test. After the first post-test, they continued with another six teaching units before having the second post-test. ------------------------------------------------------------------------------------ Treatment of the Study After the pre-test, the three sampled teachers were given 1-week of orientation and training on CLE learning processes using reciprocal peer teaching strategies. During this period, the researcher demonstrated the process of drafting lesson plans and the five teaching steps as indicated in section 2.4 of the literature review. The actual teaching of all three teachers through the CLE learning process using reciprocal peer teaching strategies was conducted over the course of 3 hours per week with 36 teaching unit hours of lesson plans tested and approved by the researcher. Post-tests 1 and 2 were administered to the students every six weeks. 138

Case Study 2: A four Solomon research design (Uslu & Çubukçu, 2017) Abstract The aim of this study is to find out the effects of strategies-based reading instruction on the students’ strategy use and reading comprehension level. The study which was conducted with a Solomon four groups experimental design also focused on the long-term impact of the instruction. English Reading Strategies Scale and Reading Comprehension Test were used as the instruments for gathering data. The sample consisted of 111 second class students enrolled in English class at a state college in Turkey. Fifty-seven students comprised the experimental group and fifty-four students served as the control group. Results gathered on the post tests revealed that strategies-based reading instruction increased the students’ strategy use and led to a statistically important improvement in their comprehension level compared with the control group. Also, results gathered on the delayed test revealed that impact of strategies-based reading instruction on the strategy use and comprehension level is maintained. Methodology Research Design This study was designed as an experimental research and the Solomon four groups experimental design was used as the pattern. In Solomon four groups design, the subjects are assigned to two experimental and two control groups randomly. One of the experimental and one of the control groups are pretested on the dependent variable. The next step is to administer the intervention to the experimental groups but not to the control groups. Lastly, each group is posttested on the dependent variable (Kirk, 2009). The aim of the Solomon four groups experimental design is to control the effect of the pretest on the dependent variable. Posttests results of the experimental and control groups are compared. If two experimental groups have the similar results, and the two control groups have the similar results, it means that pretest doesn’t have any effect on the dependent variable. However, if two pretested groups’ results are different from the others, it means that pretest has an effect on the dependent variable (Neuman, 2006). 139

Case study 3: a quasi-experimental design with pretest, immediate posttest and delayed posttest treatment and control groups (Al Ajmi, 2015) Research questions 1) Does written corrective feedback (WCF) help Arabic learners of English improve their accuracy in the use of English prepositions over time? 2) What are these students’ views about the type and effectiveness of WCF on their misuse of English prepositions in their writings? Methodology Research Design To answer the first research question, a quasi-experimental design was used due to the difficulty of randomly assigning the 50 participants into control and treatment groups. In fact, random assignment of students is commonly impossible in educational institutions (Gass, 2010, p. 11; Dörnyei, 2007, p. 117). The number of students in each group was equal, 25 students each. These Arabic speakers’ accuracy in using ten commonly confused English preposition usages (see Appendix 1) was measured over seven weeks by means of pre-test, immediate post-test, and delayed post-test, following the design of Bitchener and Knoch (2008), but with only two groups of participants. Accuracy needs to be tested in the form of pre-test and post-test, and all of the writing is new, not rewrites (Bitchener, 2008; Bruton, 2009; Sheen, 2007; Truscott, 1999, 2007). Furthermore, researchers have highlighted the value of assessing students’ subsequent writings in order to find the effectiveness of WCF on, for example, linguistic features (Bitchener, 2008; Bitchener & Knoch, 2008, 2009; Ellis et al., 2008; Bruton, 2009; Guénette, 2007; Truscott, 2007). To answer the second research question, an open-ended questionnaire was administered to the experimental group. According to Wagner (2010), “Questionnaires can be either closed or open-ended” (p. 27). The rationale behind choosing an open-ended questionnaire for this study was to explore students’ views about the type and effectiveness of WCF on their misuses of English prepositions, and this type of questionnaire can provide rich data and help identify issues that had not previously anticipated (Dörnyei, 2007; Dörnyei & Taguchi, 2010). Research Procedure 140

Quasi-experimental Design Before conducting the study, a letter was written both to the dean of the college and to the head of the English department, asking for permission to conduct the study in the college. After receiving their approvals, a letter was written to the Level B teachers asking permission to use their students for the study. Having the green light from the teachers, the researcher asked the students to fill in a consent form and to sign it. However, before they completed their consent forms, they were briefed about the study, explaining that it would not affect their marks and that their writings would be anonymous. Then, the researcher followed this outlined procedure: 141

Case study 4: A Multiple Sampling Design- Random Cluster Technique and Purposive sampling (Iqbal & Rafi, 2018) Methodology For this research, experimental method was applied and the sample was equally divided into two groups: experimental group and control group. Homogeneity of the sample was ensured so that the results of both groups could be compared. For the teaching purpose, the experimental group was taught through the Access teaching method for which the teachers were properly trained and the control group was taught through traditional method used in public schools across the province of the Punjab. The Access Program was implemented in four cities of the Punjab province i.e. Lahore, Faisalabad, Multan and Sargodha. Population of the study included all the students (N=3,469) who appeared in test to get admission in the Access Program. There were 680 students who secured admission in the Access Program after passing through written and oral tests while remaining 2,789 students could not either qualify or join the program, but continued their regular education at public school. The data were collected from more than one location, so multiple sampling techniques were used to fulfill the needs of the study. Another reason for selecting multiple sampling designs was the difference in the target groups (the Access Program and public schools) of the population which could be catered only through this design. For the Access Program, the researcher used random cluster sampling technique. A male and a female class were randomly selected from each location i.e., Multan, Lahore, Faisalabad and Sargodha. Thus, it included eight classes with 25 students in each class that makes 200 students in total. On the other hand, for public schools, purposive sampling technique was used. Only those students, who took the Access entry test (pre-test) and were offered admission, were included in study. These students either could not join the Access Program or quit it within first quarter (three months) for any reason after joining it. From them, 50 students (gender balanced) from each location i.e., Multan, Lahore, Faisalabad and Sargodha (200 students in total) were selected. This inclusion criterion for public school students was set specifically so that comparison of the Access Program and public school could be justified in terms of greater similarity of sample. 142

Case Study 5: An intact class A Cognitive Framework in Teaching English Simple Present. (Tian, 2015). Methodology All data collection was conducted on the campus of Wuhan No.27 Senior High School. Two intact classes of final year students participated in the study. The teaching and testing were conducted between July and August 2013. During their regularly scheduled class, the pretest was administered to the two classes. And then the treatment sessions followed, and the students were given the posttest immediately after the treatment session. Students in both groups were asked to fill out a questionnaire about the instruction after they took the posttest. Four weeks later, the two groups took the posttest again, along with a similar questionnaire. The researcher was not involved in either the classroom teaching or test paper marking. A middle-school teacher with three years of teaching experience taught both classes and marked all the papers from the experiment. Two intact classes of final-year students in Wuhan No.27 Senior High School participated in this study. Each class consisted of 30 students. One class was the experimental group; the other was the control group. Biodata forms were administered to the students at the beginning of the study. Information pertinent to second language acquisition, such as the students’ age, gender, age when started English learning, length of English study, motivation for learning English, textbooks used, and their self-evaluation of their knowledge of the simple present were elicited. 80% of the students were 17 years old, while the rest were 18 years old. 60% were boys, while 40% were girls. 90% of them had learned English for five years, starting from the first year in junior high school. Only very few of them had learned some English in elementary schools. For more than 85% of them, the motivation for learning English was to pass examinations, for English was a mandatory course in high school. The textbook series they used included the national textbook series published by People’s Education Press, Look, Listen and Learn, and New Concept English. All the students had learned the English simple present before. Almost all of them thought that they had mastered it. Pedagogical Treatments: The author designed two 45-minute lesson plans of a review lesson on English simple present. The first 45-minute session was based on 143