Proceedings of 2019 4th International Conference on Information Technology

2019 4th International Conference on Information Technology (InCIT2019) computer in checking the results saving the proctor’s Online assessment valuable time in checking papers manually at the same time provide clear analytics and feedback on specific Test Bank > Randomizer learning outcome clusters of the learners in terms of Tab Lock strengths and opportunities for improvement [3]. Check B. Conceptual Framework Internet In Forms Distribution Algorithm created by Al Bazar Exam [2], implementing this algorithm will first require the Taker exam maker to generate keys correspondent to the number of the exam takers present. The keys generated are for one Fig. 1. Conceptual Framework time use only which will then be used to select a preset exam form with the same number of questions. This C. Related Literature algorithm guarantees that there will be no exactly the A research done by the International Centre for same set of questions within an examination. Academic Integrity, says that over 70,000 undergraduate In a distribution technique developed by Khaled students 68% of them admitted into committing Suwais, the model considered choosing the exam academic malpractice on examinations and plagiarize on questions randomly [5]. However, the location of the assignments. Another survey was done by the Canadian students is taken into consideration to ensure that no Broadcasting Company on 54 Universities of Canada, students will receive an exam with the same set of declaring that 73% of students admitted on academic questions but needing to address that the same difficulty malpractice in the academic year 2011-2012 [9]. In the level should be looked into given a randomized question U.K., according to Guardian, they discovered an in- set. crease of 42% in academic malpractice acts involving gadgets since 2012. Also, in the U.S. Harvard University, Online examination system sets different admissions a prestigious university had experienced academic and procedure interface for users or examinees with malpractice dishonor on a take-home final exam in which different roles, the users enter into a particular interface, 125 students were accused of working jointly, according and have different set of exam according to different to the school this could prove to be the largest Ivy permissions like set A and set B [6]. The System's League academic malpractice scandal in recent memory. primary objective is providing online examination service for examiners, therefore, the examinee is the heart of the The educational philosophy of playing down the system. scores of examinations pay more thought on knowledge, comprehensive quality and the ability has not yet been On the basis of the course of the preset packet completely touchable in teaching and managing activities filtrating regulation, the moment the system discovered of every department. It brings that a few students explore any unauthorized packets of data, the exam will be quick success and instant gain, learn calmly to cope with stopped and a warning message will appear in the examinations. The examination cannot be used as the examiners' computer or else the computer will be locked only test to recognize and separate the sheep from the rending the examiner to continue the exam [7]. goats [10]. Figure 1 represents the conceptual approach of the Most of the online examination systems spawn exam development of the online examination system with 3 questions automatically are derived by the number of main components namely: A weighted value based items from the question bank following a set condition randomizer that determines unique but same level of that will simply randomize the question [11]. This difficulty and coverage of the exam content limited to structure is effortless to implement, however, it does not objective type of exams; Second a Tab locking system to prevent other browsing and window navigation function during the exam; And an analytics checking function to identify scope of learning outcomes and the corresponding performance result. 127

2019 4th International Conference on Information Technology (InCIT2019) remove the fact that it is inefficient, unable to be algorithm will now considers specific characteristics of backtracked and not very innovative as it is easy to get the exam set by the exam creator. This framework derived into an endless loop. from the focus group discussion is presented in Figure 3 aimed to address the ability to generate unique exams but II. METHODOLOGY measures and assess the same targeted learning outcomes. The study utilized a model guideline from Winston W. One key note also from the Focus Group was the need to Royce in this project as seen in Figure 2. It established be able to sizeable test bank in the consideration of the various phases and techniques which will be applicable in number of exam takers at a given instance on the developing the system. assumption of a similar group. The other target is the Tab The Royce Model generally applies a constant locking feature that allows the website to look the local backward validation test method as such the design is system from navigating from one window or browser tab accepted through it development stage. until exam was completed or terminated. Preparation TEST BANK RANDOMIZER Development -QUESTION -TEST TYPES -ANSWER -TOTAL Validation SCORE & -Correct TIME Modification -Option -TARGETED -OUTCOME OUTCOMES -WEIGHTS -WEIGHTS & -Difficult RATIO -SCORE -SINGLE/ -TIME UNIQUE Evaluation RESULTING EXAM Fig. 2. Royce model Fig. 3. Concept Exam Generator Preparation The development phase of the concept exam generator in Figure 3 under the Test bank procedure looks into the This stage is the most crucial part of the project followings fields created in the database. The fields because it gives the proponents a direction on the include the question of the item with field on correct development of the project. The proponents will answer having alternate option of selection in case for investigate on what programming language and database matching type exam. The question field included the design will be used in developing the project. A focus following four fields that will give more decision group approach was used in interviewing exam creators characteristic to be used by the randomizer namely: the and understanding assessment goals and methods. outcome cluster on topic the question is grouped, the level Another aspect of the focus group then looked into the of difficulty and importance in weighted percentage with exam proctors whom on the field are more familiar with its corresponding score if answered correctly and time the occurrence and conduct cases of academic needed to ideally complete and answer said question. malpractice. Development Validation In the development stage, following the data's In this stage, the framework of the proposed project gathered, the proponents will focus on the preparations and its segments were analyzed in order to guarantee that and plans. Writing the codes is the most laborious and the procedures are running under the expected conditions. consumes a great amount of time of the project since it is In conclusion of the series of de-bugging and testing, the a web-based system. After writing the codes the database proponents arrived in their presumption and considered design will be the major part of the project. Lastly, de- whether a couple of modifications are necessary. One key signing the assessment of the system plays a part in the observation was the limited size of the test bank database development stage. General targets in the development to further determine the ratio and extent of exam stages looked in embedding a weight system in the test uniqueness of the randomizer on the assumption of large bank and an outcomes cluster field as such the randomizer classes taking the exam. The Ratio of test bank size and 128

2019 4th International Conference on Information Technology (InCIT2019) against the size of class taking the exam at a given time referenced to Figure 3. The outcome choices will have was set a limitation of the study. fields on its percentage weight of difficulty in terms of overall average difficulty of the exam and the ratio of the Modification difficult to average questions in the generated question. It is also set in this process on whether the exam questions The proponents accumulated some information by to be generated are identical to all or unique. way of consultations. Additionally, the proponents decided to include several modifications to the proposed The proctor is also capable to import an exam from a project by adding a Comma separated values (CSV) ex- CSV file in the Set exam process and that helps the traction. The proctor can import an examination by proctor to save time instead of manually encoding the extracting it from the CSV file. The proctor can then questions and choices into the system. The proctor can adjust to how the randomization will perform. also select and assign the weighted fields of the questions in the CSV file. Generally, the weights look into a Likert Evaluation scale of 1 to 5 with 5 being highest or difficult. From this point an averaging approach was used to validate the The evaluation was done based on looking into a parts randomized selection of the questions as such two items approach in reviewing expected outcomes at a per are then checked on exam were generated. For example, development process. As for the final outcome a parts proctor has set difficult ratio to 10% with overall approach is also applied in terms of evaluating the difficulty to a score of 2. The program will cross check technical components through test scripts and the learning the randomized exam generated on whether such will or exam component through user live testing of students qualify by determining the number of question weight of and educators by observing and gathering survey 4 above against 3 below and should meet 10% and the feedback. overall average of the weight should +/- 1 scale from 2. III. RESULTING SYSTEM In the perspective of the examinee, Figure 5 illustrates the flow of the system from the point of view of the The process flow proposed in Figure 4 illustrates the examiner. flow of the system from the perspective of the proctor. Login Validate View Enter Exam Login Validate Select Take Exam Select Logout View Logout Send Query Submit Check Select View Fig. 4. Proctor Process Flow Figure 5 Examiner Process Flow Figure 4 also illustrates the proctors’ guideline in The examinee’s only requirements in using the system using the system. The proctor can set how many questions is an account and an exam key provided by the proctor. in the exam should be randomized and create a question The examiner can send a feedback to the proctor if there in 3 types namely Multiple Choice, Identification and are errors in the given exam. Assessment is automatically Matching Type. After that, the proctor can set the time generated after submitting. and period to complete the exam. The exam key is automatically generated based on how many examiners The developed system also allows the proctor to are taking the exam. During the Select Exam process the manually lock the exam to all the examiners or unlock the proctor will also set the randomizer setting based on the exam. The system based on the Tab Locking process [8] scope of outcomes the exams that will be generated. As automatically locks the examinees terminal once it opened a new tab or minimizes the screen and it will appear in the proctors' terminal that the said examiner has forfeited the exam because of at-tempted academic 129

2019 4th International Conference on Information Technology (InCIT2019) malpractice. From this the examinee can send a query to remembered, and that allows straightforward operations proctor to access this manual feature if deemed allowable. for the supervision that enabled it to attain an overall interpretation of Excellent with an average of 4.76. IV. FINDINGS TABLE IV. EVALUATION OF THE RESPONDENTS ON THE The system is tested through a test script and the data “EFFICIENCY” CRITERIA collected were analyzed based on the survey conducted with 100 College student respondents. The processed data Efficiency x̅ Verbal Interpretation is shown in Table 1 to Table 5. Time behavior 4.62 Excellent Total 4.62 Excellent TABLE I. EVALUATION OF THE RESPONDENTS ON THE “FUNCTIONALITY” CRITERIA In terms of Efficiency as measured in Response time Functionality x̅ Verbal Interpretation performance as seen in Table 4, even though it is subjective and conditional due to the location of the Purpose 4.75 Excellent infrastructure of the Internet it was tested. Overall the Suitability 4.80 Excellent result was excellent. Accurateness 4.82 Excellent Interoperability 4.77 Excellent TABLE V. SUMMARY OF THE EVALUATION OF THE PROPOSED 4.79 Excellent SYSTEM ONLINE EX-AMINATION SYSTEM WITH CHEATING PREVENTION Total USING QUESTION BANK RANDOMIZATION AND TAB LOCKING The criteria on functionality as presented in Table 1, Criteria x̅ Verbal Interpretation looked into whether the system has served it intended Functionality 4.79 Excellent function which allows examinees to take an exam online 4.77 Excellent with features embedded that deters academic malpractice Reliability for objective based exam questions. The respondents Usability 4.76 Excellent generally agree with a score of 4.79 that such features like Efficiency 4.62 Excellent unique exams per student knowingly will deter the 4.74 Excellent temptation of communicating with other students for Total answers or scope in taking the online exam. Based on the numbers from Table 5 overall the TABLE II. EVALUATION OF THE RESPONDENTS ON THE proposed system got an Excellent result on all the criteria. “RELIABILITY” CRITERIA The respondents or testers have made Reliability x̅ Verbal Interpretation recommendations and insights on how to further improve Free from errors 4.77 Excellent the project. They were all satisfied about the system that Excellent addresses academic malpractice, but further modifications Recoverable 4.73 Excellent should be needed to make the system more reliable. Those Compliant Excellent are mentioned in the recommendations part. 4.82 Total As shown in the table the Efficiency part has achieved 4.77 the “lowest” remark due to connectivity, a clarification feedback and suggestion was also noted that on a situation As indicated in Table 2, the Reliability of the that the examinees have tried to submit an unfinished intended system based on the evaluation respondents exam. They suggested that there should be a notification has attained an overall average of 4.77 with an excellent that pops out if the examinee has not answered all the remark. This shows that the system is accurate and precise questions before submitting in order to aid in efficiency in and that there are no bugs and errors found during the answering the exam. testing phase. In relation to the qualitative feedback of the user test. TABLE III. EVALUATION OF THE RESPONDENTS ON THE The developed project was described as an expanding and “USABILITY” CRITERIA comprehensive study. The proponents have analyzed 100 feedback remarks the respondents and also the Usability x̅ Verbal Interpretation suggestions of the invited panel of professors who Understandability 4.78 Excellent evaluated the process flow of the online exam generator. 4.76 Excellent Based on their statement they were satisfied with the Learnability 4.74 Excellent systems feature and functions but it still needs some Operability 4.76 Excellent alterations to make the system invulnerable to other forms academic malpractice namely physical sources of Total information beyond scope of the computer and its access to the internet and also developing feature for subjective The evaluation results on Table 3 on Usability pointed based exam questions.. out that the intended system is easy to handle, and that the functions are effortless to perform and easily 130

2019 4th International Conference on Information Technology (InCIT2019) The functionality part is where they were mostly crowd based behavior as easy question may become impressed as the tab locking will prevent the examiner to difficult and difficult questions may become hard based open a search engine or minimize the tab of the screen. on the feedback of the professor respondents. Currently, The professors also gave praise about the CSV the exams as it is just randomized based on fixed weights importation as it will save them a lot of time, because the with the difficulty level of the questions for each system currently used in the school has to manually examinee is assumed un-changed. encode questions and answers to create an exam. The professors also noted the auto exam generator based on a Include a Gaze Tracking into the system that uses the randomizer with weights which allows them to generate camera that lets the proctor observe the examinees, and if unique exams but still measure similar outcomes as the examinee tries to look at their seatmates the Gaze compared to traditional randomized exams which has Tracking feature will detect it and automatically locks the limitations that there is a probability that an examinee will examinee's exam and automatically notifies the proctor have an exam that has the difficult questions and another that someone has attempted academic malpractice. examinee has the probability that the exam questions he/she receives may have an easier exam question being The examinee assessment should be improved by addressed. On the same thought, a qualitative feedback adding a feature that lets the proctor discern if the from processors noted that even though the randomizer examinees' performance in the class is improving by claimed to balance out the exam difficulty level at the comparing their scores from the first quiz until the last same time being unique sets of exam questions. It may quiz of the class, and also from prelim to finals. The not necessary be perceived by the examinees to be fair as system should be able to generate a statistical graph and they may perceive the exam they took was either difficult charts that help the proctor to determine what term or or easy as it is dependent on the amount of preparation quiz where the students/examiners performed badly. they have done prior to taking the exam. The suggestions above were all provided by the V. CONCLUSION testers of the system, the panel of professors and educators of the project. The Online Examination System with cheating Prevention using Question Bank Randomization and Tab REFERENCES Locking has achieved a remark of excellent in all categories. In terms of the functionality, reliability, [1] Baohe, W. (2009),”Web-based examination system design and usability and efficiency, based on the remarks of 100 implemen-tation” Tianjin: Nankai University. College student respondents, they were all impressed about its functionality that addresses academic [2] Hussein Al Bazar. (2017), “Forms Distribution Algorithm for malpractice. Online Examination Systems” In the functionality part there were no bugs and errors [3] Singh, S. (2017). What are the benefits of online exams? Re- during the testing phase thus receiving an excellent trieved from https://www.quora.com/What-are-the-benefits-of- remark was achieved. The system was flexible as it can online-exams import an exam from a CSV file that according to the professors, it saves a lot of time in creating an exam. [4] Barkley, A. (2001). AN ANALYSIS OF ONLINE EXAMI- Excellent as appraised in terms of reliability, the NATIONS IN COLLEGE COURSES. Retrieved from proponents have made the software free from bugs and https://ageconsearch.umn.edu/record/36049/files/sp01ba02. errors. It is also excellent in terms of usability; All the following steps to use the system are easy to remember [5] A. Fayyoumi1, and K. Suwais “Online Exam Questions Dis- and if the user does not have an idea to use the system tribution Technique based on Terminals Locations: The Case of they can just refer to the test scripts to be able to Arab Open University”, Journal of Computations & Mod-elling, familiarize them about the flow of the system. Excellent vol.5, 2015. in terms of efficiency, the system works and gives immediate feedback in assessing the examiners. For this [6] Bai Yi-chen. The Design and Implementation of the Online reason, the system can be used to be deployed on Examination System Based on MVC Model [D].Shandong educational institutions that want to have an online University, 2012. examination system that has the ability to address academic malpractice. [7] Wu Wei, Wei xiao, Wei Shimin. Research of defending online- test cheating based on manager server [J]. Computer Engi-neering VI. RECOMMENDATION and Design, vol 8, pp. 1941-1943, 2007. Adding a feature that rates the weight of the questions [8] Smetters, D. (2013). Respondus LockDown Browser, As- through analytics as the exam as it is used through time. sessment tools for learning systems, October 2013. Retrieved This will allow the weights to normalized based on a from https://www.respondus.com/products/lockdown-browser/ [9] Canadian Broadcasting Company. (2006). Campus Cheater. Retrieved from https://www.cbc.ca/manitoba/features/universities/ [10] Wenbin, Z. (2007). \"Discuss on the normalized and scientific administration of college examination,\" Contemporary Edu-cation Forum, No. 1, Jan. 2007, pp. 20-21. [11] Zhang Yong-sheng, Feng Xiu-mei, Bao Ai-qin. The Research and Design of Online Examination System. 2015 7th Interna-tional Conference on Information Technology in Medicine and Education 131

2019 4th International Conference on Information Technology (InCIT2019) Design and Evaluation of Interactive Learning Story and User Interface Prototyping for Mobile Responsive Learning Application Theerakarn Phunkaew Chanakarn Phandan Charoenchai Wongwatkit School of Information Technology School of Information Technology School of Information Technology Mae Fah Luang University Mae Fah Luang University Mae Fah Luang University Chiang Rai, Thailand Chiang Rai, Thailand Chiang Rai, Thailand [email protected] [email protected] [email protected] Abstract—With the limitations of instructional design, it and encourage the students to earn more understanding and provides negative learning outcomes to the students. To drive their attention longer. overcome this issue, it is essential to design learning more interactive with relevant scenarios and story. Besides that, To increase students’ attention, user interface (UI) is developing a user interface for mobile responsive system one important thing to apply in this project. It explains the requires a careful consideration and users’ feedback. In this usability and purpose of the app. The features and interface study, the authors not only proposed the overall framework together make the user experience (UX) of an app, and it is of interactive learning story, but also conducted a simple the core value proposition. Without a great UI, the user evaluation to determine its quality for learning. In order to experience amounts to nothing [4]. UX is important further development, the user interface designed in this because it tries to fulfill the user’s needs. It aims to provide project follows the flat design concept and responsive mobile positive experiences that keep users loyal to the product or design. To understand the actual learning experience, this brand. Additionally, a meaningful user experience allows prototyping was continuously tested for improvement. The you to define customer journeys on your website that are findings of this study are used for the next phase of system most conducive to business success [5]. development and validation. Therefore, it is challenging to tackle the mentioned Keywords—interactive learning design, system prototype, challenges. This study aims to propose a learning story online learning system, mobile responsive application, which is designed to be associated with learning objectives information system and learning contents. Moreover, the mobile user interface is prototyped correspondingly. In order to make these I. INTRODUCTION useful designs, it is necessary to earn constructive feedbacks from the users through the means of conducting Before teaching students, a teacher has to prepare user evaluation. The results of this paper become helpful in instructional materials which are shaped by a subject finalizing the initial process of this project before actual purpose or a destination as what the purpose is, what they development. are gotten, what ways they have to learn, what activity drives them to understand substance easily. Since this II. RELATED STUDY issue, learning design is a process which can guide the teacher. It is a framework to design various ways, A. Analysis, Design, Develop, Implement, Evaluation activities, and interactions of student learning experiences (ADDIE) as subject levels and subject components via learning task, There are many methodologies to create training learning resource and learning support to develop the learning experience for students [1]. The learning design materials in an eLearning world. The ADDIE model is one focuses on a teaching-learning process, which is in a of those methodologies. It is a creating process of lesson, unit, and course. Moreover, an e-Learning media is instructional materials and performance support tools [6]. It a great supported learning and teaching to implement a is settled by system approach which is agreed usually in learning process [2] with designing for user interface and which design and develop lessons by a computer such as e- development. Learning. A closed system is a primary process to consider from the result in evaluation stage then bring information Interactive storytelling is a tool to help communication to examine and improve it depends on feedbacks. among people understanding easily that make them agree and view at the same point. It is a privileged application of ADDIE model can be processed in 5 stages; Analysis: virtual technology to support a story generation and define instructional problems and understand learning issue intervene user anytime by character-based interactive, including student learning experience and learning which uses hierarchical task network planning techniques environment, Design: design systematic as design learning [3]. It visualizes students see throughout the whole of the purpose, learning measuring, teaching method and strategy, contents with storytelling through a character motion, a and specific designing as design courseware, design counter conversation, a stimulation situation, and a flowchart and storyboard, and screen design, Development: surrounding environment. To continue learning easily and create contents, learning story, documentation, and develop conducted linking contents, the story will be separated instructional material program, Implementation: train from a large image to many small sections. During teachers and students how to use a system, Evaluation: learning, many subsections are linked together to support evaluate formative in measure each stage of ADDIE process and summative evaluation in evaluate about 132

2019 4th International Conference on Information Technology (InCIT2019) specific criteria referenced item and get users’ feedbacks results show that a search in flat text mode (compared with [7]. the traditional mode) is associated with higher cognitive load. A search for flat icons takes twice as long as for Many projects applied ADDIE in many ways to realistic icons and is also characterized by higher cognitive improve their instructional materials. Such as, in Saudi load. Identifying clickable objects on flat web pages Arabia, they used the ADDIE to build the CPD program require more time and is characterized by a significantly effectively and useful structure. A Feedback of ADDIE higher number of errors. Our results suggest replacing the helps the physicians improve CPD program’s future flat style user interfaces with interfaces based on the design iteration to develop physicians knowledge and expertise principles developed over decades of research and practice [6]. To improve current teaching system, Middle Eastern of HCI and usability engineering [16]. Country University also uses ADDIE to design instructional materials as an online-based learning system C. Mobile Responsive Website Design to help teachers manage and prepare to teach their students. Responsive web design means making websites that From the result in the first three stages, they generated many themes in the system and the next step to construct it can adapt to the size of the visitor’s viewport. The goal is [8]. for content to render differently depending on the device or B. Mobile User Interface Design screen size so that visitors have an optimal experience no matter how they access a website. The primarily benefit of If you want to design an excellent functional mobile responsive web design is that sites load quickly without app interface, design principles are hugely important. any distortions, so users do not need to resize anything to Because design principles can stick to improve the quality view content manually [17]. of a user interface design. To increase the chances of success when create user interfaces, most designers have to Three main principles drive responsive design that follow interface design principles due to interface design binds all responsive sites. First is the Fluid Grid Systems, principles, represent the high-level concepts that are used fluid grids are grid systems that scale based on the user’s to guide software design [10-11]. screen as opposed to fixed-width layouts that always appear the same. While the term is sometimes used The element that is used to design user interface is very synonymously with “liquid layouts,” fluid grids ensure that important, such as the transition and motion help make the all elements resize with one another (Target size/context = user interface expressive and easy to be used and the relative size (960/1280 = 75%)). As browsers have gotten motion provides timely feedback with the status of user narrower, new challenges have arisen most modern actions, in which the motion makes it clear when the items browsers support CSS3 media queries, which enable are selected [12]. In designing user interface, infusing websites to collect data from individual visitors and colors into the app interface that is in harmony with the conditionally apply CSS styles. The min-width media function and the user experience is a challenging task, feature allows designers to implement specific CSS styles because every tiny bit of color that is used in the app once the browser window falls below a specified width. should have a purpose and the needed behind it [13]. Last flexible Images an easy option is to use CSS’s max- Furthermore, an element of navigation is the act of moving width property, which ensures that images load in their between screens of an app (between two screens in your original size unless the viewport is narrower than the app) to complete tasks. It has enabled through several image’s width [17-18]. means: dedicated navigation components, embedding navigation behavior into content, and platform affordances As time pass by, more and more people surf the [14]. The component/element/shape is to provide a way for Internet by using mobile devices compared to a desktop users to recognize components and identify the different computer. Recently, mobile device and computer screen material surfaces such as shapes that help users to identify designers have been trying to provide users with qualified the components and the effects of how usable they are. web-browsing. Therefore, there is a need to switch to Mobile interaction is how the users interact with mobile- Responsive Web Design, which is capable of reshaping based on wearable computers with appropriate interaction itself depending on various screen sizes and resolutions devices; this reflects the complicated nature of an from the largest screen sizes to smallest on mobile devices. individual's interaction with a computer system, more of all Thus, the users will be exposed to the best experience with it includes factors such as an understanding of the user and a content visual display on the device or platform that they the task that the user wants to perform with the system, are viewing it on. This is more significant when users are understanding of the design tools, and an understanding of studying on instructional websites and pages not to software engineering tools [ 15]. decrease their concentration, motivation, and performance on their study since the responsive web design In the past few years,’ flat user interface design has automatically changes the page layout, resize the images or become the predominating visual style of operating crop them proportionally [19]. The most popular social systems, websites, and mobile apps. Although HCI and networks are fluid responsive such as the social network, usability experts have widely criticized the flat design, links, buttons, and email forms are mostly fluid responsive empirical research on flat design is still scarce. HCI present web design. Now fluid websites are somewhat the results of an experimental comparative study of visual underrepresented today, but they are the future of web search effectiveness on traditional and flat designs. The design, but creating a responsive website is a complex following types of visual search tasks were examined: (1) process, and costs certainly more than a simple website. search for a target word in text; (2) search for a target icon [20]. in a matrix of icons; (3) search for clickable objects on webpages. Time and accuracy parameters of the visual For this project fluid grids are used as the mobile search, as well as oculomotor activity, were measured. The responsive website design, as following the fluid grid are grid systems that scale based on the user’s screen as 133

2019 4th International Conference on Information Technology (InCIT2019) opposed to fixed-width layouts that always appear the prompt, and get feedback. Iterative learning by this loop- same. While the term is sometimes used synonymously sequence until ending chapter, do a quiz to overview their with “liquid layouts,” fluid grids ensure that all elements understanding and knowledge. All of these lead to connect resize concerning one another. with learning objectives and outcomes that they will receive after learning. III. INTERACTIVE LEARNING DESIGN ADDIE is a framework in this project to be a guideline Fig. 2. Examples of interactive story association. and shape a large scale to small, which is needed only in this work. Start with analyzing a current student’s learning From the example of interactive story association in environment and designing content in text, infographic, figure 2 demonstrates the learning sequence of unit 1 in and animation in form of storytelling interactive with the chapter 1. Start with study about using a computer in daily student’s experience. Next, transforms all design to actual life and their computer problems by reading content and output then implements it to students along with evaluating watching the animation. After that, they will face with the proposed system and get their feedback to develop and practice or prompt in next learning sequence which asks improve it continuously. them to discover their daily routine life about the wrong A. Learning Design working of the computer, and then they will get the To design each learning materials should focus on the feedback to react with their answer — this unit associated purpose that students will receive from a system. Linking with learning objective one about computer component and with Learning objective can set a learning story and content its working. The outcomes which student will reserve are for students. For this system links with Computing Science all three dimensions as literacy, prevention and protection, purpose especially in Digital Literacy with 3-dimension and awareness. outcomes. These outcomes lead students to recognize their behavior in real life. Learning design can be divided into IV. INTERACTIVE PROTOTYPING DEVELOPMENT six chapters learned by students. All of these mentioned User Interface and User Experience (UI/UX) has so produces learning chapters to make students more many aspects that highlighting the underlying core understand what they are going to learn, and they also get functionality while sharing designs with team members can purposes directly and outcomes usefully in daily life. be a struggle, thus, making it difficult for stakeholders to comment and give feedback. With prototyping, you can Fig. 1. Learning design framework. collect reviews at every stage of developing the product, whether adding new features or redesigning parts From figure 1, Learning design is separated into two of the product. Test what is working for the audience and parts: Learning Objectives are divided into three objectives what is not [21]. as objective 1 (O1), objective 2 (O2), objective 3 (O3). A. Mobile UI with Flat Design Moreover, learning outcome has three dimensions as Flat design is a type of design that is emphasizing literacy (L), prevention and protection (P), and awareness minimalistic style, the use of color, and clean typography. (A). In each objective has all three dimensions being in The flat design draws the attention of the user to the form of 6 chapter-stories, which are linked with learning content itself. It minimizes the design so that the user can purpose and outcomes. For example, if students study in focus on one section at a time without being distracted; chapter 1 about safe area with a safety wall, they will learn each element in flat design have a 2D appearance allowing about safety in the computer, website, and privacy settings. each element to shine in its own right. Flat design using an During studying, their outcomes have to increase in 3 icon to describe the service mean it has a simple element dimensions and link with O1 and O3. which the message is clear and easy to understand. Flat B. Interactive Story Association design has all the key attribute that makes the design as functional as it is beautiful [22]. According to design the instructional materials like contents, infographics and animations. These are created in the learning process pattern sequence. Starts with content, practice/prompt, feedback, and quiz at the end of each chapter. Students start learning with reading text in content and infographic, watching the animation, do practice or 134

2019 4th International Conference on Information Technology (InCIT2019) Fig. 3. Illustration of flat design on mobile UI. values will be helping with the responsive design, because on very large screens, the content will be stretch and As shown in the figure, we can see that the design is perfect center, but not to larger than 1000px.. following the flat design template, by using an icon for the C. Interactive Prototyping message the user though the icon is creating with the 2D appearance so that each one of them can shine its right with An interactive prototype is a semi-functional layout that lower shadow and no light. The design use color that is can give a high-fidelity preview of the actual app or clean and can draw the attention of the user itself by using website user interface (front-end) functionality. While the a bright color theme and the design minimizes so that the prototype might not have full functionality, it generally user can focus on one section at a time. gives customers and end-users the ability to click around B. Mobile Responsive UI the elements of the interface and simulate the way the app will work [24]. In this project we use the Adobe XD CC to The design of the mobile user interface is design as the create the prototyping, the Adobe XD can be animated and principle of responsive design. For example, the design can be integrated with after effects CC for more advanced uses the breakpoints principle to allow the layout to change options, and apps can be designed using voice and speech at predefined points, i.e., having 3 columns on a desktop, playback. Also, what users create for smart assistants can but only 1 column on a mobile device, uses the max and be previewed and prototypes. [25]. min values to have a width of 100% and max width of 1000px would mean that content will fill the screen, but Fig. 5. Illustration of interactive prototyping wireframe. don't go over 1000px to have a beautiful design. From the figure, we can see the wireframe which is a The flexible and optional grid consists of 12 columns layout of a web page that demonstrates what interface and can be nested. The spacing between the columns is elements will exist on the key pages. It is a critical part of 0.25 rem (one-quarter of the basic type size). The the interaction design process. Which in every prototype maximum width is determined for each project. The the wireframe will be the design of the real content after minimum spacing to the left and right-hand edge of the finish with wireframe, the wireflow will take place next, viewport is 4%. On large screens, the content is center. The wireflow is the representation of screen flow, by putting spacing between images and tiles is narrow and is the same together a set of related wireframes following the order as the column spacing; the horizontal spacing of the text they appear in the flow. The use of decision (shape) in a and other elements from tiles and images is significantly wireflow makes it possible to present multiple navigation more abundant [23]. paths in a single flow. Fig. 4. Illustration of mobile responsive UI. V. EVALUATION AND RESULTS A. Interactive Learning Design Analysis As shown in the figure, we can see the amount of column that has divided into 12 columns and the minimum By 4th stage and 5th stage from ADDIE, conducting spacing to the left and right-hand edge of the viewport is 4 the User Experience (UX) via interviewing with 5 Chiang % off, because sometimes it's great that content takes up Rai students in different grade and sex: 2 males (one from the whole width of a screen, like on a mobile device, but grade 7 and another one from grade 8) and three females having the same content stretching to the whole width of (one from grade 8 and others from grade 7). Firstly, big-screen often makes less sense to the max and min introduce about what a system does and train them to interact with the system then record one-by-one with Open Broadcaster Software (OBS) which is open source software for video recording and live streaming. Measuring with evaluation form in 6 dimensions in wording from their feeling, opinion, and experience after using the system and scoring in an overall system with different dimensions such as content, connectedness, practice, feedback, animation, and graphics. After complete recording, turn back to listen again to record time which they consume in each page from their recording in the spreadsheet of Google. Split recording into two parts of a different sex: male and female. Include input students’ experience wording and students’ score 135

2019 4th International Conference on Information Technology (InCIT2019) provided to this system. After that calculate by measuring B. User Evaluation for Interactive Prototyping in Mean (M) and Standard Deviation (SD) to find the The evaluation testing takes the role of grade 7 and middle pint and space from the Mean point in each dimension, then analyze and summarize their comment. grade 9 student at Chiang Rai, the testing is by using the The result from the evaluation of 6 dimensions is shown in mobile phone connect with the Open Broadcaster Software table 1; male students give all dimensions’ scores at a high (OBS) to record the user testing emotion and body reaction level. When female students give scores at high levels in of the user while playing with the application. The connectedness, feedback, animation, and graphics and evaluation form has six dimensions, such as the UI design others are medium level. + layout, Transition motion, component +element, navigation, and interaction of the application. TABLE I. USER TESTING RESULTS ON INTERACTIVE LEARNING DESIGN From the overall questionnaires, user testing gives feedback from the question in each dimension. Ordering Dimension Male Female and analyses the result and summarize to view the overall M Meaning M Meaning result from the user testing in each dimension: con and pro. Content 3.75 High 3.33 Medium Most of the user is satisfied with the interactive prototyping Presentation design with the component element design and the color of Connectedness 4.50 High 3.50 High the prototype that gives user testing more interest, but some Practice/Prom 4.25 High 3.33 Medium are needing to be improved for a better solution such as pt navigation of the prototype. Feedback 4.00 High 4.00 High Animation 3.75 High 4.00 High TABLE III. USER EVALUATION RESULTS ON INTERACTIVE Graphic 4.50 High 4.33 High PROTOTYPING From students’ comment by their experiences can be Dimension Male Female concluded and summarized in table 2. Ordering and M Meaning M Meaning analyzing their wording, transforming into formal wording UI design/Layout 4.00 High 4.00 High and summarizing again to view overall result from them in Transition/Motion each dimension: negative and positive. Most of them Color 3.67 High 3.67 High usually provide their experience quite same result in many Components/Elements 4.67 High 3.00 Medium dimensions like content presentation which is rather hard Navigation especially in computer contents, connectedness in each Interaction 4.33 High 3.33 Medium page is smooth and link together by Momo character’s 3.33 Medium 3.00 Medium wording. The graphic is easy to understand and clear visualization. In other hands, they wanted to more beautiful 3.67 High 3.00 Medium motion in Animation and sound is not pulling their feeling. After evaluation with the user testing, following by the overall questionnaires base on the 6 dimensions. In the questionnaires will ask the user testing in every dimension. TABLE IV. QUALITATIVE FEEDBACK ON INTERACTIVE PROTOTYPING TABLE II. QUALITATIVE FEEDBACK ON INTERACTIVE LEARNING DESIGN Dimensio Feedback Dimension Feedback n Content + Layout of the page is clearly present itself . Presentation + Mostly easy vocabulary UI + Layout position is good easy to see and play. Connectedness - Quite hard content understanding design/La - Text font should be bigger. yout + Flow of each page is smooth and the time of the Practice/Prompt + Link between contents and pages smoothly Transition transition are perfect. Feedback + They know what they are going to learn in each /Motion - Need more interest transition. page + Color of the app make the user feel very good. Animation - Effect is not needed Color - Want more decoration with the app. Graphic + Re-learning in their mind again automatically + Shape that make the icon are perfect with the interface. + Having not much during learning Compone - Some of the icon are too big and need some text to - More explain clearly in questions nts/Eleme make the icon more clear nts + User know where to click next and know what are the + Ready to follow the feedback to re-learning if Navigatio element in the layout are for their answers are wrong n - Some pages the navigation are not clear is it need to + Re-think again what they have already learned take time to find the solution before Interactio - Need a short cut button to go next and back the app - Some words in page is not correct n + Interaction of the app make the user feel interesting. - The interaction of the app has many clicking for the + Suitable time to learning and watching user. - They want more beautiful motion - Sound is quite rough From the overall questionnaires, the feedback from + Most images are understanding easily each dimension have con and pro, the con and pro are + Most of them are unity telling the good and bad part that the user likes and dislike - Some image can be capture from real item to from the overall questionnaires. In this part, the result can make clearer visual and easy to understand take the role of providing how the user interface design can be improving more to be better and answer the need for From students’ feedback conclusion and their user testing. evaluation can be described that this system has some points to be improved mainly Animation about graphic and sound. The overall system is the most positive feedback in 6 dimensions and has less negative feedback. For negative feedback is the key to develop all instructional materials continue to make more profit to them. For positive feedback, remain and improve to be better more. 136

2019 4th International Conference on Information Technology (InCIT2019) VI. DISCUSSION AND CONCLUSIONS ACKNOWLEDGMENT A. Concluding Remark The author would like to acknowledge the generous In this paper, the authors constructed instructional support of School of Information Technology, Mae Fah Luang University. learning materials for students grade 7-9 in Thailand by using ADDIE with the content of Digital Literacy in REFERENCES Computing Science, which is a new subject for students. Start from analyzing problems, design materials, and [1] “Learning Design - The Project.” [Online]. Available: develop, then implement and get the evaluation and feedback to turn materials back to develop again to http://www.learningdesigns.uow.edu.au/project/learn_design.htm. improve them better. The combination between learning design and interactive storytelling is quickly tooling to help [Accessed: 22-May-2019]. in design learning story and content before delivery to the developers to do user interface in wireframe and wire flow [2] “Differences between Educational Design, Learning Design and by Mobile User Interface Design easily and less edit layout about the content position to be indicated in mobile and Instructional Design | Instructional Design Australia.” [Online]. web responsive screen via Mobile Responsive Website Design. Available: http://instructionaldesign.com.au/content/differences- B. Results Discussion between-educational-design-learning-design-and-instructional- The estimation shows that male students prefer this design. [Accessed: 22-May-2019]. system and have more satisfactory than females. This means that background experience and sex are factors that [3] M. Cavazza, F. Charles, and S. J. Mead, “Character-Based affect the learning process and learning ability. Such as male students can study about technology contents better Interactive Storytelling,” IEEE Intell. Syst., vol. 17, no. 4, pp. 17–24, than females because male students play the game more than females. So they can use the system smoothly and 2002. rapidly understand the interaction. In part of sexuality expression, when they have any question about the system, [4] cmarix, “Mobile App UI Design For The Future: 7 Principles To females rarely ask to differ with males – they ask anything and go forward uninterruptedly. However, in the form of Follow,” 3-10-2018, 2018. . learning content, females learning rapidly and answer correct more than males. This means that the learning [5] Evan, “UX Design: The Importance of User Experience - content of females is better than males. Rocket55,” 03-02-2018, 2018. . From the result, most of the users are satisfied with the interactive prototyping design, but most of all male student [6] R. K. Almomen, D. Kaufman, H. Alotaibi, N. A. Al-Rowais, M. prefer the design more than the female student, which it will affect the design base on the designing prototype that Albeik, and S. M. Albattal, “Applying the ADDIE—Analysis, is needed to respond both of the genders. In part of interest, the female student will not be engaging a lot on the design Design, Development, Implementation and Evaluation— of element, color, icon or a small decoration different from a male student, and they will be more detail on how can it Instructional Design Model to Continuing Professional Development be more. for Primary Care Physicians in Saudi Arabia,” Int. J. Clin. Med., vol. C. Future Studies To continue improve the instructional materials such as 07, no. 08, pp. 538–546, 2016. content, infographic, and animation to link with learning [7] “ADDIE Model - InstructionalDesign.org.” [Online].Available: experience and learning environment of students and link with learning purpose in each unit should be improve https://www.instructionaldesign.org/models/addie/. [Accessed: 01- wording in content and select the natural word to make more understand, improve infographic to clear more visual, May-2019]. and improve animation about graphic to be more beautiful and sounder has more emotion and feeling. More concern [8] A. H. Al-Bulushi and S. S. Ismail, “Developing an Online Pre- about friendly wording and text for the age of the student and learn more about how to produce the animation to service Student Teaching System Using ADDIE Approach in a satisfy them to drive their attention longer and also entertain them as well. Middle Eastern University,” Theory Pract. Lang. Stud., vol. 7, no. 2, From the feedbacks of the overall questionnaires, make p. 96, Feb. 2017. the interactive prototyping development in the process of modifying the user interface prototype better to [9] “Using Course Guide Book (Computing Science).” [Online]. development and improvement in the future. Moreover, all the design is modified to respond to users’ needs and has Available: http://oho.ipst.ac.th/cs-curriculum-teacher-guide/. still been in the condition of the flat design with the responsive web design. [Accessed: 03-May-2019]. [10]Nick Babich, “The 4 Golden Rules of UI Design | Adobe Blog,” 02- 07-2018, 2018. [11]Eric Bieller, “How To Design A Mobile App User Interface Like A Pro,” 08-22-2018, 2018. [12]MATERIAL DESIGN, “Understanding motion - Material Design.” . [13]WIKIBOOKS, “title=System software: User interfaces - Wikibooks, open books for an open world,” 30.4.2019, 2019. [14]MARERIAL DESIGN, “Understanding navigation - Material Design.” [15]A. Zimmermann, N. Henze, X. Righetti, and E. Rukzio, “Mobile Interaction with the Real World,” in Proceedings of the 11th International Conference on Human-Computer Interaction with Mobile Devices and Services - MobileHCI ’09, 2009, p. 1. [16]I. Burmistrov, T. Zlokazova, A. Izmalkova, and A. Leonova, “Flat Design vs Traditional Design: Comparative Experimental Study,” 2015. [17]cody arsenault, “10 Essential Benefits of Responsive Web Design - KeyCDN,” 09-16-2017, 2017. [18]Interaction design foundation, “Responsive Design – Let the Device Do the Work | Interaction Design Foundation,” 2018. [19]M. H. Baturay and M. Birtane, “Responsive Web Design: A New Type of Design for Web-based Instructional Content,” Procedia - Soc. Behav. Sci., vol. 106, pp. 2275–2279, Dec. 2013. [20]N. Subić and T. Krunić, “The Higher Education Technical School of Professional Studies in Novi Sad,” 2014. [21]Ruchie Goyale, “Why is prototyping important? – zipBoard,” 08-02- 2017, 2017. [22]canva, “Design trend: How to create a flat design in 3 easy steps – Learn.” [23]“User Interface / Responsive UI.” [24]Corporate Blog, “The Main Steps of The User Interface Design | Intetics,” 03-22-2017, 2017. [25]Adobe, “What is Adobe XD? | Adobe XD tutorials,” 05-14-2018, 2018. 137

2019 4th International Conference on Information Technology (InCIT2019) A Heat Map Generation to Visualize Engagement in Classes Using Moodle Learning Logs Konomu DOBASHI Curtis P. HO Catherine P FULFORD Faculty of Modern Chinese Studies Learning Design and Technology Learning Design and Technology Aichi University College of Education College of Education Nagoya, Japan University of Hawaiʻi at Mānoa University of Hawaiʻi at Mānoa [email protected] Honolulu, U.S.A. Honolulu, U.S.A. Meng-Fen Grace Lin [email protected] [email protected] Learning Design and Technology College of Education University of Hawaiʻi at Mānoa Honolulu, U.S.A. [email protected] Abstract— In this study, course materials, mainly in the form In recent years, applications such as course management of PDF files, were uploaded to the Moodle learning management systems (CMS), learning management systems (LMS) and e- system and face-to-face lessons were conducted. Student books have progressed rapidly. Generally, since these clickstreams of the course materials and the results of quizzes systems have a questionnaire survey function, faculty were accumulated. The quizzes, which used a 5-alternative members can easily perform traditional action research and format, were prepared from the course materials. Data from the immediately produce necessary data for analysis. In addition course material clickstreams and the results of the quizzes were to this questionnaire function, various user data are recorded. analyzed from an educational data mining and learning For example, in Moodle, the date and time when access to analytics perspective. A scatter chart and heat map were created course materials was initiated, the name of the user who to classify students into four groups based on deviations from logged in to the lesson, the name of the learning material the average value of clickstreams and quiz scores. The groups viewed, the IP address of the accessing device, etc., are were composed of (1) those with high clickstream and high quiz recorded in the system. These metrics are accumulated so that scores, (2) those with low clickstream but high quiz scores, (3) administrators and the teachers in charge of a course have those with low clickstream and low quiz scores, and (4) those easy and immediate access [2]. with high clickstream but low quiz scores. It is shown that creating a heat map based on an analysis of the results of 13 In this paper, we propose a method to detect the level of quizzes and student clickstreams can provide an early student engagement in learning on a weekly basis using indication of student engagement and can be used to signal the Moodle log data from face-to-face blended-type lessons need for a corrective intervention in the case of disengaged taught by the author to about 50 students. The collection of students. this type of data enables the use of data mining to discover problems and produce new findings leading to lesson Keywords— learning analytics, clickstream, course material, improvement [3]. Here, based on the number of clickstreams quiz score, heat map of the course material and student quiz results, deviations were determined. Based on the number of clickstreams of the I. INTRODUCTION course material link and the quiz result, the deviation was obtained, it was divided into four groups, and a heat map was Educational data mining is closely related to action created and discussed. The clickstream here is the number of research, which has a rather long history. Action research in clicks on the Moodle course material links only, not all mouse education typically takes the form of a research study that clicks. gathers data to review a teacher-created lesson with the goal of developing or improving the teacher’s instructional The four established groups included (1) students with a methods [1]. To this end, the teacher plans and carries out high clickstream and high quiz scores, (2) students with a low data collection in order to analyze the lesson and confirm the clickstream but high quiz scores, (3) students with a low educational effect. Ideally, the teacher then considers the clickstream and low quiz scores, and (4) students with a high analytic results and reflects on her/his teaching method based clickstream but low quiz scores. Especially in Group 3, the on these results. Analysis of the learning behavior of students lower part of the group appears to include students who have in the class can take into account a variety of factors, such as learning difficulties. We analyze the results of 13 quizzes and attendance status and quizzes. Anything related to what show that this analysis can lead to the discovery of teachers teach and learners learn can become the subject of disengaged students. A discussion of how these results can be action research. In addition, because surveys on teaching used as the basis for student guidance follows. methods and teaching practices are fundamental, the results of action research are extremely important as they are II. RELATED RESEARCH considered to be strongly related to class evaluations. Indeed, a class evaluation questionnaire conducted at many As a result of educational data mining and learning universities is considered to be a type of action research analytics, it is possible to analyze log data accumulated in conducted by the entire organization. This method is also one LMS or e-book systems. It is expected that various statistical of the more effective ways to assess student reactions. relationships are hidden in such data. For example, there is 138

2019 4th International Conference on Information Technology (InCIT2019) likely to be a significant correlation between a student’s low by combining quiz scores and the number of times the course material clickstream and the student’s performance on student browsed the teaching material. The online learning the final exam. With proper analytics, it is possible to log accumulated in an LMS serves as an observation variable, investigate which period of time in a student’s course and the learning log itself can be treated as a feature value for material browsing history has the most influence on final test data analysis. score. Studies applying deep learning to the course material clickstream in order to predict student results at the end of the To date, learning logs for a large number of learners have term, as well as studies using in-class course material been accumulated, such as in the case of MOOCs. Many clickstreams to identify students not engaged in the current methods have been proposed for using the learning logs to lesson, have also begun to appear [4]. find outliers that correspond to disengaged students, and many data mining methods have been suggested for using The accumulation of data regarding various transaction learning logs to uncover important learning insights. Typical types in various systems continues to grow. Applying data data mining methods include correlation rules, decision trees, mining technology to these data has the potential to uncover support vector machines, Bayesian networks, clustering, new findings that were not previously apparent. GIMO, outlier detection, information theory approaches, text mining developed by Mazza and Milani, is a Moodle plug-in capable and the like. Here, we propose a method to visualize of analyzing the history of access to Moodle's course disengaged learners by creating a heat map using teaching materials and quizzes for lecture-type lessons [5]. GIMO material clickstreams and quiz scores. allows the user to visualize results for the class as a whole or for individual students using a table heat map and a B. Online materials and Moodle course logs histogram. As described above, an LMS in face-to-face classes can be used in a support role to enhance the Utilizing the log data accumulated in an LMS makes it interaction between teachers and students, acquire data possible to analyze the data from a perspective that differs related to student learning, and help students learn more from that of the action research that has been pursued to this effectively. As such, it is important to make this kind of point. It is expected that the analytical results produced here functionality available to teachers. will be used primarily for class improvement. A number of studies to detect outliers have been Because the education field is vast, student populations conducted in the field of education. Pytlarz et al., for and class offerings are quite diverse. In this paper, learning example, sought a way to identify students at an early stage log data were collected for an \"Introduction to Social Data who are in a potentially dangerous learning situation. Based Analysis\" class at Aichi University in Japan. (The class was on the traffic volume of the campus network, they proposed under the supervision of the author.) Students in this class are a GPA for attendance and teaching methods as well as expected to have already learned the basics of Word and learning time outside of class, and proposed several models Excel prior to enrollment and each student is expected to have to predict student success [6]. In a study of Massive Open the ability to operate a personal computer. First-year through Online Courses (MOOCs), Gitinabard and colleagues made fourth-year students are eligible for enrollment. In the class predictions of dropouts using student access of teaching discussed in this study (Fall 2017), 62.7% of the students materials and forum logs. They were able to quickly identify were males. The approximate age range of the students was learners who were at risk of becoming unsuccessful and from 18 to 22 years. The course is an introduction to statistics showed that their approach was useful for early learner using Excel. The 15-week class begins with learning Excel’s intervention and guidance [7]. basic statistical operations, representative values, variance, standard deviation, simulation, frequency distributions and III. STUDENTS DATA pivot tables, attribute correlation, covariance, correlation analysis and regression analysis. A. Online classes and outlier detection During each classroom session, students open various In an educational context, the purpose of outlier detection course materials according to the teacher’s instruction and is to detect disengaged learners as early as possible and to use the materials to learn about data processing with Excel. prevent them from dropping out or failing out of classes. In In the first half of the class, while viewing the course this study, disengaged learners were detected for materials on Moodle, students will simultaneously open an simultaneous classes in a blended class using Moodle and Excel screen on their personal computers. In the second half online course materials. There are several possible factors for of the lesson, assigned Excel exercises are completed by the a learner to fall into a disengaged state, but it is assumed that students, who are then asked to submit a work file of the an outlier occurs when (1) the learner fails to open the course completed exercises. Each pair of students is provided with a material as instructed by the teacher, and (2) the learner fails monitor that shows course materials and allows the teacher in to read the course materials or is unable to understand the charge to demonstrate course materials and computer contents of the reading. operations. Students can also open course materials from the Moodle screen on a classroom computer and browse them If these conditions continue, student quiz scores will tend freely on their own. to be low. In this sense, such conditions are regarded as abnormal behavior that presents a stumbling block to learning The course material learning log is automatically and is likely to lead to the occurrence of outliers. With respect recorded when a student logs in to Moodle, clicks the table of to (1) above, detection is possible through an examination of contents on the entry page, and browses the course material. a student’s course material browsing frequency based on the During the four months of the class, course materials from student’s course material clickstream data accumulated in Moodle were available to students 24 hours a day, so that the Moodle. Regarding (2), it is possible to identify the learner students could browse the material outside of class hours, who opens the teaching material but whose quiz scores are whenever they were connected to the Internet. In the first 139

2019 4th International Conference on Information Technology (InCIT2019) lesson of the semester, students self-registered with Moodle assessments, this makes it easy to analyze Moodle data from in the classroom and participated in the lesson. During this a whole class or individual student perspective. initial session, the teacher explained how to read the course materials on Moodle and described how the student learning In attempting to create a pivot table using time series data logs would be recorded. From this point on, Moodle collected recorded in minutes and seconds, as in the case of Moodle's the course log data of each student. course logs, a huge cross table is often generated, making it difficult to view the results on the screen of a personal The course material learning log data used for this paper computer. To address this problem, various time increments were collected during the fall semester of 2017 over a period such as month, day, hour, minute, etc., were used in of 139 days, from September 20, 2017 to February 5, 2018. preprocessing the data. The time data were discretized at The learning logs were downloaded from Moodle in an Excel predetermined time intervals according to necessity, and format; in preprocessing, values were calculated with an multiple time series were generated. Excel macro developed by the author. As a result, when performing time series analysis, various C. Aggregation of course material learning logs time categories can be flexibly selected, allowing the user to analyze by the minute, hour, day, week, half-year, or year. In this paper, we examined the number of times that Further, it is possible to narrow the data to be processed. By course materials were opened, together with student quiz establishing multiple time categories for multifaceted results, in order to develop a supporting function to identify analysis, it is possible to display multiple timelines on the potentially unsuccessful students. The quiz questions were pivot table and use them as a filter function. created appropriately based on the instructional design. Quiz results were easily obtainable using Moodle's quiz feature. For example, it is possible to use a combination of time However, some preprocessing was necessary for aggregating runs of 15 minutes and 1 minute and display some of the 15- the number of times that a student accessed the course minute intervals minute-by-minute. By using the filter material. The log data accumulated in the current version of function of a pivot table and selecting multiple time Moodle are only provided in a list format of time series text categories, aggregation and analysis in various time zones data; some additional ingenuity was required to make become possible, and the possibility of performing more effective use of the data. Insofar as LMS log data tend to be detailed and multifaceted analysis increases. “big data,” with values accumulating every year, research on data mining methods for discovering useful knowledge and Here, the course material clickstream corresponding to trends from such “big data” is ongoing [8]. the question range of a quiz is the number of times that the course material was opened from the end of the previous As noted, original Moodle logs are displayed only as time week’s quiz to the end of the current week’s quiz. series data in text format. As such, they provide only a rough picture of the classroom situation and are generally Fig. 1. An example of a TSCS(Time Series Cross Section) table with insufficient to capture the state of students. Excel macros multiple timelines. Student names have been anonymized for privacy have been developed that download and use Moodle course protection (Introduction to Social Data Analysis, 10/18/2017-10/25/2017. logs in Excel or CSV format. Dierenfeld and Merceron show Cell B2 is a filter for Month; B4 to AT9 are hierarchical multiple timelines various kinds of learning analytics methods using Excel pivot for “Date,” “Hour,” “30minutes,” “15minutes,” and “Minute.” Excel screen tables [9]. Konstantinidis has also developed Excel macros to copy.). process Moodle logs in order to analyze page views and overall usage [10]. IV. FOUR GROUPS OF LEARNERS In this study, the teacher in charge typically instructs Moodle's course material clickstreams record when and students to open various course materials during each class. how course materials were opened by teachers and students; Students can also browse course materials on Moodle freely, log items and data are summarized in a time series in the form as the materials were open to all during the class period. of a list, and can be downloaded and used. An example is Quizzes were presumed to be appropriately prepared from given in Figure 1 (details are summarized in a separate paper) lesson course materials and linked with the contents of the [3]. course materials. The degree of the engagement by students for a given lesson was tried to examine by the number of Excel's pivot table function is commonly used as a tool times the related course material was accessed. Quiz scores for aggregating discrete data and creating a two-dimensional were used to assess the degree of student comprehension of frequency cross-tabulation table. By using an Excel pivot the contents of the lesson. It was assumed that there is a table, it is possible to simultaneously create a frequency correlation between the number of times a student opened the distribution for multiple discrete data values and generate a course materials and the student’s proper learning of the two-dimensional cross table. The Python library also has the course content, as reflected by the student’s final test score. ability to create a pivot table. However, the user interface of Excel is more appealing and various processes can be easily added after an Excel pivot table is created. When there are many items that can be counted, as in Moodle course logs, we can effectively use a pivot table to select multiple items and conduct cross-tabulations. The pivot table has a variety of useful functions. For example, columns and rows of items can be readily exchanged and filters can be used to narrow the analytical focus after the initial table is generated. In the case of classroom/student 140

2019 4th International Conference on Information Technology (InCIT2019) A positive correlation was assumed to mean that the student For example, the student's reading of the course materials was properly opened the course materials and understood the erroneous, or the student’s understanding of the contents of content of the course. Conversely, in the case of a negative the lesson was insufficient, or the contents of the course correlation, several explanations were assumed possible. materials themselves were inadequate, and so on. To detect outliers, we first produced a scatter chart by computing the deviation from average for both the number of times a student opened course material and the student’s quiz score (Figure 2). We then divided the students into four groups based on the results (Figure 3). Here, deviation is computed as the observed value, , minus the average, ̅. That is, Fig. 2. Scatter chart example for the third week (October 4, 2017. Student Based on these deviations, the students were divided into the 20, as described in Figure 3, appears in the lower right) following four groups: A B CDE F A. Group QI log duration quiz day Course material viewings and quiz scores are both above the class average. The indication is that the opening of course 9/20/17 materials affects quiz scores for this group. Students who consistently remain in this group are expected to have good 10/4/17 10/4/17 results and tackle their classes effectively. clickstream quiz score devi.click devi.score quadrant B. Group QII Student01 51 4 -6.2 -2.95 QIII Course material viewings are lower than average, but quiz scores are higher than average. Students belonging to this Student02 65 6 7.8 -0.95 QIV group tend to have higher quiz scores without frequently Student03 opening course materials. The expectation is that these Student04 60 6 2.8 -0.95 QIV students have previously learned material similar to the content of the course, but it is also assumed that the material 47 8 -10.2 1.05 QII is not read simply. Student05 61 10 3.8 3.05 QI C. Group QIII Student06 32 6 -25.2 -0.95 QIII Course material viewings are lower than average; quiz scores are also lower than average. Members of this group are Student07 62 10 4.8 3.05 QI assumed to have a tendency to be uninterested in the contents of the lesson, as this group has fewer course material Student08 37 6 -20.2 -0.95 QIII openings than the average and quiz scores that are relatively low. It is expected that some of these students will fall into Student09 56 8 -1.2 1.05 QII the category of unsuccessful students. Student10 37 8 -20.2 1.05 QII D. Group QIV Student11 56 6 -1.2 -0.95 QIII Course material viewings are above average, but quiz scores Student12 are lower than average. This group includes students who Student13 66 4 8.8 -2.95 QIV have not read the course materials even though they opened them, students who are unable to understand the topic even 48 8 -9.2 1.05 QII though they read the materials, etc. It is expected that international students and those who are not interested in the Student14 50 10 -7.2 3.05 QII class, as well as Group QIII students, fall into this group. Student15 Student16 63 4 5.8 -2.95 QIV V. RESULT AND DISCUSSION 46 6 -11.2 -0.95 QIII A. Heat map generation Student17 64 4 6.8 -2.95 QIV A scatter chart was created by using the deviations in course material views in column D of Fig. 3 and the Student18 59 10 1.8 3.05 QI deviations in quiz score in column E. Group QI members are Student19 identified by the blue color in column F; Group QII members Student20 54 8 -3.2 1.05 QII are indicated by light blue; Group QIII is red; and Group QIV is pink. Since the scatter chart is created using the average 96 0 38.8 -6.95 QIV values and deviations of viewings and quizzes, the data always divide into four groups (i.e., quadrants); however, Student21 44 2 -13.2 -4.95 QIII there can be differences between the groups formed each Student22 week. The maximum difference in average scores between 56 10 -1.2 3.05 QII groups during 2017 was 0.512 for Group QI; the minimum Student23 72 10 14.8 3.05 QI Student24 37 4 -20.2 -2.95 QIII Student25 41 8 -16.2 1.05 QII Student26 67 10 9.8 3.05 QI Student27 48 6 -9.2 -0.95 QIII Student28 48 2 -9.2 -4.95 QIII Student29 73 6 15.8 -0.95 QIV Student30 59 6 1.8 -0.95 QIV Student31 Student32 78 8 20.8 1.05 QI 68 8 10.8 1.05 QI Student33 69 8 11.8 1.05 QI Student34 62 10 4.8 3.05 QI Student35 56 8 -1.2 1.05 QII Student36 67 8 9.8 1.05 QI Student37 51 8 -6.2 1.05 QII Student38 64 8 6.8 1.05 QI Student39 68 8 10.8 1.05 QI Student40 50 8 -7.2 1.05 QII AVERAGE 57.200 6.950 STDEV.S 12.696 2.480 Data 40 40 Fig. 3. Deviation calculation results and a heat map example (Week 3, October 4, 2017) 141

2019 4th International Conference on Information Technology (InCIT2019) difference was 0.063 for Group QIV. It can be seen in Figure The heat map in Figure 5 was created from Figure 4. It 3 that Student 20 is the apparent outlier in the fourth quadrant shows results after sorting by cell value (color). The numbers of Figure 2. in column A indicate the number of students. Column N shows results for the final exam (fifteenth week). Here, it is Figure 4 connects the heat maps for the 12 lessons possible to visualize the proportion of students in each of the conducted from the third week to the fifteenth week of the various groups. In this case, for the final exam, the term. It provides the means to visualize the approach of percentages are 37.5% (QI), 20.8% (QII), 35.4% (QIII), and students to all the various lessons. The weekly heat maps 6.3% (QIV) respectively. (chapter by chapter) were created according to the procedure explained above and concatenated in chronological order. ABCDEFGHIJ KLMN The rightmost column in the figure shows the heat map of the course material clickstreams throughout the term and test heatm ap performance at the end of the term. Blank cells indicate absences. Only students who took the final exam are W eek 3rd 4th 5th 6th 7th 8th 9th 10th 11th 1 2 th 1 3 th (1 4 th ) 1 5 th represented. C hap.10 C hap.11 C hpa.12 F inal Q uiz C hap.1 C hap.2 C hap.3 C hap.4 C hap.5 C hap.6 C hap.7 C hap.8 C hap.9 Q IV Q IV Q IV Q IV Q IV 48 Q IV Q IV Q IV Q IV Q IV Q IV Q IV Q III 47 Q IV Q IV Q IV Q III Q IV Q IV Q IV Q III 46 Q IV Q IV Q IV Q III Q IV Q III Q IV Q III 45 Q IV Q III Q IV Q III Q III Q III Q IV Q III 44 Q IV Q III Q III Q III Q III Q III Q III Q III Q III 43 Q IV Q IV Q III Q III Q III Q III Q III Q III Q III Q III 42 Q IV Q IV Q IV Q III Q III Q III Q III Q III Q III Q III Q III 41 Q IV Q IV Q IV Q III Q III Q III Q III Q III Q III Q III Q III A BCDEFGHIJK L M N 40 Q IV Q IV Q IV Q IV Q IV Q III Q III Q III Q III Q III Q III Q III Q III heatm ap 39 Q IV Q IV Q IV Q IV Q IV Q IV Q III Q III Q III Q II Q III Q III Q III Q II W eek 3rd 4th 5th 6th 7th 8th 9th 10th 11th 12th 13th (14th) 15th 38 Q IV Q IV Q IV Q IV Q III Q IV Q III Q III Q II Q II Q III Q II Q II Q II Q uiz C hap.1 C hap.2 C hap.3 C hap.4 C hap.5 C hap.6 C hap.7 C hap.8 C hap.9 C hap.10 C hap.11 C hpa.12 F inal 37 Q IV Q IV Q IV Q IV Q III Q IV Q IV Q IV Q III Q II Q II Q II Q II Q II Q II Q II Student44 Q I Q I Q I Q II Q I Q I Q I Q I Q I Q IV QI QI QI 36 Q IV Q IV Q IV Q IV Q III Q IV Q IV Q IV Q IV Q II Q II Q II Q II Q II Q II Q II Q II 35 Q IV Q III Q IV Q III Q III Q IV Q IV Q IV Q IV Q II Q II Q II Q II Q II QI Q II Q II Student39 Q I Q I Q I Q I Q I Q I Q I Q I Q I Q II Q I 34 Q IV Q III Q IV Q III Q III Q III Q IV Q IV Q IV Q II QI Q II QI Q II QI Q II QI Student47 Q I Q I Q IV Q I Q I Q I Q IV Q I Q IV Q I Q I Q I Q I 33 Q IV Q III Q IV Q III Q III Q III Q IV Q III Q IV Q II QI Q II QI Q II QI Q II QI Student27 Q II Q I Q I Q IV Q I Q I Q I Q IV Q IV QI QI QI 32 Q IV Q III Q IV Q III Q III Q III Q IV Q III Q III Q II QI Q II QI Q II QI Q II QI Student48 Q II Q II Q I Q IV Q I Q I Q I Q I Q I Q IV QI Q IV Q I 31 Q III Q III Q III Q III Q III Q III Q IV Q III Q III Q II QI Q II QI QI QI Q II QI Student14 Q IV Q I Q I Q I Q I Q IV Q IV Q I Q I Q IV Q I 30 Q III Q III Q III Q III Q III Q III Q III Q III Q III QI QI QI QI QI QI QI QI 29 Q III Q III Q III Q III Q III Q III Q III Q III Q III QI QI QI QI QI QI QI QI Student25 Q IV Q II Q II Q II Q IV Q I Q I Q I Q I Q I Q II Q I 28 Q III Q III Q III Q III Q III Q III Q III Q III Q III QI QI QI QI QI QI QI QI Student20 Q IV Q III Q I Q I Q I Q IV Q I Q I Q I Q I Q I Q I Q I 27 Q III Q II Q III Q III Q III Q III Q III Q III Q III QI QI QI QI QI QI Student41 Q I Q I Q I Q III Q II Q II Q I Q II Q I Q II QI Q II Q I 26 Q III Q II Q III Q III Q II Q III Q III Q II Q III QI QI QI Student22 Q II Q I Q I Q III Q I Q I Q II Q II Q II Q II Q I QI 25 Q III Q II Q III Q II Q II Q II Q III Q II Q III Student43 Q II Q I QI QI QI Q III Q I Q IV Q I 24 Q III Q II Q III Q II Q II Q II Q III Q II Q III 23 Q III Q II Q III Q II Q II Q II Q II Q II Q III S tu de nt28 Q III Q IV Q I Q II Q I Q I Q IV Q I 22 Q II Q II Q III Q II Q II Q II Q II Q II Q III S tu de nt32 Q IV Q I Q II Q II Q II Q I Q IV Q II Q III Q II Q II Q II 21 Q II Q II Q II Q II Q II Q II Q II Q II Q III Student16 Q II Q IV Q II Q II Q II Q III Q II Q II Q II Q II Q IV Q II Q I 20 Q II QI Q II Q II Q II Q II Q II Q II Q II Student06 Q I Q III Q III Q II Q I Q II Q I Q I Q I Q II Q II Q I QI 19 Q II QI Q II Q II Q II Q II Q II Q II Q II Student21 Q I Q I Q IV Q II Q I Q III Q I Q I Q III Q II QI Q IV Q I 18 Q II QI Q II Q II Q II Q II Q II Q II Q II 17 Q II QI Q II Q II Q II Q II Q II Q II Q II Student29 Q I Q III Q I Q I Q II Q II Q II Q I Q I Q III Q I Q II Q II 16 Q II QI Q II Q II Q I Q II Q II Q II Q II Student46 Q I Q III Q IV Q II Q I Q I Q IV Q II Q I Q I Q II Q III Q I 15 Q II QI Q II Q II Q I Q II Q II Q II Q II Student45 Q II Q I Q IV Q I Q I Q I Q II Q I Q III Q III Q II Q II 14 Q II QI QI Q II Q I Q II Q II Q I Q II Student02 Q IV Q I Q II Q I Q II Q I Q I Q IV Q III Q III Q II 13 Q II QI QI Q II Q I Q II Q II Q I QI Student13 Q III Q II Q I Q II Q I QI QI Q III Q II Q II 12 Q I Q I Q I Q I Q I Q I Q I Q I Q I 11 Q I Q I Q I Q I Q I Q I Q I Q I Q I S tu de nt12 Q I Q IV Q II Q II Q III Q I Q I Q II Q II Q III 10 Q I Q I Q I Q I Q I Q I Q I Q I Q I Student19 Q III Q II Q I Q II Q I Q II Q III Q I Q IV Q IV Q IV 9 QI QI QI QI QI QI QI QI QI S tu de nt30 Q IV Q II Q I Q IV Q III Q I Q III 8 QI QI QI QI QI QI QI QI QI S tu de nt09 Q IV Q IV Q III Q II Q III Q II Q II 7 QI QI QI QI QI QI QI QI QI Student10 Q II Q II Q III Q II Q II Q II Q II Q II Q III Q II Q II Q II 6 QI QI QI QI QI QI QI QI QI 5 QI QI QI QI QI QI QI QI QI Student42 Q I Q I Q IV Q III Q II Q I Q II Q IV Q III Q III Q IV Q II Q I 4 QI QI QI QI QI QI QI QI QI Student15 Q II Q I Q IV Q I Q IV Q II Q II Q III Q III Q II Q II Q II Q III 3 QI QI QI QI QI QI QI QI QI Student36 Q III Q III Q I Q IV Q III Q II Q IV Q IV Q IV Q I 2 QI QI QI QI QI QI QI QI QI Student40 Q I Q I Q II Q II Q IV Q II Q III Q II Q III Q III Q II 1 QI QI QI QI QI QI QI QI QI Student03 Q IV Q I Q IV Q II Q III Q II Q II Q III QI Q III Q III Fig. 5. Example heat map showing ratios by group Student37 Q IV Q I Q IV Q I Q II Q III Q III Q II Q II Q II Q III Q II Q III Student05 Q III Q III Q II Q II Q I Q II Q III Q III Q II Student18 Q IV Q II Q I Q II Q II Q IV Q IV Q III Q III Q II Q III Q III Q II B. Finding disengaged students Student26 Q III Q III Q IV Q III Q IV Q IV Q I Q IV Q IV Q III Q IV Q IV As the quiz structure was multiple choice, with five- alternatives, quiz scores could rise and fall with a probability Student38 Q IV Q I Q IV Q IV Q III Q II Q II Q III Q IV Q III Q IV Q III of 20% in each group. Therefore, data corresponding to the vicinity of the X axis may move to another group—in Student11 Q II Q I Q I Q III Q III Q II Q II Q I Q III Q III Q III particular, to an adjacent group. A large correlation coefficient for number of course material openings and quiz Student34 Q I Q III Q III Q II Q I Q IV Q IV Q III Q I Q III Q III score was considered to be an indication of greater concentration on the lesson. Looking at the scatter chart of Student04 Q I Q III Q III Q II Q II Q II Q I Q III Q III Q II Q III the course material clickstreams and the quiz scores, it is desirable that there be more results in Group QI and fewer in S tu de nt17 Q I Q IV Q III Q III Q IV Q III Q III Q IV QI Q III Groups QIII and QIV. It is difficult to judge whether or not students understand the contents of a lesson simply by the S tu de nt23 Q III Q II Q III Q IV Q IV Q III QI Q III Q II Q III clickstreams of course material. However, this research showed a correlation (0.495, p < 0.05) between the final test Student31 Q III Q III Q II Q IV Q III Q III QI Q III score and the number of times over the entire course that the course material was opened. Student33 Q II Q III Q IV Q III Q III Q III Q III Q III Q I Q I Q IV We administered the first quiz in the third week of the S tu de nt07 Q IV Q III Q III Q III Q III Q III Q III class. A heat map was created from the clickstreams of the course materials and quiz scores. From Figure 4, the S tu de nt24 Q III Q IV Q III Q III Q I Q II Q III Q III Q III Q III Q III continued appearance of the red (QIII) and pink colors (QIV) indicates a continuation of a below-average state, and thus an Student08 Q III Q II Q IV Q III Q III Q III Q III Q IV Q III Q III Q III increased tendency of the student to become unsuccessful (e.g., Student 01, Student 07, Student 08). The absence of Student35 Q III Q III Q III Q IV Q III Q III Q III Q II Q III Q II Q IV Q III Q III Student01 Q III Q II Q III Q III Q III Q III Q III Q III Q III Q III Q III Q III Fig. 4. Weekly heat maps. Clickstreams during and outside of class are included (term: Fall 2017; course: Introduction to Social Data Analysis) There was no quiz in the first and second weeks. Blanks indicate student absences. The heat map is shown after sorting in descending order by frequency of QIII. Because the average number of course material openings and quiz scores differs from one week to the next, it is understood that students can appear in two or more groups and that they can change groups. Notably, there are many students who are shown consistently in blue (QI) in the upper part of the table (e.g., Student 44.), indicating that they are excellent students with excellent grades. In contrast, there are many students who are shown often in red (QIII) in the middle and lower sections of the table (e.g., Student 01). These students are highly likely to have less than average results. 142

2019 4th International Conference on Information Technology (InCIT2019) blue (QI) is also judged to be a sign that the student will fall VI. CONCLUSION into the unsuccessful group. Student 01, for example, showed light blue (QII) through the fourth week, but red (QIII) In this study, we found that students whose state did not continuing from the fifth week onward, and appears to be correspond to the blue (QI) group and who had a final exam falling into the group of unsuccessful students. result that places them in the red (QIII) group showed a tendency to be disengaged. Furthermore, if the student fell In cases where a student moves from blue (QI) to another into a category other than blue (QI) and thereafter appeared group, a good learning state can be maintained if the blue in a group other than blue (QI), the course materials tended color returns (Student 44). However, when the student state not to be opened and quiz scores tended not to rise. This does not return to blue (QI), observation and guidance may seems a clear sign of a student being unsuccessful. In be necessary, as this indicates a learning state that is not good particular, when red (QIII) and pink (QIV) continuously (e.g., Student 10, Student 26, Student 35, Student 38, Student appear as the student’s state, it is necessary for the teacher in 40). When a student's blue (QI) state continues, the number charge to intervene so that the student can better tackle the of times the course material is opened and the student’s quiz lesson reading and learn the course material more effectively. score are at an average level or higher, and learning is In addition, when there are many absences (blanks in the heat considered to be good (Student 39, Student 44). map), the learning trend is in question because the state of learning cannot be determined. C. Comparison with classical outliers Student page view behavior is likely influenced by the Table 1 shows the outliers from the clickstream and final quality and relevance of the materials, how the teacher test results and the heat map group membership pattern of the teaches, etc.—factors not addressed in this presentation. How outlying students. The students listed here were listed to communicate to the students the results of the analyses because they were never in Group Q1 (blue) and were in described here and how to respond are issues in need of Group III (red) in terms of their final exam score. For each of further attention. The scatter chart and heat map generated these students, we used Hotelling’s T2 to check their outlier during and outside of class from the Moodle course log can status. Specifically, we calculated be used by teachers to review and improve their approach to teaching. where is the observed value, is the average value, and ACKNOWLEDGMENT is the standard deviation. As shown in the table, Student 07 proved to be an outlier This work was supported by JSPS KAKENHI Grant Number 18K11588. according to Hotelling’s T2 theory both for course material clickstreams and quiz scores. In addition, because Student 01 REFERENCES has a small number of times opened the course materials, Student08 has a low quiz score, so each corresponded to [1] M. Taylor, “Action research in workplace education,” Quebec: Human abnormal values of Hotelling’s T2 theory. Student 35 is at the Resources Development Canada. National Literacy Secretariat. 2002, bottom of the heat map in Figure 4 but was not classified as pp. 95. both outliers. [2] Moodle https://moodle.org/ TABLE I. LIST OF STUDENTS WHO DID NOT HAVE BLUE (Q1) AND WHOSE FINAL EXAM SCORE IS RED (QIII) [3] K. Dobashi, “Interactive Mining for Learning Analytics by Automated Generation of Pivot Table,” In: Ahram T. (eds) Advances in Artificial Students Pageviews Final exam Heat map Hotelling’s T2 theory Intelligence, Software and Systems Engineering. AHFE 2018. Advances in Intelligent Systems and Computing, vol 787, Springer, 2017/9/20 Groups Outliers P-value Cham, 2018, pp.66-77. 2018/1/17 2018/1/17 Click Quiz Click Quiz [4] M. Ueno, “Online outlier detection system for learning time data in E- QI QII QIII QIV abse learning and It's evaluation,” Proc. of Computers and Advanced Technology in Education (CATE2004), 2004. Student01 364 18 0 1 9 1 2 1.597 0.408 0.117 0.685 [5] R. Mazza, and C. Milani, “Gismo: a graphical interactive student Student07 189 17 0 0 6 1 6 5.546 0.736 0.000 0.465 monitoring tool for course management systems,” In International Conference on Technology Enhanced Learning. Milan, 2004, pp.1-8. Student08 467 81 0 8 2 2 0.386 8.004 0.701 0.000 [6] I. Pytlarz, S. Pu, M. Patel and R. Prabhu, “What can we learn from The “Groups” columns show the number of times the student was a college students’ network transactions? Constructing useful features member of each heat map group; “abse” indicates an absence. Outliers and for student success prediction,” International Educational Data Mining P-values are based on Hotelling’s T2 theory, gray cells indicate abnormal Society, 2018. values. [7] N. Gitinabard, F. Khoshnevisan, C.F. Lynch, and E.Y. Wang, “Your By using Hotelling's T2, it was also possible to detect Actions or Your Associates? Predicting Certification and Dropout in abnormally high scores. Consequently, we examined the MOOCs with Behavioral and Social Features,” International correspondence with the heat map. Results indicated a Educational Data Mining Society, 2018. tendency for students with a large number of blue (QI) states to have abnormally high scores for teaching material views [8] C. Romero, R. Cerezo, A. Bogarín and M. Sánchez-Santillán, and quiz scores according to their Hotelling's T2 values. Here, “Educational process mining.” In Data Mining and Learning Analytics: we calculated Hotelling's T2 values using only one variable, Applications in Educational Research (eds S. ElAtia, D. Ipperciel and which is different from using the two variables of teaching O. R. Zaïane). John Wiley & Sons, pp.1-28, Hoboken, NJ, USA. 2016. material views and quiz scores. [9] H. Dierenfeld, and A. Merceron, “Learning analytics with excel pivot tables,” Proceedings of the 1st Moodle Research Conference (MRC2012), pp.115-121, Heraklion, Crete-Greece, Sept.2012. [10] A. Konstantinidis, and C. Grafton, “Using Excel macros to analyse Moodle logs,” Proceedings of the 2nd Moodle Research Conference (MRC2013), pp.33-39, Sousse, Tunisia, Oct.2013. 143

2019 4th International Conference on Information Technology (InCIT2019) i-Sleep: Intelligent Sleep Detection System for Analyzing Sleep Behavior Supakit Dhamchatsoontree Chaiyapat Sirisin Department of Computer Engineering Department of Computer Engineering Faculty of Engineering, Mahidol University Faculty of Engineering, Mahidol University Nakhon Pathom, Thailand Nakhon Pathom, Thailand [email protected] [email protected] Monika Proncharoensukkul Konlakorn Wongpatikaseree Department of Computer Engineering Department of Computer Engineering Faculty of Engineering, Mahidol University Faculty of Engineering, Mahidol University Nakhon Pathom, Thailand Nakhon Pathom, Thailand [email protected] [email protected] Abstract—Sleeping is a naturally mechanism of body for help Furthermore, several machine learning algorithms are used to to repair the body. However, to monitor the sleep quality is not analyze the data from the trials and training model, which can an easy task. In this research, we purpose sleep detection system, be used to classify an individual’s sleep postures. 6 postures which can classify sleep postures and calculate Sleep Quality on bed consist of supine lying, left lying, right lying, sitting on Index (SQI). Pressure sensing sensor, called i-Sleep sensor, with bed, changing posture and getting out of bed. We compared 7 48 embedded force sensors has been created in order to classify popular machine learning methods for sleep posture the sleep postures, several machine learning algorithms were classification. The experiment results showed that K-Nearest adopted to classify the sleep posture. From the experiment, the Neighbors (KNN) method got the highest accuracy at 86.7%. successful rate of sleep posture detection is 86.7%. Finally, web The result from this research can be used to measure the sleep application was implemented to show the real-time data, and quality index or analyze the behavior of each individual or sleep quality index in each day. person who has sleep problem, whether they have adequate sleep or not. Keywords— Sleep posture, Sleep Quality Index, Sleep detection system, Pressure sensing, i-Sleep II. LITERATURE REVIEW I. INTRODUCTION Sleeping is natural repeatedly mechanism of mind and body, inhibition of all voluntary muscles, and reduced Sleep is caused by a complex pathology. That is a natural interactions with surroundings. Normally, when we are phenomenon of living organism and changes in various sleeping, most of body’s systems work on anabolism state. physical pathways to relax in a sleep posture. There is a lower Anabolism state recruits immune system, nervous system, level of consciousness like temporary unconsciousness, also bone and muscular system. All of these processes are no response to external stimuli or less movement. And sleep is important to emotional and self-regulation memory and an important for human health because sleep helps repair and understanding efficiency. [1] refresh the body, relax and restore energetic activities. But not getting enough sleep will affect the body and mind, resulting Sleep posture classification has been proposed in several in less efficiency in the next day. And if this issue becomes a researches. We can divide the input device for sleep posture serious problem, it will cause a neurosis. classification into two groups: sensor-based technology and camera-based technology. There are many ways to manage sleep problem. Medicine is used most for helping sleep better. However, using medicine A. Sensor-based Technology may cause health effects later including drug tolerance and addiction. Moreover, drug withdrawal, can result the reverse In sensor-base technology, most of researches used Force- symptoms such as insomnia. The use of sleeping pills can sensor resistor (FSR) sensor for detecting physical pressure, manage sleep problem in the short term. But in the long term, squeezing and weight. FSR mechanism is a resistor that relaxation is the best way, such as listening to music, to help change its resistive value depending on how much its pressed. the body reduce stress and help sleep better without any side Meanwhile, some researches proposed piezoelectric ceramic effects to the body. sensors, which can produce an electrical potential when it is subjected to mechanical vibration. In addition, sleep posture and the period of sleep can show whether that person has an illness or not. The sleeping in the Hsia et al. proposed a pressure sensitive bed system for same posture may affect health, such as Office Syndrome and analyzing and comparing of sleep posture classification [2]. disease associated with muscle and bone. Although the illness Two layouts of FSRs mat, consisted of 16 strip sensors and has been treated, but if the person still has the same sleeping 56 sensors was conducted. Bayesian inference with Kurtosis behavior, symptoms may occur recurrently. and Skewness parameter were used to classify three main postures (supine, lying left, and lying right). They also To address this shortcoming, this research aims to create a compared three methods, which are PCA+SVM, Raw system for identifying sleep posture by using a pressure Data+SVM and Descriptive Statistics+SVM for classifying sensors. i-Sleep sensor is introduced to measure the human sleep posture. The average classification accuracy is 60%, force on bed, placed at various points under the mattress. 83%, and 77% respectively. In addition, sensor mats have 144

2019 4th International Conference on Information Technology (InCIT2019) been proposed in several researches [3-6]. A part of the type B. Hardware design of sensors, optimization of the number, location, and size of Pressure sensing sensor or i-Sleep was designed with 48 pressure sensor are interesting themes that several researches focus. 2048 pressure sensor mat was presented in [4 ]. They FSR sensors on 2 foam plates, as shown in Fig. 2. The first used 3 methods, filtering, morphological operation and plate was 57 centimeters width and the second plate was 61 scaling, for data normalization. K Nearest Neighbor was centimeters width. And both were 86 centimeters height. adopted for classifying 8 classes. Sabri Boughorbel et al. [5] These pressure sensing sensors were stacked together, so that introduced machine learning algorithms for sleep posture there was one piece of 118x86 centimeters i-Sleep sensor. classification. Four classes, including supine, pone, lying left, and lying right were classified. Fig. 1. Overview of i-Sleep system. Piezoelectric Ceramic sensor was also used to get Fig. 2. i-Sleep sensor. information on bed. Bed posture classification by neural i-Sleep sensor was placed under the mattress. Thus, volunteers network and Bayesian network using noninvasive sensor for would not directly touch the i-Sleep sensor. elder care was proposed by Viriyavit et al. [7]. Sensor panel, with 2 FSRs and 2 Piezoelectric Ceramic sensors, was To obtain the data, ESP32, a low-cost, low-power system introduced and placed in the thoracic area. Combination of on a microcontroller with integrated Wi-Fi and dual-mode neural network and Bayesian Network for classification was used to create the AI model. B. Camera-based Technology Vision technology is one of the devices that can determine sleep posture. Depth camera is one of popular devices that used in vision approach. Grimm et al. [8] proposed the technique to avoid fall when people sleep on bed by using depth camera. Disparity maps were converted to depth maps and bed aligned map to solve alignment problem into many grids. They used Convolutional Neural Network (CNNs) in 3 layers for classification human position on bed. Signal Processing was adopted with Depth Data. Data were record in 512x424@30 fps depth image. Camera position was set on top of a tripod and it is directly facing down on top of the subject. With the image processing technique, three features were extracted, which are FFT of whole 2D grayscale depth image, 2D cross-section scans of 3D depth data and Gabor feature. Finally, they propose 2D cross-section scans and its Fourier transform signals can be used for distinguishing different sleeping postures. However, the main problem of camera-based technology is privacy because we have to setup camera to record activities all night. III. METHODOLOGY A. System Architecture The objective of this study was to develop an automated pressure sensing bed system which could provide sleep postures and estimate sleep quality. We developed both hardware and software systems in order to provide patient’s assistance. Intelligent sleep detection system, i-Sleep system, was proposed in this research for analyzing sleep behavior, as illustrated in Fig. 1. In system architecture, we used 48 FSR sensors on two form plates. Each sensor obtained the pressure data in different location by microcontroller and sent its data through cloud for collecting data into database. Machine learning model was trained by collected data from 35 volunteers. This model is used to predict the current sleep posture and store result of classification into database for processing sleep quality. Furthermore, web application was implemented for display real time sleep posture and summarize the sleep behavior for estimation of sleep quality. Fig. 3. Calibration method 145

2019 4th International Conference on Information Technology (InCIT2019) Fig. 4. Data preparation The preprocessed data is categorized into three levels: raw data, normalized data, and color scale data. Fig. 5. Training set and testing set Bluetooth was used. By putting this sensor in series • Raw data refers to the pressure data that collected electrically with a resistor of known value, where the total from i-Sleep sensor after calibration and removed voltage across the whole circuit was 5V, applied force would missing value. Range of raw data was between 0- cause the voltage going into the MCU to fall between 0-5V, 4095. which was the common operating range of many MCU’s analog to digital converter (ADC). For a 12 bit ADC, the • Normalized data refers to the data that used the maximum voltage of 5 V corresponds to the digital value of normalization process. We used standardization, 4095. shown in equation (1), after collected the raw data C. i-Sleep system from i-Sleep sensor. Due to the range of data was wide, each attribute might affect to the accuracy. In order to classify sleep posture, machine learning Therefore, we tested the model by normalized the technique is adopted in i-Sleep system. We can divide into data for reducing the possibility of inconsistent data. three parts as followed. Data preprocessing ������ = $% & (1) ' First a baseline value of each sensor had to be established, since each sensor’s response characteristic is a little different • Color scale data, we have purpose of separating the from all the other due to the variation in constructing them. data that we need the smallest numbers of data set First, a total of five output values from each sensor was and equal numbers of data in each set. We divided averaged and plus with some constant for protecting the error. data into four sets due to corresponding to our After we got the threshold, new input data would be minus by purpose and these sets might show significant results. threshold for setting zero for each sensor, as show in Fig. 3. Machine Learning To create sleep posture classification model, we collected the pressure data from the participants with 5 sleep postures It is a tool for prediction or classification the data. In this consist of lying supine, lying left, lying right, sitting on bed research, sleep posture classification model was created by and getting out of bed. Participant was asked to perform each using supervised learning technique. For the classical mahine sleep posture in 1 minute. The sensors were read from every learning classifiers [9], we considered 7 algorithms, including 10 seconds, as show in Fig. 4. Gaussian Naive Bays, Bernoulli Naïve Bays, Neural Network, Support Vector Machine (SVM), Logistic Regression, However, the dataset was not clean enough for training Random Forest and K-Nearest Neighbors (KNN). model. Missing value happened in some circumstance due to quality of sensor. Thus, we edited data in the dataset by using With the advantage of machine learning technique, the the average value fill in the lost data. Then, we split training ambiguous pattern can be identified. For example weights of set with 80% and testing set with 20%. In training set, data was male and female are different. It can make variety pattern of used to build the machine learning model for analyzing and pressure data although sleep with the same posture. classifying sleep posture. In testing set, data was used to estimate performance of machine learning model, as shown in Fig. 6. Majority vote on web application Fig. 5. Data categories Fig. 7. Sleep prediction Furthermore the pressure data is continuous value and each 146

2019 4th International Conference on Information Technology (InCIT2019) Fig. 8. Sleep dashboard • Component 4: Sleep efficiency • Component 5: Sleep disturbance (a) (b) (c) Finally, we add 5 component scores together for estimating sleep quality score. There are 3 levels in sleep (d) (e) quality score: • 75% and above: Good sleep quality Fig. 9. Sleep posture (a) Lying supine (b) Lying right • 50% to 75%: Fair sleep quality (c) Lying left (d) Sitting on bed (e) Getting out of • Less than 50%: Poor sleep quality bed. IV. EXPERIMENT RESULTS sleep posture is active in different zone. For example, when human sleep in supine posture, center of i-Sleep sensor will be A. Experimental set-up activated, or left side of i-Sleep sensor will be activated when human sleep most to the left or lying left. However, each The experiment was done by 30 volunteers, 15 males and activated zone has different pattern. Thus it is difficult to use 15 females. They were different in gender, height and weight. only rule-based to classify the sleep posture. They were also diverse in term of body pattern and how they fall as sleep. The average height and weight of 30 people was D. Web appplication 150-180 centimeters and 50-80 kilograms. Each of participate After we received the classified sleep posture result, we slept on the bed to collect the pressure data. Volunteers performed 6 postures, including lying supine, lying right, will display on web application. The server was developed by lying left, sitting on bed, changing posture and getting out of Python and the data is collected into the mongoDB. For the bed, which collected about 1 minute long per posture. web application, we have two main modules: However, if there is a failure during sending the data to cloud, the current pressure data will not be counted. The steps of • Realtime sleep posture prediction: It retrieved three collecting sleep posture data illustrate in Fig. 9. latest predition results from the database through API. Then, Majority vote, illustrated in Fig. 6, is used to B. Experimental results determind the current sleep posture before showing on web application., as shown in Fig. 7. From the experimental setup, 1983 sleep posture records were collected, which could be divided to lying supine (388 • Sleep dashboard: We summarized the sleep posture in records), lying right (379 records), lying left (377 records), daily or monthly for analyzing sleep behavior and sitting on bed (385 records), changing posture (99 records) and estimation of sleep quality. In sleep quality, we use getting out of bed (355 records). We divided the data into 2 Pittsburgh Sleep Quality Index (PSQI) as a guideline parts: 80% for Training set and 20% for Testing set. The for calculate the sleep quality index, as show in Fig.8. results of the experiment are as follows: For the sleep quality index of this research, we calculated Raw data based on five components: The data in Raw data format, in range between 0-4095, • Component 1: REM cycles was tested with 7 algorithms, and the result of sleep posture • Component 2: Sleep latency classification is presented in Table 1. From the results, random • Component 3: Sleep duration forest, KNN, and logistic regression got the high accuracy with 95.2%, 94.9%, and 94.7% respectively. Standardized data Due to the range of the raw data is wide, we compared the 7 traditional machine learning algorithms with normalized data. Table II shows the result of sleep posture classification with normalized data. From the results of sleep posture classification with normalized data. All algorithms achieved high accuracy in prediction. However, SVM got the highest accuracy with 95.8%. TABLE I. SLEEP POSTURE CLASSIFICATION RESULTS WITHRAW DATA Algorithm Accuracy (%) Gaussian Naïve Bayes 84.3 Bernoulli Naïve Bayes 90.6 Multi-Layer Perceptron (MLP) 92.4 Support Vector Machine (SVM) 37 Logistic Regression 94.7 Random Forest 95.2 K-Nearest Neighbors (KNN) 94.9 TABLE II. SLEEP POSTURE CLASSIFICATION RESULTS WITH NORMALIZED DATA 147

2019 4th International Conference on Information Technology (InCIT2019) Algorithm Accuracy (%) V. CONCLUSION Gaussian Naïve Bayes 93.9 Bernoulli Naïve Bayes 93.1 In this work, we implemented i-Sleep for identifying sleep Multi-Layer Perceptron (MLP) 94.5 posture by using a pressure data. We designed 48 FSRs Support Vector Machine (SVM) 95.8 sensor, called i-Sleep sensor, for detecting pressure on the Logistic Regression 94.5 bed. Then, data was collected 30 participants from 15 males Random Forest 95.2 and 15 females. All data were categorized into raw data, K-Nearest Neighbors (KNN) 94.5 standardized data and color scale data. Data was training in various models. From the experiment, when models were Color scale data testing by unseen data, K-nearest neighbors algorithm achieved the highest performance with 91.1, 88.8, and 86.7 Data in color scale data has the short-range value, which is accuracy in raw data, normalized data, and color scale data between 0 and 3. Table III shows the result of classification respectively. Furthermore, the results of classification were when using color scale data. From the TABLE III, shown on web application. User can find the sleep posture classification results are not significant different, except history and sleep quality of each day via web application. Gaussian Naïve Bayes and Bernoulli Naïve Bayes. In addition, we have tested the i-Sleep system with unseen data. We use For future work, we could reduce number of sensors, and 100 sleep posture records from extra two volunteers, and 20 develop modern technique into classification method such as records of each sleep posture were collected. Table IV convolution neural network, or deep learning. describes the classification results of unseen data test in raw data format, color scale data format , and standardize data ACKNOWLEDGMENT format. This research was supported by National Research From the TABLE IV, K-Nearest Neighbors achieved the Council of Thailand (NRCT) under the project named “Sleep highest accuracy in testing with unseen data in all data Irregularities Monitoring and Data Analytics for Elderly formats: raw data, normalized data, and color scale data with Persons using Assistive Technology”. 91.1%, 86.7%, and 88.8% respectively. Finally, we select KNN classification for i-Sleep system because normalized REFERENCES datasets are appropriate for KNN. Moreover, KNN got high accuracy with unseen data test. In this research, k was set as [1] Stroke, O. o. (2019, 02 08). Brain Basics: Understanding Sleep. odd value for protecting prediction result in the same class and Retrieved from: https://www.ninds.nih.gov/Disorders/Patient- then we set k is 5. CaregiverEducation/Understanding-Sleep#top TABLE III. SLEEP POSTURE CLASSIFICATION RESULTS WITH [2] C.C. Hsia, K.J. Liou, A.P.W. Aung, V. Foo, W. Huang, and J. Biswas, COLOR SCALE DATA \"Analysis and comparison of sleeping posture classification methods using pressure sensitive bed system,\" 2009 Annual International Algorithm Accuracy (%) Conference of the IEEE Engineering in Medicine and Biology Society, Gaussian Naïve Bayes 77.5 pp. 183-192, 2009. Bernoulli Naïve Bayes 89.1 Multi-Layer Perceptron (MLP) 95.7 [3] M. Alaziz, Z. Jia, J. Liu, R. Howard, Y. Chen and Y. Zhang, \"A Body Support Vector Machine (SVM) 95.9 Motion Monitoring System Using Bed-Mounted Wireless Load Cells,\" Logistic Regression 93.9 2016 IEEE First International Conference on Connected Health: Random Forest 95.7 Applications, Systems and Engineering Technologies (CHASE), pp. K-Nearest Neighbors (KNN) 94.5 183-192, 2016. TABLE IV. SLEEP POSTURE CLASSIFICATION RESULTS WITH [4] M. B. Pouyan, S. Ostadabbas, M. Farshbaf, R. Yousefi, M. Nourani and UNSEEN DATASET M. D. M. Pompeo, \"Continuous eight-posture classification for bed- bound patients,\" 2013 6th International Conference on Biomedical Accuracy (%) Engineering and Informatics, pp. 121-126, 2013. Algorithm Color Normali [5] S. Boughorbel, F. Bruekers and K. Groot, \"Pressure-Sensor System for Gaussian Naïve Bayes Sleeping-Posture Classification.\" 8th International Conference on Raw data scale zed Methods and Techniques . Netherlands: Utrecht, 2012. 358-361. data data [6] A. Mineharu, N. Kuwahara, and K. Morimoto, \"A Study of Automatic Classification of Sleeping Position by a Pressure-sensitive Sensor.\" 80.0 82.2 4.4 2015 International Conference on Informatics, Electronics & Vision (ICIEV). Graduate School of Science and Technology Kyoto Institute Bernoulli Naïve Bayes 80.2 82.2 77.7 of Technology, Kyoto, Japan. Multi-Layer Perceptron (MLP) 80.0 77.7 75.7 [7] W. Viriyavit, V. Sornlertlamvanich, W. Kongprawechnon, P. Pongpaibool and T. Isshiki, \"Neural network based bed posture Support Vector Machine (SVM) 37.7 62.2 62.2 classification enhanced by Bayesian approach,\" 2017 8th International Conference of Information and Communication Technology for Logistic Regression 71.1 82.2 75.7 Embedded Systems (IC-ICTES), 2017. Random Forest 80.0 77.1 77.7 [8] T. Grimm, M. Martinez, A. Benz, and R. Stiefelhagen, \"Sleep position classification from a depth camera using Bed Aligned Maps.\" 2016 K-Nearest Neighbors (KNN) 91.1 88.8 86.7 23rd International Conference on Pattern Recognition (ICPR). Cancun, Mexico: IEEE, 2016. 316-324. [9] P. Tan, M. Steinbach, V. Kumar, \"Introduction to data mining (First Edition)\" March 25, 2006. 148

2019 4th International Conference on Information Technology (InCIT2019) Thai Named Entity Recognition Using Bi-LSTM-CRF with Word and Character Representation Suphanut Thattinaphanich Santitham Prom-on Department of Computer Engineering, Department of Computer Engineering, Faculty of Engineering Faculty of Engineering King Mongkus University of Technology Thonburi King Mongkut’s University of Technology Thonburi Bangkok, Thailand Bangkok, Thailand [email protected] [email protected] Abstract—Named Entity Recognition (NER) is a handy tool the increase of computation power. In this research paper, we for many natural language processing tasks to identify and present the neural network architecture, Bidirectional Long extract a unique entity such as person, location, organization Short-term Memory with Conditional Random Field (Bi- and time. In English and Chinese, NER has been thoroughly LSTM-CRF), implemented with word and character repre- researched and is able to be applied in more practical settings. sentation for the named entity recognition task. With the Its development in Thai is still limited because of rare resources nature of Bi-LSTM, the model can learn the information and language difficulties such as the lack of boundary indicator from the successive word and the precedent word in the for words, phrases and sentences. In this paper, we present sentence. In addition, the model absorbs the information of an application of Bi-LSTM-CRF with word/character level each popular word from word and character representation. representation, to solve this problem. Firstly, we prepared texts by tokenizing a sentence to a bunch of words. We then prepared II. RELATED WORK word representation and Bi-LSTM character representation. In the end, we built a recurrent neural network combined with For the Thai language, NER research is still in an infantile CRF to learn the sequence of text and extract the knowledge state. A few works attempted to address this issue. Firstly, to build NER recognition to overcome this problem. Our model Charoenpornsawat et al. (1998) [4] used context words, part was evaluated by the NER opensource corpus from a Facebook of speech (POS) and heuristic rules to extract the information group ThaiNLP. The results of our model yielded precision, as a feature. Then they applied the Ripper and Winnow recall, and F1 at 91.79%, 91.51% and 91.65% respectively. algorithm to classify the entity names with the result of 92.17% accuracy. Chanlekha et al. (2004) [5] used maximum Index Terms—Named Entity Recognition, Recurrent Neural entropy with heuristic information from rules and word- Network, Bi-LSTM, Conditional Random Field, Thai Language occurrences to extract the named entity. The result of the experiment yielded the accuracy of approximately 87.7% I. INTRODUCTION including name of person (90.44%), organization (89.87%) and location (82.16%). Tirasaroj et al. (2009) [2] forged Name Entity Recognition (NER), also known as entity the conditional random fields model to extract NER and extraction, is one of essential elements in Natural Language experimented the pattern and the factors affecting the model. Processing (NLP) tasks. It is used in information extraction The result for InterBEST2009 news showed that the pattern to identify and segment named entities and classify them into had some effects on the system. The patterns for the experi- several predefined classes. In widely spoken languages such ment include BOI (Begin, Inside, Outside) and BOIE (Begin, as English and Chinese, massive studies have been conducted Outside, Inside, End) for Person, Location and Organization in various algorithms for this task. Nevertheless, Thai named entities. For token word evaluation, the most accurate pattern entity recognition is limited due to the characteristics of the was BOI which could archive 86.5% F1 score. Saetiew et language. Thai does not have orthographical information or al. [6] used likelihood probability of tokenized words to boundary indicators to separate words, phrases or sentences. identify the person entity from texts with 85.15 F1 score for This creates a challenge for the machine learning model to InterBEST 2009 news corpus. Phatthiyaphaibun Wannaphong learn the feature from the hand-engineering feature. [7] improved Tirasaroj et al. [2] by adding Part-of-Speech as a feature to the CRF model. The result obtains slight Given the limitation, only few studies exist for Thai name improvement at 86.9% F1 score. entity recognition task. In the past decade, the most common methods for this task was a machine learning approach sim- For the popular language, NER is one of the fundamental ilar to English and other languages. One successful strategy research fields in NLP. Many pieces of research can archive for solving this task was using the Conditional Random over 90% accuracy with state-of-the-art algorithms in 2010s: Field (CRF) model which provided excellent outcomes [1][2]. They are machine learning and deep learning algorithms, Recently, deep learning application has come to supersede combined with other outstanding techniques. Santos et al. the conventional methods in multiple fields including image recognition and natural language processing due to its ability to generalize knowledge in the large size of the dataset and 149

2019 4th International Conference on Information Technology (InCIT2019) (2014) [8] lifted the character-level embedding idea to per- it = σ(Wiht−1 + Uixt + bi) form with the neural network then compared the performance of word-level embedding and character-level embedding as a ft = σ(Wf ht−1 + Uf xt + bf ) feature with an F1 score of 82.21% for Spanish and 71.23% for Portuguese. Chiu et al. (2015) [9] presented Bi-directional c˜ = tanh(Wcht−1 + Ucxt + bc)) LSTM-CNNs architecture to challenge the name entity recog- nition. They performed Bi-LSTM-CNN combined with word ct = ft ct−1 + it c˜t embedding, character embedding feature, and Lexicons to the CoNLL-2003 dataset with an F1 score of 91.62%. Xeuzhe et ot = σ(Woht−1 + Uoxt + bo) al. (2016) [10] constructed Bi-directional LSTM-CNNs-CRF. The model computed word-level representation using GloVe ht = ot tanh(ct) [11] as word embedding and character-level representation using the convolution neural network. After they prepared all where σ is the element-wise sigmoid function and is the the input, they fed them into Bi-LSTM and passed the output element-wise product. xt is the input vector (word embedding vector of Bi-LSTM through a CRF layer to compute the tag and character embedding) at time t, and ht is the hidden state in the last phase. The best result for ConLL2013 shared- vector at time t that stores the information from this state and task was 91.21% F1 score. Similar to Xeuzhe et al. [10], the past. Ui, Uf , Uc, Uo are the weight matrices of gate for Lample et al. (2016) [12] built a Bi-LSTM-CRF with word input xt. Wi, Wf , Wc, Wo are the weight metrices for hidden and character representation. They, however, used LSTM to state ht. bi, bf , bc, bo are the bias vectors. form character representation. The result for ConLL2013 test set was 90.94% F1 score. Bi-directional LSTMs is an ordinary LSTM network with an additional layer that passes the data from the backward From these results, it is clear that the deep learning direction. This idea is popular in text processing because method should also be used to accomplish this task for we trained the text model with a complete sentence. This Thai. Therefore, we constructed a deep learning architecture technique provides past and future information. Instead of including word representation, character representation, Bi- learning the context from the previous word, with bidirec- directional Long-short term memory and CRF components tional LSTM, we can trained the model using both the to solve the named entity recognition issue. precedent word and the successive word. III. MODEL COMPONENTS C. Word-Level Representation A. Tokenization Word embedding is vector representation with low dimen- sional space. These vectors are generated from the large body Thai is a run-on language which lacks the boundary of text such as Wikipedia. Hence, they contain the informa- indicator that demarcate word boundaries; all words in the tion that represents the contextual similarities between words. document are stuck together and you need to preprocess There are several ways to generate word embedding such as them first. In this current state, there are many approaches to word2vec [16], GloVe [11] or the embedding layer from the separate a sentence into a pack of words including algorithm- pre-trained language model. For our model, we used a word based, dictionary-based, machine learning-based and deep embedding layer from ULMFit [17]. ULMFit is an effective learning-based approaches. Our word tokenization method transfer learning method to build a language model that can is a dictionary-based word segmentation combined with be applied to many NLP tasks. ULMFit uses AWD-LSTM maximum matching algorithm and Thai Cluster Text called [18] to build a language model. While training the pre-trained newmm [13]. Since NER task may not require much accuracy language model, ULMFit predicts the next word to learn the in tokenization because there is a BOI tag that handles this information and create a language model. This method using problem, we selected newmm because this algorithm can deep learning to understand the context from surrounding deliver a good accuracy with less processing time. words and update word vectors. Our word embedding is extracted from a general-domain corpus language model B. Bi-directional LSTM which contains common features from a tremendous text corpus. Recurrent Neural Networks [14] are a powerful and robust type of neural networks. The idea of architecture is the D. Character-Level Representation sequential information where the output can resurface itself. This method is regarded as a short-term memory because Character-level representation is a word embedding that it can remember the output that loops back to itself. So contains information from a list of characters. In general, the recurrent network has significant inputs: the present, and there are two remarkable ways to generate character-level recent past. However unfortunately, if the distance of infor- representation including CNN and LSTM. These techniques mation node is quite far, RNN will be unable to learn that are powerful to obtain morphological information from the information. This problem is called long-term dependencies. characters instead of using hand-engineering prefix and suffix information about the word. Reimers et al. [18] explained that Long-Short term memory networks [15] are a particular there was no significant difference in the result for NER. class of RNN. This architecture enables the mechanism to So in this paper, we used Bi-LSTM to perform this task. learn long-term dependencies. By using the cell state, it Figure 1 shows the Bi-LSTM model for fusing the character allows the information to flow from the previous state and representation from the set of characters. update every time it passes to the next state. For the formulae of LSTM unit at time t are: 150

2019 4th International Conference on Information Technology (InCIT2019) Then we fed them into Bi-LSTM network. Finally, the output vectors of Bi-LSTM were fed to the CRF layer to decode the best tag sequence. Fig. 1. Character-level Representation using Bi-LSTM networks E. Conditional Random Field Fig. 2. Bi-LSTM CRF with Word/Character representation Architecture For sequence labeling jobs, CRF is one of the most IV. NETWORK TRAINING excellent models to predict the chain of labels from analyzing the relationship of the word. For example, in the NER task, In this section, we give the detail about training parameter the word with label I-PER cannot be followed with tag I- for our neural network. We used Keras library to building the ORG. Hence, instead of decoding the label independently, neural network model. Our computation power was a Tesla we modeled the label sequence using a conditional random P-100-PCIE-16GB from google cloud. The model training field [1]. time took around 3-4 hours for 50 epoch. x = {x1, x2, ..., xn} represents an input sequence where A. Word Embedding xi is the input vector of ith word and y = {y1, y2, ..., yn} represents tag prediction sequences from sequence x. P is the We used public word embedding from the Thai2Fit project matrix of output score from the Bi-LSTM network. P has the [20]. Thai2Fit implements ULMFit and trains Thai Language size of n x k where k is the number of unique tags. Pi,j is Model from Thai Wikipedia including 130000 documents. the score of jth tag of the ith word in a sentence. We can This word embedding contains 60000 word vectors with 400 define the score as: dimensions. The input word for our model contained 250 words for one document because the input document came nn from many sources and may have a long paragraph. If an input word was not long enough, the leftover vectors would S(x|y) = Ayi,yi+1 + Pi,yi (1) be assigned as padding and the word that was not in Thai2Fit would be assigned as an unknown word. i=0 i=1 B. Character Embedding where A is a matrix of transition scores and Ai,j describes the score of transition from tag i to tag j. y0 and yn indicate Character embedding was generated from all characters the start and end tags from the sentence. in Thai2Fit word embedding containing 399 characters. We trained the character embedding from scratch, which means A softmax of all possible tags can represent the probability these character embedding resulted from random initials for for sequence y as: every character at the start. The unknown character such as Japanese and Chinese words which did not include lookup ta- exp(S(x, y)) (2) ble was mapped as an unknown character. The input character P (y|x) = for input vector contained 32 characters for one word. If input character was not long enough, the leftover vectors would be y ∈Yx exp(S(x, y ))) assigned as padding. For the recurrent dropout for Bi-LSTM layer, Reimers et al. [18] showed that a variation dropout In the training phase, we use maximum conditional likeli- (0.5) could yield better result. Thus, we set the dropout at 0.5. The output dimension of character representation from hood estimation to maximize the log probability of the correct Bi-LSTM was 64. tag sequence by: C. Optimization Algorithm log(P (y|x)) = S(x, y) − log( exp(S(x, y ))) (3) The optimization parameter for this neural network model was Adam with batch size = 32. The initial learning rate of y ∈Yx the model was 0.001. The state unit of main Bi-LSTM was Decoding is to predict the label y* with the highest score given by: y∗ = argmaxS(x, y ) (4) y ∈Yx F. Bi-LSTM-CRF After we gathered all the component, we then developed our neural network model. Figure 2 shows the detailed overview of the architecture. The input of the main Bi-LSTM- CRF model is character-level representation generated from Bi-LSTM concatenated with word embedding from ULMFit. 151

2019 4th International Conference on Information Technology (InCIT2019) set at 512 units and the dropout equaled to 0.6. We trained 50 epoch in every experiment and selected only the best result. V. EXPERIMENT Fig. 3. Example of data A. Datasets and applying Bi-LSTM character representation. The fourth experiment (Model 4) was performing Bi-LSTM-CRF with For Thai named entity recognition, the dataset was quite pre-trained word embedding from ULMFit and applying Bi- small. The standard dataset for training named entity was LSTM character representation. The evaluation metric was in BEST2009 of which the label explained only whether this terms of precision, recall and F1 scores. The precision is the word was NER. However, it did not identify that it was number of correctly predicted NE token divided by the total person entity, location entity, organization entity or other token of NE which is extracted by the system. The recall entities. Later, Tirasaroj et al. [8] presented a 3 class dataset is a measure of the number of correctly predicted NE token containing person, organization and location. At present, divided by the total number of manual annotated NE token. there is an opensource dataset from PyThaiNLP project [2] And F1 score is a weighted mean of the precision and the that builds the expanding dataset of Tirasaroj dataset using the recall from the following formula: crowdsourcing to gather the multiple tag dataset. This dataset includes 13 classes, 6148 sentences, 50593 tokens tokenized by the newmm method with NER tags in the BOI format. For our experiment, we randomly split the dataset with 80:20 ratio, that is 4918 training sentences (40545 tokens) and 1230 testing sentences (10048). Table I shows the summary number of our dataset and Table II shows proportion of NER tag. TABLE I DATASET DESCIPTION Type All Train Test F = 2 ∗ P recision ∗ Recall (5) Sentence 6148 4918 1230 P recision + Recall Word 197704 40237 NER Token 50593 157467 10048 40545 TABLE II As shown in Table III, applying only pre-trained word NER TAG DESCIPTION embedding could improve 3% for precision and recall so F1 score increased up to 3%. Using only character representation Type Word Token could slightly enhance precision for 1.7% and recall dropped Date 1823 5676 slightly. Therefore, this model could improve F1 score at ap- Email proximately 0.5%. However, when we combined two factors, Law 11 79 we obtained a huge gain of 4.9% for precision and 7.1% for Len 194 40545 recall, for which we gained a significant overall improvement Location 125 at 91.65% F1 score. Money 4488 400 Organization 579 9239 In table IV, we constructed the experimental to compare the Percent 5577 2002 performance of our best model with, the state-of-the-art NER Person 160 12342 model for Thai language of Phatthiyaphaibun Wannaphong Phone 3159 407 [7], CRF model using NER and POS tag as features. Our Time 108 14698 model outperformed the CRF + POS model by obtaining URL 905 391 4.75% F1 score improvement. ZIP 114 2695 25 1834 25 B. Main Result C. Hyperparameter We tested our Bi-LSTM-CRF experiment with two config- In this section, we experimented how the hyperparameter urations including word representation and character repre- affected the model. This experiment base used the best sentation. We constructed the 4 tests to evaluate the result performance from subsection B, Bi-LSTM-CRF using word of applying word and character representations with Bi- representation and character representation. The main factor LSTM-CRF models. The first experiment (Model 1) was for this experiment was LSTM units for the main model, the performance of Bi-LSTM-CRF with initially random dropout and optimizer as shown in Table IV. The result de- word embedding training from scratch and did not include scribes that the hyperparameter did not affect much the model character embedding. The second experiment (Model 2) was as the result was changing around 0.64%. The variation the performance of Bi-LSTM-CRF with pre-trained word dropout could yield a better in every experiment compared to embedding from ULMFit without character embedding. The naive dropout similar to Reimers et al.’s result [18]. However, third experiment (Model 3) was performing Bi-LSTM-CRF for the number of LSTM units and optimizers, the F1 score with initially random word embedding training from scratch was improved when the number of LSTM units increased and Adam as optimizer equaled at 512. Nevertheless, when 152

2019 4th International Conference on Information Technology (InCIT2019) TABLE III hyperparameters slightly affect the model. More studies will F1 SCORE FOR BI-LSTM MODEL shed some light further in the development of Thai NER. Firstly, we need more dataset from reliable sources, since Type Model 1 Model 2 Model 3 Model 4 the corpus from PyThaiNLP is crowdsourcing. The data may Date 93.59 92.93 92.62 95.96 contain methodological errors. Second, there are alternative Email 44.44 76.19 0.00 100.00 methods to generate word embedding. We can develop and Law 62.22 71.02 0.50 71.35 experiment with the other word embedding to improve the Len 83.44 83.24 79.55 89.49 result. Lastly, multiple strategies to implement the model Location 79.55 84.66 79.52 87.73 should be further explored by adding an additional feature Money 89.58 94.02 91.15 95.46 such as ELMO embedding or BERT embedding. Organization 78.39 81.48 79.26 85.20 Percent 94.40 75.68 88.89 93.20 ACKNOWLEDGMENT Person 90.80 95.14 91.29 97.01 Phone 79.90 91.60 92.33 98.02 We would like to thank Mrs.Nutcha Tirasaroj, NECTEC Time 84.92 84.17 86.37 89.73 and PyThaiNLP for providing the NER corpus, Mr.Charin URL 95.50 98.85 98.33 98.85 Polpanumas for the Thai2Fit word embedding and Depart- ZIP 66.67 85.71 72.73 100.00 ment of Computer Engineering, King Mongkut’s University Average 85.64 88.85 86.18 91.65 of Technology Thonburi for their generous support for this work. TABLE IV F1 SCORE FOR BI-LSTM MODEL REFERENCES Type Best Model CRF + POS [1] J. Lafferty, A. McCallum, and F. Pereira. ”Conditional random fields: Date 95.96 93.76 Probabilistic models for segmenting and labeling sequence data.,” In Email 100.00 100 Proc, ICML, 2001. Law 71.35 59.24 Len 89.49 93.23 [2] N. Tirasaroj and W. Aroonmanakun, Thai Named Entity Recognition Location 87.73 80.65 Based on Conditional Random Fields, in International Symposium on Money 95.46 91.69 Natural Language Processing (SNLP), Thailand, pp. 216-220, 2009. Organization 85.20 80.69 Percent 93.20 83.84 [3] Python Thailand Group (PyThaiNLP). Thai Named Entity Recognition Person 97.01 91.63 Corpus [Online]. Available: https://github.com/PyThaiNLP/pythainlp. Phone 98.02 95.76 Time 89.73 84.86 [4] P. Charoenpornsawat, B. Kijsirikul, and S. Meknavin, Feature-based URL 98.85 96.57 Proper Name Identification in Thai, in Proc. of National Computer ZIP 100.00 90.9 Science and Engineering Conference: NCSEC98, Thailand, 1998. Average 91.65 86.9 [5] H. Chanlekha and A. Kawtrakul, Thai Named Entity Extraction by the LSTM was placed at 1024, the F1 score would slightly incorporating Maximum Entropy Model with Simple Heuristic Infor- drop. On the other hand, Nadam optimizer can yield the best mation, in Natural Language Processing (IJCNLP), China, 2004. result at unit = 1024. But in the end, F1 score of unit = 512 with Adam can archive the better result than unit = 1024 with [6] N. Saetiew, T. Achalakul, and S. Prom-onm, ”Thai Person Name Nadam. Recognition (PNR) Using Likelihood Probability of Tokenized Words,” in 5th International Electrical Engineering Congress, Pattaya, Thailand, TABLE V 8-10 March 2017. HYPERPARAMETER RESULT [7] W. Phatthiyaphaibun. Thai Named Entity Recognitions for PyThaiNLP Optimizer Variation Drop out Precision Recall F1 [Online]. Available: https://github.com/wannaphongcom/thai-ner. Adam LSTM Unit 0.1 Adam 0.5 91.53 90.91 91.22 [8] C. Santos and V. Guimares, ”Boosting Named Entity Recognition with Adam 256 0.1 92.02 90.80 91.41 Neural Character Embeddings,” In Proceedings of NEWS 2015 The Adam 256 0.5 91.74 90.40 91.06 Fifth Named Entities Workshop, page 25. Adam 512 0.1 91.79 91.51 91.65 Adam 512 0.5 91.15 90.86 91.01 [9] J. P. Chiu and E. Nichols, Named Entity Recognition with Bidirectional Nadam 1024 0.1 91.99 90.77 91.38 LSTM-CNNs, Transactions of the Association for Computational Lin- Nadam 1024 0.5 91.65 91.08 91.36 guistics, vol. 4, pp. 357370, 2016. Nadam 256 0.1 90.69 91.72 Nadam 256 0.5 91.38 91.30 91.2 [10] X. Ma and E. Hovy, End-to-end Sequence Labeling via Bi-directional Nadam 512 0.1 91.93 90.83 91.34 LSTM-CNNs-CRF, Proceedings of the 54th Annual Meeting of the Nadam 512 0.5 92.28 90.00 91.38 Association for Computational Linguistics (Volume 1: Long Papers), 1024 91.91 91.19 91.12 2016. 1024 91.56 [11] J. Pennington, R. Socher, and C. Manning, Glove: Global Vectors CONCLUSION for Word Representation, Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2014. This paper presents Bi-LSTM-CRF with word and char- acter representations to extract named entities from sen- [12] G. Lample, M. Ballesteros, S. Subramanian, K. Kawakami, and C. tences from PyThaiNLP corpus. The result of the model Dyer, Neural Architectures for Named Entity Recognition, Proceedings yield an incredible achievement for precision, recall, and F1 of the 2016 Conference of the North American Chapter of the Associ- at 91.79%, 91.51%, and 91.65% respectively. In addition, ation for Computational Linguistics: Human Language Technologies, 2016. [13] K. Chaovavanich. Dictionary-based Thai Word Seg- mentation using maximal matching algorithm and Thai Character Cluster (TCC) (newmm) [Online]. Available: https://github.com/PyThaiNLP/pythainlp. [14] D. Rumelhart, G. Hinton, and R. Williams, ”Learning representations by back-propagating errors,” Nature 323, no. 6088, pp. 533–536, October 1986. [15] S. Hochreiter and J. Schmidhuber, ”Long Short-Term Memory,” Neural Computation 9, no. 8, pp. 1735–1780, 1997. [16] T. Mikolov, I. Sutskever, K. Chen, G.S. Corrado, and J. Dean, ”Dis- tributed Representations of Words and Phrases and their Composi- tionality,” Proceeding NIPS’13 Proceedings of the 26th International Conference on Neural Information Processing Systems - Volume 2 Pages 3111-3119, Nevada, 2013 153

2019 4th International Conference on Information Technology (InCIT2019) [17] J. Howard and S. Ruder, Universal Language Model Fine-tuning for Text Classification., in ACL, 2018. [18] S. Merity, N. S. Keskar, and R. Socher, Regularizing and Optimizing LSTM Language Models., CoRR, vol. abs/1708.02182, 2017. [19] N. Reimers and I. Gurevych, Optimal Hyperparameters for Deep LSTM-Networks for Sequence Labeling Tasks., CoRR, vol. abs/1707.06799, 2017. [20] C. Polpanumas. Thai2Fit [Online]. Available: https://github.com/cstorm125/thai2fit. 154

2019 4th International Conference on Information Technology (InCIT2019) Surveillance System for Abnormal Driving Behavior Detection Parkpoom Chaisiriprasert Karn Yongsiriwit College of Digital Innovation and Information Technology College of Digital Innovation and Information Technology Rangist University Rangist University Pathum Thani, Thailand Pathum Thani, Thailand [email protected] [email protected] Abstract— The objective of this research is to detect detects the behavior that will cause danger by focusing on the face. The purpose of this system is to estimate the direction abnormal car driving behaviors that may result in accidents. of the eye to detect dangerous behavior in which the detection The behaviors include exhaustion from driving for a long time, direction in the experiment is divided into three parts: head direction of movement and accuracy of behavior monitoring taking eyes off the road, and the abnormal eye-blink from the from the results that are accurate and accurate sleepiness. The researchers are interested to develop a mobile application that analyzes car driving behaviors using a face Heming Zhang, Xiaolong Wang, Jingwen Zhu, and C. detection algorithm. Our application is easy to install and to use on any Android based smartphone. Concretely, our mobile Jay Kuo [2] studied the face detection using a telephone application considers three factors: (1) time of driving; (2) device. The face detection is widely popular in this research. abnormal eye-blinking; and (3) taking eyes off the road. We They present a face detection frame using neural networks integrate these factors with the acceleration of the driving algorithms. The results of the experiment are in the standard, vehicle to analyze and calculate abnormality level. Therefore, we send a voice notification to the driver when the calculated WIDER-face and FDDB are shown to be satisfactory and abnormality level exceeds 10. We perform experiments and Faster speed compared to state-of-the-art experimental results show that our mobile application is feasible in realistic situations. Quanyou Zhao and Shujun Zhang [3] presents the detection of face direction. Firstly, they detect the face by Keywords- Driving Surveillance, Driver Behavior, Machine filter the background from the foreground based on the Learning, Facial detection, Smart phone detected face color. Thereafter, the face can be detected using AdaBoost Model. The experiment shows that their approach I. INTRODUCTION works effectively. At present, car accident is one of the most common Chin-Lun Lai, Jun-Horng Chen, Jing-Ying Hsu, and issues around the world, especially in Thailand. The mortality ChihHong Chu [4] propose a face detection approach based rate from the car accident in Thailand during 2017 was 36.2 on the analysis of spatial and temporal elements. The face per hundred thousand of people. The survey from the Road detection can be easily applied in order to increase reliability. Safety Center shows that the most common cause of car accidents during Thai new year week is drunkenness at Memorizing pages using spatial spectrum analysis and 36.59%, then the poor driving visibility at 18.53. detection simple edge, the approach will clearly segregate faces whether a person's face or not. The experimental results To detect the behaviors that will cause accidents during show that this method can be implemented in portable driving and prevent the accident not to occur is crucial. The warning signs for drowsiness or anxious emotions, for devices. example, eye-blinking and taking eyes of the road, should be Ruian Liu, Mimi Zhang, and Shengtao Ma [5] studies the considered. The reason is because such behaviors cause the drivers to lose attention on the road, thus an accident could design of face detection and tracking systems based on occur at anytime. Adaboost's face detection theory. They develop with Visual The researcher therefore developed the driving behavior C++ programs in Windows platform. This system recognizes detection system. The system works as an application on the function of face detection and increases detection rates smartphones. The system will detect the current face and speed with the integration of the MIT face database and condition of the driver. Thus, the system will analyze the face the custom-built face database. Furthermore, they use the and the eyes of the driver. Concretely, the system firstly detects whether the driver’s eyes are in normal condition by average adjustment algorithm (Camshift) to track faces. This measuring the size of both eyes compared to the average eye work improves the algorithm for Adaboost face detection by size of the driver. Secondly, the system detects the abnormal using a detection method with detection to increase face eye-blinking. Whenever, the driver’s eyes and face are not in tracking performance. a normal condition, the system will notify to the driver with sound. Thereafter, the system will recommend nearby rest Zheng Jun, Hua Jizhao, Tang Zhenglan, and Wang Feng areas for the driver to take a break. [6] studies on the face detection approach by identifying the face color using LBP Operators formula. The approach II. RELATED WORK determines the number of pixels of the skin that is larger than the threshold to detect the face region. Their experiment Kenji Miyoshi, Hiroki Nomiya, and Teruhisa Hochin [1] shows that the face detection approach using LBP is feasible have presented the detection of dangerous behavior by with high accuracy. estimating gestures and directions of drivers. The system 155

2019 4th International Conference on Information Technology (InCIT2019) III. RESEARCH METHODOLOGY We present in this section the approach for detecting abnormal car driving behaviors focusing on the driver’s face. The approach is divided into 5 steps as follows: A. Data preparation and processing steps Fig 3. Example of facial landmarks obtained from training data Data preparation from the library dlib (Davis King), D. Processing the direction of movement of the face allows the detection of the face region from the image Our system draw the distinctive features on the face received from the front camera of the smartphone. The system detect the face region by applying Histogram of which will get various points throughout the face to be oriented gradients and Linear Support Vector techniques displayed so that the system can detect the direction of integrated with facial data from iBUG 300-W dataset. movement effectively. As shown in Figure 4, the movement of the face in case of displacement from the region. This will cause the face landmarks undetectable as depicted in Fig. 5. Fig 1. Face area detection Fig 4. Example of facial landmarks when movement B. Spatial analysis for finding faces To find the face, we use the default value of the library dlib which devided the image to 25 zones (5x5). The 25 zones are used to determine which position of the zones contain the eye, eyebrows, nose, and mouth. For example in Fig.2, the left eye is in zone at the position 7. The right eye is in the zone at the position 9. Some zones without the face are not needed, thus they are cut by applying hard-negative mining by training with the iBUG 300-W dataset. Once this step has been passed, only the face zones can be identified. Fig 5. Example of facial landmarks can not detection Fig 2. zone code 5x5 E. Blink behavior processing C. Finding facial features In this work, in addition to the face detection system, we analyze the blinking behavior of the eyes to detect abnormal This step will draw the facial features of the face, which behavior. Concretely, the system continuously determines the will consist of the face, eyebrows, eyes, nose, and mouth. number of times that the driver’s eye blinks. If the value is Using the facial landmarks of the library dlib that has been greater than the threshold, the system will send a warning trained with iBUG 300-W, the dataset will show the various sound to the driver. We detect the eye blinking by measuring points of the face. As shown in Fig. 3. the size of the eye using Eye aspect ratio (EAR) having P1 to P6 are each position of the eye (Tereza Soukupova and Jan Cech [7] in Fig.6) using the following formula: ‫ ܴܣܧ‬ൌ หȁܲʹ െ ܲ͸ȁห ൅ ȁȁܲ͵ െ ܲͷȁȁ ʹȁȁܲͳ െ ܲͶȁȁ 156

2019 4th International Conference on Information Technology (InCIT2019) button is used to stop the application. The driver can press the red button again to start using the application. Fig 6. Eye aspect ratio (EAR) [7] A. System notification conditions IV. THE WORKING PRINCIPLE OF THE SYSTEM We consider 3 factors of abnormal behavior to detect and The main operation screen, when starting to use, the prevent the risk of car accidents. The detail of each factors application will switch to the camera mode. The application are as follows: will perform real-time detection to monitor the behavior of abnormal behavior by analyzing the face, eyes and the Factor 1: blink of eye. The criteria for blinking eyes are not duration of driving. The screenshot is shown in Fig. 7. more than 20 times or have closed eyes for more than 2 seconds. Fig 7. The main operation screen Factor 2: Taking your eyes off the road while driving for 2 From the main operation screen, the white bar on the seconds, such as eating food. Talking on the phone, smoking, screen on the right-hand side of the camera displays the sum and more. Dr. David Hurwitz said from the results of the of the abnormal value. The value is calculated by considering study that having the driver's distracted from looking at the three factors: (1) time of driving; (2) abnormal eye-blinking; and (3) taking eyes off the road. Whenever, the value is road for 2 seconds or longer can increase the risk of an exceeding 10, the bar will reach the maximum. Thus, a accident (Dr. David Hurwitz) warning sound will notify to the driver Factor 3: Driving for a long time. The driver should take a On the left-bottom of the screen, the three factors as rest from driving the car every hour otherwise the driver mentioned above are display as colored bars. Firstly, the red would have fatigue and reduce the driving capability. bar indicates the time of driving. We have set the time limit for driving to be 5 hours, thus the red bar ranges from 0-5. By considering factors 1) blinking 2) taking eyes off the Secondly, the green bar indicates abnormalities of eye- road and 3) driving for a long time. For example, when blinking. The normal eye-blinking is every 5 seconds, or an starting driving for a period of 1 hour, it may result in taking average of 12 to 20 times per 1 minutes. If the number of eye- eye off the road 1 time, but there is no abnormal blink of an blinking exceeds these values, therefore it is abnormal. eye. When the accumulated value is calculated, the value is Finally, the blue bar indicates whether the driver focusing on at level 2 which is in the criteria that the system does not have the road or not by detecting the face landmarks. These status bars will increase based on their own calculated values in the to send sound notification. Example for accumulation of timely manner. However, if the driver has no longer been abnormal behaviors is shown in the following table. detected with these abnormal factors within 60 seconds, the bars will be reduced. TABLE I The value of the accumulated behavior The black box represents the velocity of the car. In case Risky behavior Accumulation there is no movement of the car, it is set to zero. The red of abnormal Driving for Abnormal Take your eyes a long time Blink off the road behavior 1 - 1 2 1 - 2 3 1 - 5 6 … … … … 5 2 3 10 The system will alert as a sound (Fig. 8) when the abnormal behavior detected by each of the factor value reach level 5 or the accumulated value from 3 factors reaches the level 10. The sound alert will continue unless the driver presses the button to stop the alert. Thereafter, the driver can select to restart the system, find nearby rest areas, change the setting, and read the manual as shown in Fig. 9. 157

2019 4th International Conference on Information Technology (InCIT2019) wrong detection but the system alert is FN. In the experiment, we simulate 10 scenarios of driving as shown in Table 2. Table 2 Experimental results with precision and recall Risky behavior Driving Abnor Take Precision Recall for a long mal your Blink eyes off 0.76 0.93 time the road 0.78 0.97 (Hours) - 0.91 0.97 5 5 0.94 1.00 1 4 - 0.94 1.00 1 4 4 0.97 0.93 2 3 1.00 0.93 2 3 0.97 1.00 3 3 0.94 0.97 4 0.97 1.00 3 4 0.92 0.97 1 4 4 2 Average 3 4 2 4 4 5 3 5 Fig 8. Notification screen The results of the experiment showed that the precision from all the 10 scenarios is in the range of 0.76 to 1.00. Fig 9. Menu display screen Moreover, the recall is in the range of 0.93 to 1.00. From the average precision and recall values as 0.92 and 0.97 V. EXPERIMENTAL RESULTS respectively, show that our approach is feasible. For testing the accuracy of risky behavior detection during vehicle driving, the accuracy and accuracy of the VI. CONCLUSION equation formula are used as follows. This work presents the driving surveillance system by ܶܲ detecting facial behavior. The system works as a mobile ”‡…‹•‹‘ ൌ ሺܶܲ ൅ ‫ܲܨ‬ሻ application to detect face, eyes-blinking, and driving time. When the behavior of any one factor reaches the level 5 or ܶܲ behavior of all three factors combined reaching the level 10, ܴ݈݈݁ܿܽ ൌ ሺܶܲ ൅ ‫ܰܨ‬ሻ the system will sound a loud alarm. The system also has a TP is the correct detection and the system alerts. FP is map menu for finding nearby places to rest. From the test of the correct detection, but the system does not alert. For the the system, the precision is 0.92 and the precision is 0.97 which is reliable and acceptable. Forevermore, working with the library dlib algorithm, the performance is satisfactory. For the future development, we would add more functionality to the system and consider more criteria to improve the detection of abnormal behaviors. REFERENCES [1] Miyoshi, K., Nomiya, H., & Hochin, T. (2018). Detection of Dangerous Behavior by Estimation of Head Pose and Moving Direction. 2018 5th International Conference on Computational Science/ Intelligence and Applied Informatics (CSII), 121-126. [2] Zhang, H., Wang, X., Zhu, J., & Kuo, C.J. (2018). Accelerating Proposal Generation Network for Fast Face Detection on Mobile Devices. 2018 25th IEEE International Conference on Image Processing (ICIP), 326- 330. [3] Zhao, Quanyou & Zhang, Shujun. (2011). A face detection method based on corner verifying. 10.1109/CSSS.2011.5974784. [4] Lai, C., Chen, J., Hsu, J., & Chu, C. (2013). Spoofing face detection based on spatial and temporal features analysis. 2013 IEEE 2nd Global Conference on Consumer Electronics (GCCE), 301-302. [5] Liu, R., Zhang, M., & Ma, S. (2010). Design of face detection and tracking system. 2010 3rd International Congress on Image and Signal Processing, 4, 1840-1844. [6] Jun, Z., Jizhao, H., Zhenglan, T., & Feng, W.H. (2017). Face detection based on LBP. 2017 13th IEEE International Conference on Electronic Measurement & Instruments (ICEMI), 421-425. [7] Soukupová, T., & Cech, J. (2016). Real-Time Eye Blink Detection using Facial Landmarks. 158

2019 4th International Conference on Information Technology (InCIT2019) Predicting Short Trend of Stocks by Using Convolutional Neural Network and Candlestick Patterns Kietikul Jearanaitanakij Bundit Passaya Department of Computer Engineering Department of Computer Engineering Faculty of Engineering Faculty of Engineering King Mongkut’s Institute of Technology Ladkrabang King Mongkut’s Institute of Technology Ladkrabang Bangkok, Thailand Bangkok, Thailand [email protected] [email protected] Abstract—Candlestick chart pattern is a technical tool that investment. Z. Quan and C. Tsai [5] used the content-based encapsulates the price of the asset for multiple time frames into image retrieval technique to extract three different features a single price bar. The expertized trader can predict the price from candlestick patterns. The system then retrieved similar trend of the asset by looking at the pattern of some adjacent historical candlestick patterns represented by low-level candlesticks. This paper proposes the architecture for features and predicted the movement of the stock based on the predicting the short trend of the stocks by using the past information. The experiment on the 10-year dataset convolutional neural network and the candlestick patterns. The collected from the Dow Jones Industrial Average Index experiments are conducted with a set of candlestick pattern indicated a fairly good accuracy with 75.75% F-score. images collected from various stocks in the stock exchange of Thailand (SET). Each image captures six to twelve adjacent This paper proposes the method for predicting the short- candlesticks. The experimental results indicate that the term movement of the stock price by applying the proposed method can correctly predict the short trend for most convolutional neural network (CNN) to the candlestick stocks with acceptable accuracy. In addition, the proposed pattern. Some preliminary runs are performed on various architecture achieves better accuracy and training time than architectures of CNN to determine the best setting of that of the well-known architecture, ResNet-18. prediction system. In addition, the results in the experimental section show the comparison between the proposed method Keywords—Candlestick; prediction; convolutional neural and the well-known ResNet-18 architecture. network; deep learning; stock exchange of Thailand The rest of this paper is organized as follows. In section II, I. INTRODUCTION some basic concepts of the candlestick chart patterns are explained. In addition, this section also describes the Candlestick is an ancient Japanese technical analysis, characteristics of the candlestick dataset collected from the originally used in trading rice in 1600's. Munehisa Homma is stock exchange of Thailand. Section III proposes the the legendary Japanese pioneer who was exceptionally architecture of CNN that is suitable to the problem. famous person in using the past prices to predict the future Experimental results and comparison with another price movements [1]. Candlestick pattern provides many architecture of CNN are given in Section IV. Finally, section useful trading signals. It can be used alone or in combination V provides the conclusion and the possible future work. with other technical analysis tools. Candlestick chart is more visually attractive and easier to interpret than the traditional II. CANDLESTICK CHART PATTERNS bar charts. It is also easy to observe the price action in a certain period. A. Candlestick Bar One bar of candlestick can tell us about the price There has been much research about applying candlestick patterns in predicting the stock behaviors. H. Li et al. [2] movement in a certain timeframe. trained Radial Basis Function Neural Networks (RBFNN) with localized Generalization Error Model (L-GEM) to 1. The opening price: If the candlestick bar is in green enhance candlestick pattern, i.e., morning star pattern. Their color (as shown in the left bar of Fig. 1), the opening work can prevent up to 69% of false prediction of the morning price locates at the bottom side of the rectangle. On star pattern on dataset are collected from Shenzhen stock the other hand, if the bar is in red color, the opening market. M M. Goswami et al. [3] combined Self Organizing price is at the top side of the rectangle. In case that the Map with Case Based Reasoning to identify profitable opening price and the closing price are same, the candlestick patterns in a short-term price fluctuation of candlestick bar will be represented by a short yellow horizontal line since the candlestick bar does not have National Stock Exchange of India (NSE). They compared the a body. method with other existing methods and claimed that their method was an expert knowledge based and does not depend 2. The closing price: The closing price indicates the on data. W. Xiao et al. [4] improved their previous work [2] price of the stock at the end of the time frame. It by using a Multiple Classifier System (MCS) to predict the locates at the opposite position of the opening price. trend of a stock price based on multiple candlestick patterns. For example, if the opening price is at the top side of The Localized Generalization Error Model (L-GEM) was still the red candlestick bar, the closing price will be at the used for training the Radial Basis Function Neural Networks bottom side of the same bar. (RBFNN) in the system. Their experimental results show that the improved method statistically outperforms a random 3. The highest/lowest prices: The line piercing through the center of the candlestick body is considered as the 159

2019 4th International Conference on Information Technology (InCIT2019) candle wick. The upper and the lower tips of the adjacent candlestick bars. Therefore, those basic patterns candle wick represent the highest and the lowest mentioned above can at least persuade the readers that a series prices of the stock in the same time frame, of adjacent candlesticks can influence the short trend of stock respectively. movements. Fig. 1. Two examples of the candlestick bar The dataset used in this paper composes of 1800 instances of candlestick pattern images collected from the stock B. Candlestick Chart Patterns exchange of Thailand (SET). Those instances are uniformly Most people watching candlestick chart perhaps look for distributed over 3 possible classes, e.g., Up, Down, Sideway. Each color image (40x40 pixels) is associated with one of the reversal patterns of the stock price. A trend reversal signal possible short-trend classes {Up, Down, Sideway}. The short- implies that the prior trend is likely to change, e.g., up, down, trend period means the trend of the stock price within a couple no change. There are many candlestick reversal patterns [1]. of weeks. If the stock price moves above the highest price of Some well-known patterns can be listed as follows along with the current pattern, the trend of the stock is ‘Up’. On the their examples in Fig. 2. Although our research does not contrary, if the stock price moves below the lowest price of predict these patterns, it is nice to know about their behaviors. the current pattern, its short trend is ‘Down’. Lastly, ‘Sideway’ short trend means that the price of the stock moves 1. Hammer and Hanging Man: These two patterns share within the highest and the lowest prices in the current pattern. the same shape of a candlestick, i.e., long lower In order to reinforce the candlestick patterns, we capture only candle wick and small real body, but different places patterns that have the volume above 14-days simple moving in the trend. Hammer candlestick appears at the average. bottom of the trend, while hanging man candlestick occurs at the top. Fig. 2. Examples of some candlestick chart patterns Fig. 3. Sample of candlestick patterns captured from stocks in SET 2. Engulfing: This pattern is a major reversal signal with Fig. 3 shows samples of candlestick patterns from 3 two opposite color real bodies. If a green real body classes. Some patterns are similar to the major candlestick wraps around, or engulfs, the previous red real body, patterns explained in the section II-B. However, many of them the pattern is likely to be reversed in the uptrend. On are not even close to those major patterns. Therefore, this the other hand, if the red real body engulfs the dataset will be a challenge for using CNN to predict the short previous green one, the downtrend is possible to trend of the stock price. occur. III. PROPOSED ARCHITECTURE 3. Star: A star is represented by a small real body that The steps in searching for the architecture of CNN for the gaps away from the previous large real body. It can candlestick patterns dataset can be described as follows. occur at both the bottom (morning star) and the top Step 1: Compare the accuracy results among different number (evening star) of the trend. of convolutional layers, each with only one fully connected layer, by performing preliminary runs for 100 epochs. Then There are still plenty of other candlestick patterns which choose the best one which achieves the best accuracy. have not been listed here due to space limitations. It is worth Step 2: Vary the number of fully connected layers of the best to note that the purpose of this paper is not to identify the architecture in step 1 and pick the best architecture which candlestick pattern. Instead, this paper aims to study how well produces the best accuracy in the preliminary runs. CNN can predict the short trend of the stock by analyzing an image of the candlestick pattern which contains six to twelve 160

2019 4th International Conference on Information Technology (InCIT2019) Step 3: Conduct the full experiment on the best architecture in step 2 for 2,000 epochs by using 3-fold cross validation. Afterward, measure the test accuracy on the unseen patterns. For each of 3 classes, the number of patterns in the cross validation set and the test set are 450 and 150, respectively. Each set contains patterns which are uniformly distributed over 3 classes. The experimental results from preliminary runs are shown in Tables I and II, while the best CNN architecture derived from above steps is presented in Fig. 4. TABLE I. ACCURACY FROM PRELIMINARY RUNS BY USING A FEW CONVOLUTIONAL LAYERS FOLLOWED BY SINGLE FULLY CONNECTED LAYER Number of Convolutional Layers Accuracy (%) Fig. 4. The best CNN architecture for the candlestick patterns dataset 2 51.12 TABLE III. EXPEIMENTAL CONDITIONS Condition Value 3 55.17 Tool PyTorch 0.4.1 4 52.60 Loss function Cross entropy Resilient backpropagation TABLE II. ACCURACY FROM PRELIMINARY RUNS BY USING 3 Optimizer CONVOLUTIONAL LAYERS FOLLOWED BY A FEW FULLY CONNECTED Learning rate 0.01 LAYERS Training epochs 2000 40 Number of Fully connected Layers Accuracy (%) Batch size Convolution stride 1 1 55.17 Max Pooling stride 2 2 56.64 Max Pooling 2x2 window size 3 56.01 Intel Core i5, 12 GB RAM Machine GTX-1060, 6GB RAM Table I indicates that 3 convolutional layers produces the best GPU Best so far accuracy among other candidates on the unseen patterns. Thus, the number of convolutional layers is fixed at 3 and the Final architecture number of fully connected layers is varied from 1 to 3, as shown in Table II. As a result, the final CNN architecture for Cross Entropy Loss the candlestick patterns dataset contains 3 convolutional layers and 2 fully connected layers, as shown in Fig. 4. Since 3 the image size in the dataset is small (RGB, 40x40 pixels), the 2 kernel sizes of the first and the second convolutional layers are 1 defined as 3x3 and 2x2, respectively. In addition, each 0 convolutional layer is followed by a ReLU activation function and 2x2 max pooling layer, respectively. It is worth to note 1 that the kernel size of the third convolutional layer is 1x1. This 107 kernel can drastically reduce the number of parameters by 213 reducing the number of channels from 12 to 3, yet still 319 preserve the extracted features in spatial input. As a result, the 425 last convolutional layer does not require the max pooling layer 531 like the previous two layers. 637 743 IV. EXPERIMENTS 849 In order to derive the stable results, the experiments are 955 conducted by using three-fold cross validation. The dataset is 1061 mixed and uniformly distributed on three output classes (Up, 1167 Down, Sideway). For each class, 150 patterns are isolated as 1273 an unseen test set, while the rest 450 patterns are used in the 1379 process of three-fold cross validation. All experimental 1485 conditions are listed in Table III. 1591 1697 1803 1909 Proposed ResNet Fig. 5. Cross Entropy Loss between two architectures Fig. 5 shows a plot of the training loss (cross entropy) after the end of each epoch through out the training period in a fold of the cross validation. The loss value of the proposed architecture (solid line) at the beginning of the training period is pretty high because the proposed network is trained from scratch. However, the network quickly reaches the convergence in the latter epochs with the loss value less than 1. In order to check the predicting ability after reaching the convergence, the network is evaluated by the validation set. 161

2019 4th International Conference on Information Technology (InCIT2019) Table IV shows the validation accuracy for each fold of the the number of trainable parameters in the proposed method. dataset. The accuracies from 3 folds are quite dissimilar so we The candlestick images are rescaled to 224x224x3 so that they use the average value 81.06 % as the validation accuracy. can be able to be fed into ResNet-18. This comparison provides the reader a feel of how well the proposed TABLE IV. VALIDATION ACCURACY OF THE PROPOSED architecture can classify the dataset comparing to the famous ARCHITECTURE CNN architecture. The comparisons between two architecture are shown in Fig. 5 and Table V. ResNet-18 has similar cross Fold number Validation Accuracy (%) entropy loss pattern as the proposed architecture but always has higher value. It is interesting that the proposed architecture 1 88.18 produces the test accuracy better than ResNet-18 by 13.29%, as shown in Table V. In addition, the training time of the 2 80.45 proposed architecture is significantly smaller than ResNet-18, even though the number of parameters needed to be trained is 3 74.55 approximately equal. The reason is that ResNet-18 composes of 18 layers. Therefore, it takes time to wait for the signal to Average 81.06 be ready at the fully connected layers. SD 6.83 V. CONCLUSION It is nice to visualize how well the proposed architecture The CNN architecture for predicting the short trend of can extract features from the original image. Fig. 6 illustrates stocks in the stock exchange of Thailand by using the feature maps and max-pooling outputs from the first channel candlestick pattern images is proposed in this paper. The of all convolutional layers in CNN. The red candlesticks are proposed architecture performs fairly well in predicting the extracted as the white bars, while the green candlesticks are in unseen patterns. It can predict the short trend of the stock from dark grey. The number of features is drastically reduced, yet the image input containing 6 to 12 adjacent candlesticks the candlestick pattern is still preserved. The images from the without knowing the pattern of those candlesticks in advance. last two layers may look very dark but they do have embedded The proposed architecture also produces better test accuracy features which are difficult to discern by human eyes because when compared with the well-known architecture, ResNet-18. the size of the images is so small (9x9 pixels.) Therefore, the One possible future work is to extend the proposed proposed architecture can effectively capture the important architecture to consider the volume of each candlestick along features from the original image. with its pattern. This can increase the predicting accuracy since the volume plays an important role on the stock trend. Fig. 6. Feature maps and max-pooling layers from the proposed CNN architecture REFERENCES TABLE V. COMPARISON BETWEEN PROPOSED ARCHITECTURE AND [1] S. Nison, Japanese Candlestick Charting Techniques: a contemporary RESNET-18 ARCHITECTURE guide to the ancient investment technique of the Far East, New York Institute of Finance, USA, 1991, pp. 1-16. Criterion ResNet-18 Proposed [2] H. Li, W. W. Y. Ng, J. W. T. Lee, B. Sun, D. S. Yeung, “Quantitative Test accuracy 57.92 % 65.62 % Study on Candlestick Pattern for Shenzhen Stock Market,” IEEE International Conference on Systems, Man and Cybernetics, Training time (minute) 620 150 Singapore, pp. 54-59, October 2008. Number of trainable 30,900 30,846 [3] M M. Goswami, C K. Bhensdadia, A P Ganatra, “Candlestick Analysis parameters based Short Term Prediction of Stock Price Fluctuation using SOM- CBR,” IEEE International Advance Computing Conference, India, pp. The last step is to check the generalization of the network 1448-1452, March 2009. with an unseen test set. The test accuracy of the proposed network is 65.62% which is fairly acceptable since it is much [4] W. Xiao, W. W. Y. Ng, M. Firth, D. S. Yeung, G. Cai, J. Li, B. Sun, better than the random guess with one-third accuracy from 3 “L-GEM based MCS aided candlestick pattern investment strategy in classes. the Shenzhen stock market,” International Conference on Machine Learning and Cybernetics, China, pp. 243-248, July 2009. As the final experiment, the proposed architecture is compared with the well-known ResNet-18 (Residual [5] Z. Quan, C. Tsai, “Stock prediction by searching similar candlestick Network) architecture [6]. ResNet-18 is composed of a series charts,” IEEE International Conference on Data Engineering of Residual blocks which can effectively compress the Workshops (ICDEW), Australia, pp. 322-325, April 2013. original input image. As a conventional usage of ResNet-18, the pretrained parameters of ResNet-18 are preserved during [6] K. He, X. Zhang, S. Ren, J. Sun, “Deep Residual Learning for Image the training period. One fully connected layer with 60 neurons Recognition,” IEEE Conference on Computer Vision and Pattern is inserted between the output layer of ResNet-18 (512 Recognition (CVPR), Las Vegas, USA , pp. 770-778, June 2016. neurons) and the output layer (3 neurons). This comparison is fair because the modified ResNet-18 architecture requires 30,900 trainable parameters which approximately equals to 162

2019 4th International Conference on Information Technology (InCIT2019) Improved TTC per Fuel Cost with DGs by Using Evolutionary Programming Suppakarn Chansareewittaya Mahamah Sebakor School of Information Technology School of Information Technology Mae Fah Luang University Mae Fah Luang University Chiang Rai, Thailand Chiang Rai, Thailand [email protected] [email protected] Abstract— The using of the evolutionary programming - The electrical power capacity of this device must be (EP) is presented to optimize the allocation of distributed in the range of 50 MW to 100 MW. generation (DG) units. The contents of this paper are to increase the power transmission capacity by considering the There are many types of DGs such as gas turbines plant, biomass plant, fuel cells plant, wind turbine farm, and solar ratio of the total transfer capability (TTC) and the economic energy farm [5]. These DGs are used to evaluate for their dispatch (ED). Moreover, the ratio of TTC and the installation optimal parameters. After that, these optimal parameters are cost of DG is evaluated to observe the worth of operation. The applied to DGs to enhancing the operation of the power allocations of DG are type, location, and electrical power system [6]. This means the installation of DGs requires the optimal allocations to reach maximum performances. The capacity. The two types of DG, solar energy wind are used in optimal allocations of DG mean the suitable type, location, this paper. The modified IEEE 30-bus system is selected to and electrical power capacity. The performances mean demonstrate this using the EP. Test results indicate that the maximum total transfer capability (TTC), minimum losses, allocation of DGs from EP can increase TTC from the base minimum TTC per cost of installation of DG, minimum TTC per fuel cost of generators and, etc. case. Moreover, the best ratio of the TTC and cost of installation of DG and the ratio of TTC per ED are evaluated. There are many pieces of research that present the This means the installation of DGS with optimal allocation by optimal allocations of DGs. In [7], the chaotic free-search using EP can improve the TTC per fuel cost. The maximum (CFS) algorithm is used to optimize DG. Test result shows that the proposed algorithm performs fast convergence performance of the power system is reached with the best property and preciseness. MATLAB/Simulink is used to TTC per fuel cost without any violence in the power system. construct the distribution power system. Two distributed generation is constructed and connect to the feeder. The Keywords—distributed generation, optimization, intelligent electronic device (IED) is set to observing [8]. In conclusion, IED performs the usage of a protection evolutionary programming, economic dispatch, heuristic apparatus and maximized operation performance well. P. A. Souza et al. [9] analyze the injection of active and reactive method powers from photovoltaic generation systems distributed to the distribution network. The real Brazilian systems are I. INTRODUCTION used as a test system. At the present day, the demanding of electricity is increased year by year. The responsibilities of the operator The proposed of this paper is the usage of EP for to are various such as build new power plants and extend the evaluate the optimal type, location, and electrical power electrical transmission network. One of the popular choices capacity of multi-type DG to enhance TTC. Moreover, the is to install Flexible Alternating Current Transmission main objective function is to evaluate the maximum ratio of System (FACTS) controllers [1]. FACTS controller is the TTC per fuel cost of the generators. The ratio of TTC per electrical and electronics based device that can adjust their installation cost of DG is evaluated in parallel. Two types of parameters [2]. The advantage of FACTS controllers is DG are used. The First is photovoltaic (PV). The second is these controllers do not release the pollutions. However, one a wind turbine (WT). The optimal power flow is defined to of the disadvantages of FACTS controllers is these use for the power flow in this paper [10, 11].The modified controllers are not increased electrical real power directly to IEEE 30-bus system is used to demonstrate this propose. the power system. SVC can only be injected or absorb the reactive power (var) to the power system [3]. Other FACTS II. PROBLEM FORMULATION controllers can only adjust and apply their parameters to the line or bus of the power system to improve the A. The general purposed of Economic Dispatch performances. One of the well-known problems of the electrical power There is another interesting choice to respond to this filed is the economic dispatch (ED) [12]. ED presents the demanding. This choice is to install distributed generation benefit per operation cost. According to the general (DG) into the power system. There are many definitions of proposition of electrical power system engineering, ED is DG such as an independent modular power generation used to present the way to minimize the fuel cost of all system. CIGRE Working Group [4] defines the definitions generators in the power system. However, the TTC must be of DG are as follow. generated enough to the load demand of the power system. - This device must not be a central power generation system that requires power to load. - This device must be connected to the power distribution system only. 163

2019 4th International Conference on Information Technology (InCIT2019) There are two types of constraints. The first is equality. The Qmin , QGmiax the lower and upper limit second is inequality constraints. These constraints are used Gi reactive power of generator at to control the limit of operation for safe conditions. bus i, The general objective function of ED is expressed in (1) Vi ,Vj voltage magnitudes at bus i and [13]. j, mm (1) Vimin ,Vimax ,Vjmin ,Vjmax the lower and upper limit values   ( )ED = Fi ( Pi ) = ai Pi2 + bi Pi + ci of voltage magnitude at bus i i=1 i=1 and bus j, Where ED the total fuel cost of generators, δi ,δj voltage angles of bus i and j. Fi the cost function of the ith generator, C. Power balanced constraint with DGs Pi real power of the ith generator, m number of generators in the electrical The power balanced constraint with DGs equation is power system, and expressed in (9). ai, bi, and ci the cost coefficients of the ith generator. NDG (9) B. Total Transfer Capability: TTC  PG − PDG,i = PD + PL The definition of TTC is the amount of electric power NG i=1 that can be transferred over the interconnected transmission Where network in a reliable manner [14-17]. Power means real and reactive power. This power will be flow in the power system NG total number of generator, network. In addition, the equality constraint such as power equation is used. Moreover, the inequality constraints used PG power of each generator, to control the limitation of each parameter [18]. The NDG total number of DG units, equation of TTC determining is expressed in (2). PDG,i real power of generator from DG at bus ith, PD total load demand, and PL total real power loss. D. TTC per ED ND NL In this paper, the TTC and ED are evaluated. These 2  TTC = PDi + PLi (2) values are used to calculate the ratio of power and fuel cost. i=1 i=1 This ratio is defined as the benefit of operation. This means Subject to the high ratio requires less cost to operation. The equation of this ratio is expressed (10). N (3) max F1 = TTC (10) ED PGi − PDi + ViV jYij cos(θij − δi + δ j ) = 0 j =1 N (4) The standard unit of TTC is MW. The general unit of ED is $/h. Thus, the unit of the objective function is QGi − QDi + ViV jYij sin(θij − δi + δ j ) = 0 MWh/$. The meaning of this objective is the highest TTC j =1 in one hour per 1$. PGmiin ≤ PGi ≤ PGmiax ∀i ∈ NG (5) E. TTC per cost of DGs QGmiin ≤ QGi ≤ QGmiax ∀i ∈ NG (6) The ratio of TTC per installation cost of DGs is expressed in (11). The aim of this evaluated ratio is used to Vimin ≤ Vi ≤ Vimax ∀i ∈ N (7) observe the benefit of the installation cost of DGs. The technical details of DG which are used in this paper are V min ≤Vj ≤ V max ∀i ∈ N (8) shown in Table I [6], [19]. j j PDi real power of ith load, TABLE I. TECHNICAL DETAILS OF DG PLi real power of ith losses, PGi real power of generator at of ith Type IC OC Commercial Plant ($/MW- ($/MWh) size (KW) factor generator Wind (%) QGi reactive power of generator of PV year) 10.9 200 0.0 100 20 ith generator 206,000 25 618,000 F2 = TTC (11) CDG Pmin , Pmax the lower and upper limit power Gi Gi of generator at bus i, 164

2019 4th International Conference on Information Technology (InCIT2019) Subject to ximin , ximax the lower and upper limit value of the ith u individual, and ND _ SNK is [0,1].  CDG = CPV1 ⋅ IC j ⋅ PDG,ij i=1 j∈Tech (12) ND _ SNK 3. Mutation The new offspring is generated by using as (15) and  + CPV2 ⋅ OC j ⋅ PDG,ij ⋅ a j ⋅ 8, 760 (16). i =1 j∈Tech Where cost of DGs ( )x 'k,i = xk,i + N0,σ 2 (15) CDG cumulative present values related to fix k ,i CPV1 cost, investment cost of DG type jth, ( )σk,i =  fmax − fk  ICj capacity of the DG type j at bus ith, ximax − ximin  fmax + ag  (16) PDG,ij cumulative present values related to   CPV2 variable cost, DG technologies used in the study, Where value of the ith individual of the kth j operation cost of DG type jth, x 'k ,i offspring, OC j injected real power of DG at bus ith. DG,i P plant factor of DG unit type j. xk,i value of the ith individual of kth parent, aj III. EVOLUTIONARY PROGRAMMING ( )N0,σ 2 Gaussian random number, The EP is one of the popular modern heuristic methods k ,i [20-23]. EP starts with the initialize of the individuals by randomizing the parameter. The objective function values ximin , ximax the lower and upper limit value of the ith can be evaluated with these individuals. After that, the variables, fitness value is evaluated and the selection is used to keep fk the individual with the best objective function value. The fmax fitness value of the kth individual, mutation is used to change the parameter of each individual a maximum fitness value of the parent, and will be used to evaluate in the next iteration. This g is 0.9, process is repeated until the maximum iteration is reached. present iteration number. The details of the EP are followed. 4. Fitness function 1. Representation of Solution The fitness value will be given by using (17). Each individual contains the variables. The equation fk = Kf * F (17) (13) shows the vector which presents variables. Each variable content real number. The details of the variable are Where fitness value of kth individual, and as followed. fk set as 1. VkT = [PGi ,VGi , PDi , Loci , DGi ] (13) Kf Where individual vector, 5. Selection VkT In this paper, selection method is based on tournament scheme. PGi real power of ith generator excluding generator of slack bus, 6. Stop criteria The evaluation will be stopped if the number of VGi voltage magnitude of ith generator, iteration is reached the maximum number of iteration. PDi real power of of ith load bus, IV. EXPERIMENTAL RESULTS The modified IEEE 30-bus system is selected to Loci locations of ith DG, and demonstrate the proposed method. The base case TTC is 164.300 MW. The base TTC/ED is 0.175MWh/$. The test DGi parameter of ith DG. system diagram is shown in Figure 1. 2. Initialization First, the population is set. After that, each individual is initiated. The randomized method is applied to this step. The equation of initialization is expressed in (14). ( )xi = ximin + u ximax − ximin (14) Where xi value of the ith individual, 165

2019 4th International Conference on Information Technology (InCIT2019) According to the previous tables, the EP can evaluate the optimal parameter of the DGs. All optimal parameters include type, location, and electrical power capacity. These parameters can improve the TTC from the base case. Moreover, the TTC/ED and TTC/CDG are improved. EP can evaluate the best ratio of the TTC/ED and TTC/CDG simultaneal. This means the operation requires the lowest fuel cost to generate TTC with the lowest cost of installation of DGs. Moreover, this operation can operate without any violence of the operation. The details of the optimal parameters of the modified IEEE 30-bus system are shown in Table VI. Figure 1. The modified IEEE 30-bus system TABLE VI. DETAILS OF OPTIMAL PARAMETERS OF THE MODIFIED IEEE 30-BUS SYSTEM The evaluation is set as a batch. Each batch contents 10 runs to evaluate the best objective function value. Moreover, Angle Load Generation Injected the minimum objective function value, average value, and standard deviation value are evaluated. The population size Bus Magnitude Degree MW Mvar MW Mvar Mvar of the EP is 30. The number of competitors in the Selection of EP is 15. The maximum iteration is 100. The generator 1 1.043 0 0 0 221.548 -27.847 0 co-efficiency is shown in Table II [24]. 2 1.04 -5.049 21.7 12.7 22.749 108.803 0 3 0.995 -7.535 4.776 2.388 0 0 0 4 0.989 -9.1 15.124 3.184 0 0 0 5 0.996 -8.631 0 0 0 0 0.188 - 6 0.973 10.394 0 0 0 0 0 - 7 0.962 10.759 45.372 21.691 0 0 0 TABLE II. GENERATOR CO-EFFCIENCY OF IEEE 30-BUS SYSTEM - Unit 8 0.946 11.389 59.7 59.7 0 0 0 1 ai bi ci - 2 ($/MWh)2 ($/MWh) ($/h) 3 9 0.998 13.785 0 0 0 0 0 4 0.0200 2.00 0 5 0.0175 - 6 0.0250 1.75 0 0.0625 10 1.012 15.491 11.542 3.98 0 0 0 0.0250 3.00 0 0.0083 - 1.00 0 11 0.998 13.785 0 0 0 0 0 3.00 0 12 1.041 -14.58 22.288 14.925 0 0 0 3.25 0 - 13 1.097 12.437 0 0 30.494 44.497 0 - 14 1.026 16.115 12.338 3.184 0 0 0 - 15 1.033 16.311 16.318 4.975 0 0 0 The best TTC and ED are 325.9560W and 1917.969 - MWh/$, respectively. The best, average, and minimum of TTC, losses, TTC/ED, and TTC/CDG are shown in Table 16 1.015 15.542 6.965 3.582 0 0 0 III. - 17 1.003 15.932 17.91 11.542 0 0 0 - 18 1.001 17.472 6.368 1.791 0 0 0 - 19 0.988 17.795 18.905 6.766 0 0 0 TABLE III. MAXIMUM, AVERAGE, AND MINIMUM OBJECTIVE - Value FUNCTION VALUE 20 0.993 17.324 4.378 1.393 0 0 0 Maximum - Average Minimum TTC Losses TTC/ED TTC/CDG 21 1.025 16.004 34.825 22.288 0 0 0 Average (MW) (MW) (MWh/$) (MWh/$) CPU time 325.9560 35.8750 0.1705 0.0002834 - (min) 319.8159 27.8045 0.1482 0.0001458 22 1.037 15.908 0 0 36.574 62.973 0 295.2090 18.5300 0.1236 0.0000518 23 1.085 -16.48 3.2 1.6 19.771 45.36 0 0.76 - 24 1.027 16.025 17.313 13.333 0 0 0.042 - 25 1.012 13.612 0 0 0 0 0 - 26 0.976 14.484 6.965 4.577 0 0 0 - 27 1.02 11.584 0 0 39.25 16.409 0 The best optimal values from EP are shown in Table IV. - 28 0.972 10.824 0 0 0 0 0 - TABLE IV. THE BEST OPTIMAL VALUES FROM EP 29 0.978 14.124 4.776 1.791 0 0 0 30 0.954 -15.97 21.094 3.781 0 0 0 Value TTC Losses TTC/ED TTC/CDG V. CONCLUSION Maximum (MW) (MW) (MWh/$) (MWh/$) The EP is used to evaluate the optimal allocation of DG 325.9560 18.5300 0.1705 0.0002834 in this paper. Two types of DG are selected to use. Test results indicate that the optimal allocation of DGs is The CDG is 1153573.534$. The details of the evaluated by using EP. This optimal allocation of DGs can installation of DG are shown in Table V. make the benefit of the operation. Moreover, the installation of DGs with this optimal allocation using EP is worthwhile TABLE V. DETAILS OF THE INSTALLATION DG for future planning. PV Wind Total size (MW) Install at bus No. 17 25 3.379 Installed size (MW) 1 2.379 166

2019 4th International Conference on Information Technology (InCIT2019) REFERENCES [21] S. Chansareewittaya and P. Jirapong “Power transfer capability enhancement with optimal maximum number of facts controllers [1] S. Chansareewittaya, “Enhancing Ratio of TTC per Fuel Cost using using evolutionary programming”, Proceedings of IEEE Evolutionary Programming with UPFC,” Proceeding of IECON2011, Melbourne, Australia, Nov. 2011. International Conference on Business and Industrial Research (ICBIR) 2018, Bangkok, Thailand, May 2018. [22] L. L. Lai, Intelligent System Applications in Power Engineering: Evolutionary Programming and Neural Networks, John Wiley & [2] Y. H. Song and Y. H. Johns, “Flexible AC Transmission System Sons, 1998. (FACTS),” IEE Power and Energy Series 30, 1999. [23] K. Y. Lee and A. E. Mohamed, Modern Heuristics Optimizaion [3] FACTS Terms & Definitions Task Force of the FACTS Working Techniques, New York, John Wiley & Sons, 2008. Group of the DC and FACTS Submittee, “Proposed Terms and Definitions for Flexible AC Transmission System (FACTS),” IEEE [24] S. Chansareewittaya, “Optimal Allocations of FACTS Controllers Transactions on Power Delivery, vol. (4), 1997. for Economic Dispatch using Evolutionary Programming,” Proceeding of the 21st International Computer Science and [4] A. Headley et al, “CIRED working group no 4 dispersed Engineering Conference (ICSEC) 2017, Bangkok, Thailand, generations,” CIRED working group, June 1999. [Online]. November 2017. Available: http://www.cired.be/WG04-Report%20.pdf [5] J. Paska, “Distributed generation and renewable energy source in Poland” Proceedings of the Electric power quality and utilization 2007, Barcelona, 9-11 October 2007, pp. 1-6, 2007. [6] R. Jomthong, P. Jirapong and S. Chansareewittaya, “Optimal choice and allocation of distributed generations using Evolutionary Programming”, Proceedings of the CIGRE-AORC 2011, Chiang Mai, Thailand, October 2011. [7] B. Liu, X. Qian, J. Li, and Y. Zhang, “Optimal sizing of distributed generation based on chaotic free-search algorithm in an island microgrid,” Proceedings of the Chinese Automation Congress (CAC), pp.7103 – 7106, 2017. [8] C. J. Chou and W. P. Hong, “Real time distributed generation monitoring at substation based on feeder IED functions and load profiles,” Proceedings of the 7th International Symposium on Next Generation Electronics (ISNE), pp. 1-3, 2018 [9] P. A. Souza, G. B. D. Santos, and D. B. V. Mariano “Analysis of active and reactive power injection in distributed systems with photovoltaic generation,”Proceedings of the Simposio Brasileiro de Sistemas Eletricos (SBSE), pp. 1 – 6, 2018. [10] G. C. Ejebe, “Available transfer capability calculations,” IEEE Transactions on Power Systems, vol. 13, no. 4, Nov. 1998. [11] G. C. Ejebe, J. G. Waight, S. N. Manuel, and W. F. Tinney, “Fastcalculation of linear available transfer capability,” IEEE Transactions on Power Systems, vol. 15, no. 3, Aug. 2000. [12] S. Chansareewittaya, “Hybrid MODE/TS for Environmental Dispatch and Economic Dispatch,” ECTI Transactions on Electrical Engineering, Electronics, and Communications (ECTI-EEC), Vol. 17, No. 1 (2019), pp. 78-86. [13] S. Chansareewittaya, “Hybrid BA/ATS for Economic Dispatch Problem,” Proceeding of the 22nd International Computer Science and Engineering Conference (ICSEC) 2018, Chiang Mai, November 2018, Thailand. [14] S. Chansareewittaya, “Optimal Power Flow for Enhanced TTC with Optimal Number of SVC by using Improved Hybrid TSSA,” ECTI Transactions on Computer and Information Technology (ECTI- CIT), Vol. 13, No. 1 (2019), pp. 9-20. [15] H. Saadat, Power System Analysis, McGraw-Hill, 1999. [16] M. R. AlRashidi and M. E. El-Hawary, “Applications of computational intelligence techniques for solving the revived optimal power flow problem,” Electric Power Systems Research, vol. 79, issue 4, pp. 694-702, April 2009. [17] Y. Ou and C. Singh, “Assessment of available transfer capability and margins,” IEEE Transactions on Power Systems, vol. 17, May 2002. [18] S. Chansareewittaya and P. Jirapong, “Power Transfer Capability Enhancement with Multitype FACTS Controllers using Hybrid Particle Swarm Optimization,” Electrical Engineering, Vol. 97, Issue 2 (2015), pp. 119-127, Springer Publishing. [19] R. Jomthong and P. Jirapong, “Optimal Choice and Allocation of Distributed Generations using Evolutionary Programming,” Proceedings of the IASTED International Conference Power and Energy Systems (AsiaPES 2010), Phuket, Thailand, November 2010. [20] P. A. Vikhar, “Evolutionary algorithms: A critical review and its future prospects,” Proceeding of the 2016 International Conference on Global Trends in Signal Processing, Information Computing and Communication (ICGTSPICC). 167

2019 4th International Conference on Information Technology (InCIT2019) A Novel Combination Method of Beamforming and OSTBC for MIMO Broadcast System Mahdi Kashiha Paeiz Azmi Behrad Mahboobi Electrical and Computer engineering Electrical and Computer engineering Electrical engineering faculty Science and Research Branch of faculty faculty Tarbiat Modares University Tarbiat Modares University Islamic Azad University Tehran, Iran Tehran, Iran Tehran, Iran [email protected] [email protected] [email protected] Abstract— In this paper, a new robust design of the available information is the upper bound of perturbation. For combining Beamforming (BF) and Orthogonal Space-Time example, in [4, 5], by the use of variance of sub-channels, the Block Coding (OSTBC) in the presence of channel perturbation system has been statistically robust designed. for Multi-Input Multi-Output (MIMO) Multi-User Broad-Cast (BC) system is proposed. In order to improve the error The problem of making a system robust, eventually leads performance of our OSTBC-based MIMO system, BF is used to an optimization problem. One of factors that causes variety due to the existence of channel information in source side. By in the available literature in this domain is the definition of the using worst-case robust design, the channel perturbation is objective function in the resulted optimization problem. In bounded by a predefined variation. Firstly the maximum value problems which transmitting power plays an important role, of the pairwise error probability (PEP) is achieved for each the optimization problem can be considered as total receiver. Then BF matrix design is accomplished via minimizing transmitting power and made the system robust [6], [7]. In the maximum value of the whole receivers worst PEP. For the issues with focus on quality of service (QoS), first of all, a first time, in this paper and due to the complexity issue of the function as QoS is defined which can consider it as cost problem, this problem and generally such problems are function of optimization problem [8]. For example in [9], the addressed by using Cauchy-Schwarz inequality and Schur- QoS function is the received signal power to noise power ratio complement. Because of using the upper bound of PEP, the (SNR). proposed method shows near optimal results. Based on symbol error rate (SER) and PEP, the provided method yields better In another problem a PEP or mean square error (MSE) results compared with other existing methods. function may be considered as the QoS cost function [1], [2], [10] [11]. If the data rate is important in some cases, mutual Keywords— MIMO, OSTBC, Downlink Beamforming, information can be chosen as the cost function and consider Broadcast, Imperfect CSI, Worst-Case Robust Optimization. the resulted designed system in both asymptotic full and partial CSIT [12]. I. INTRODUCTION In this paper, in addition to upgrading our OSTBC-based One of the wireless communication damages is multipath system by the use of Beamforming we are going to make it fading phenomenon that dealing with it has become one of the robust against channel perturbation. This system is expanded greatest challenges these days. One way to solve this problem from single-receiver mode to multi-receiver mode and make a is to use MIMO system which is based on sending different MIMO-BC channel system. During robust design of the data sequences and their repetitions. According to existence of system, we encounter triple optimization problems. Each of channel state information in the transmitter side (CSIT), optimization problem is solved by an innovative approach. MIMO systems divide into two subgroups which are BF- Throughout the problem, Cauchy-Schwarz inequality and based when CSIT exists and space-time coding (STC) based Schur-complement play special novel roles to solve the when CSIT is absent. When a STC-based system has partial problem. For a detailed comparison, the final optimization CSIT, one way to increase the quality of transmission is to problem is check in both modes of existence and not existence combine the system with BF. So far, little works has been done of perturbation. Its results are the witness of better in this area including [1], [2]. In [1], single user BF design is performance of our design with respect to available relevant based on worst-cast robust design while in [2], statistical designs. robust optimization is discussed, both in the presence of channel perturbation. In section two of this article, the assumed scenario is described which goes far, up to where besides the detector In fact, perturbation represents all kinds of error in a design of described system, the problem formulation of how system, including channel estimation error. Perturbation to obtain the Beamforming matrix become clear. In third challenges the performance of the system. When there is section, it is going to make the system robust for single perturbation, the system design should be done as robust receiver and multiple receivers with the assumption of design so that system shows acceptable performance [3]. Two existence or not existence of the perturbation. In the fourth types of robust design are statistical robust design and worst- section, the designed system along with relevant approaches case robust design. In statistical robust design, statistical are simulated and evaluated. Finally in section five, we information of perturbed parameter such as mean or variance describe the conclusions and summarized achieved key points. is available, while in worst-case robust design, the only Notations: Upper (lower) boldface letters will be used for matrices (vectors). Signs [∙]∗, [∙]������������, [∙]������������, ������������������������(∙), [∙]������������������������, ������������������������, ⨂ XXX-X-XXXX-XXXX-X/XX/$XX.00 ©20XX IEEE 168

2019 4th International Conference on Information Technology (InCIT2019) denote complex conjugation, transposition, Hermitian transposition matrix trace, element of i-th row and j-th column of a matrix, ������������ × ������������ identity matrix and Kronecker product respectively. In addition, functions ℜ{∙}, ℑ{∙}, |∙| represent real, imaginary part and absolute value of a complex number respectively. Finally, ������������(∙), ������������(∙), ������������������������������������(∙) , ������������������������(∙) are probability of an event, tail probability of standard normal (Gaussian) distribution, exponential and natural logarithm function respectively. II. SYSTEM MODEL Fig. 1. Scheme of system model. Consider a MIMO-BC system consisting of a transmitter =LLRl Tr Ξ(R ( sl ) ⊗ IN ) − Tr Ωl (I( sl ) ⊗ IN ) with N transmitting antenna and L receivers that each of them has Q receiving antenna. First of all, transmitter codes its raw (4) data by the use of an OSTBC, then the result is beamformed by a BF matrix named W. Finally as its shown in Fig.1, the andWΩ���h��������� e≜re2uℑs�e���d��������������p���������a���������������������r������a���������m������������������e������t�e.rsBaarseeddeofninfeinaaslΞde≜te2ctℜor�����m����������������e������n������������������������t������i���o���������������n������e������d� resulted data are sent to the receivers. above, the detected data is whose LLR is greater than the LLR of the rest of the modulated data. If we want to look at sending/receiving structure in more details, it will be as follow. Firstly, ������������ number of equivalent In order to get the best BF in accordance with the channel modulated raw data from modulated symbol set are chosen. conditions and under the maximum transmit power constraint, The selected ������������ symbols are sent to the OSTBC coder as a PEP-related cost function is chosen. For this purpose, we vector in a time slot. Regard that the total number of time slots obtain the upper bound of PEP for each user, named Worst- equals �������������. The OSTBC coded data for the total time slots is as case PEP (WCPEP), as follows: follow in matrix form: C =(ℜ(s) ⊗ IN ) ˆ + j (ℑ(s) ⊗ IN ) ˆ (1) PEPi =P (Ck → Cl | Fi ,C =Ck ) Where �������������, ������������� ∈ ℂ������������������������×������������� are the equivalent total OSTBC ≤ e− di2 (5) coding matrix. Then, the coded data is sent to the receivers after multiplication by a BF matrix as ������������ ∈ ℂ������������×������������ . The 8σ 2 received data in the receivers is as below in vector form: zi yl (t) =Flx(t) + zl (t) ,t =1− Tˆ,l =1− L (2) 2 =WCPEPi Where ������������������������ ∈ ℂ������������×������������ , ������������������������(������������) ∈ ℂ������������×1 are the channel matrix Where the related exponent is ���������������2��������� = �������������������������������������������������������������������������������������������������������������������������������������������������,������������� between the transmitter and the l-th receiver and the and ������������������������,������������is defined for OSTBC as below: equivalent noise vector in the related receiver which is white and has complex normal distribution as �������������������������������������, Σ������������ = ������������������2�������������������������������������������. Ak,l =(Ck − Cl )(Ck − Cl )H =αIN (6) In the receiver side, by the use of OSTBC orthogonality property, the detector function based on Maximum Likelihood ∑Where α= T sk (m) − sl (m) 2 ∈ + . In the (ML) becomes as follow for all time slots: m=1 ( ) ( )Hl : ln P (Yi | s =sl ) > ln P (Yi | s =sm ) following, the BF design is done by the assumptions of the absence and existence of perturbation respectively. To obtain , m≠= 1,…, 2Tn the BF in the absence of perturbation and under transmitting l power constraint of ������������������������ℎ, the maximum value of WCPEP for all users is minimized and simplified as below: (3) maximize t Xr ,t Where ������������������������, 2������������are the event in which is detected, and the (7) cardinality of the used modulation symbol respectively. s.t.Tr ( Xr ) − Pth ≤ 0 Finally, after taking the logarithm from the detection criterion, Log-Likelihood Ratio (LLR) will be as follows based on Xr  0 matrix forms of parameters: t ≤ Tr (GiXr ) ∀i = 1,…, L Where ������������������������ ≜ ������������������������������������������������������������ ∈ ℂ������������×������������, ������������������������ ≜ ������������������������������������ ∈ ℂ������������×������������. Due to the linearity of cost function and constraint, the problem has a global maximum that can be achieved by the interior point method. 169

2019 4th International Conference on Information Technology (InCIT2019) With the assumption of perturbation presence, channel is maximize t Xr ,t Fi + ∆Fi L { }modeled as Fi= . Where ∆Fi represents the Tr ( X r) − Pth ≤ 0 i=1 < ωi 0 { }perturbation part whose boundary is L.  ΔFi ≤ δi  Xr  i=1 In first design step, the amount of WCPEP for various s.t.  G i X 2 (12) perturbations is maximized and then, the result is re-  r maximized for all users to get the worst case scenario. Finally, BF matrix is obtained in such a way that the resulted value is ( )t ≤ Tr Gi Xr − 2 δi ωi minimized under the maximum transmit power. The first part problem is as follows: ; ∀i = 1, L maximize WCPEPi For linearization of nonlinear constraint, we perform an ∆Fi equivalency as below: (8) ( )s.t.Tr ∆Fi∆FiH − δ 2 ≤ 0 G i X 2 < ωi   ⇔ … i r Based on the final cost function and the definition of ���������������2��������� = Fi X r ������������������������ ��������������������������� + Δ��������������������������������������������������������������������������������������� + Δ��������������������������������������, the problem becomes: ⇔  ωi IQN ( )vec  (13) ( )...  Fi X 0 vecH ωi  { }minimize di2 r ∆Fi (9) Finally, the design problem becomes: ( )s.t.Tr ∆Fi∆FiH − δ 2 ≤ 0 maximize t i Xr ,t ,ωi Using some simplifications along with Cauchy-Schwartz, Tr ( X r) − Pth ≤ 0 the lower bound of the cost function becomes as below: 0 ( )di2 > Tr FiWWH FiH  Xr  ( ) ( )− 2 Tr FiH FiWWH WWH ⋅Tr ∆Fi∆FiH  ( )s.t. ωi IQN vec Fi X r  (14) 0 Fi X r  ( ) (10) vec H ωi  Where ΔFi = βWW H FiH and ( )t ≤ Tr Gi Xr − 2 δi ωi  δi  ; ∀i = 1, L   ∈ ++ ( )β  by the  Tr FiH FiWW HWW H  Because of linearity of cost function and constraints, the  problem has a global maximum. After obtaining ������������������������ and using spectral factorization such as Cholesky, the optimal BF matrix fulfillment of power constraint. Among the approximations can be obtained. Of course the answer is not unique, but any solution satisfying the optimization problem, can be our final used in design procedure, the usage of PEP boundary instead BF matrix which has no effect on the designed system. The only difference among them is phase differences. For of itself, and quadratic perturbation term ignorance due to the procedural unity, the solution with minimum phase could be chosen as the final answer. low amount of perturbation are noteworthy. Although the later simplification reduces the complexity of optimization problem from exponential to polynomial, it increases the error probability in an ignorable amount. Finally, the cost function equals: In second step, the resulted cost function is minimized for III. SIMULATION RESULT all users. Then it is maximized according to the square of BF matrix. In this section, we verify our robust design with other available methods including equal power allocation. Suppose ( )maximize  T−r2 G iXr Xr2 L   a MIMO-BC system to simulate, including a 4-antenna Xr minimize δi G i    transmitter and 4 receivers, each equipped with 2 receiving i i=1   antenna. In this simulation, the number of generated symbols    (11) is 4 × 105. The equivalent channel noise has unit variance  and minimum modulation distance also equals to 2. According to mentioned concepts, first of all randomly generated data is s.t.Tr ( Xr ) − Pth ≤ 0 modulated by QPSK modulation and results in symbols. Then symbols are encoded by a 4x8 Alamouti-based OSTBC.  Xr  0 Finally before sending in channel, the resulted data is beam formed by the designed BF matrix. In receivers, we detect the Where G i  FiH F . Here, the semi-final optimization primary data and count the error rate by the use of LLR criterion which is obtained in the previous section. This problem becomes as follows: 170

2019 4th International Conference on Information Technology (InCIT2019) simulation is performed for about seven SNR points from −5 eventually the better performance belongs to equal power up to 12 ������������������������. allocation design. It is illustrated in the intersection of two curves resulting from equal power allocation and non-robust In figures, in addition to BF with equal power allocation, design. which its BF matrix is ������������ = ��������������������������ℎ⁄�������������������������������������, there are two other BF design methods, including robust design in the presence of To sense the advantages of our design better, it should be channel perturbation and design in the absence of channel compared to other similar works. However, because there is perturbation (nonRobust-Design). After encoding by no similar multi-user MIMO work, this comparison is Alamouti OSTBC, in these two methods, we beam-form the performed with single-user statistically robust design of [2]. results by designed BF matrices in (14) and (11). After that, In simulation, the number of transmitting and receiving we are going to send then via transmitting antenna. In the antenna are considered 4 and 2 respectively. The result is receiver side, as described above, we detect the primary data shown in Fig. 4. It shows that for a reasonable error probability by the use of (4) criterion. Regarding the curves, we see the of 10−6 , our method consumes 20% less power than its presence of WCPEP curve in addition to SER for all methods. counterpart. It reflects the superiority of our approach. Given the simulation of assumed system in the absence of IV. CONCLUSION channel perturbation, in Fig. 2, we see the coincidency of results obtained from both BF design approaches with/without In this work, we present a multi-user version of [1] derived consideration of channel perturbation. It is a natural from the combination of STC and BF. It is observed that the consequence, because by ignoring the perturbation parameter, designed system via this combination has the benefits of both our robust design problem converges to the BF design in the techniques, including simplicity in detection and higher absence of perturbation. Finally, we see that the results of performance in transmission. Design in BC mode has the these two design have better SER/WPEP performance than advantage of multi-user. To get closer to the real conditions, equal power allocation approach. the design is made robust by the use of Worst-Case Robust Design. In order to make the resulted problem solvable, we In Fig. 3, which is derived in the channel having benefit from the several clever and unique tricks including perturbation, the issue is slightly different. Due to involving Schur-complement and Cauchy-Swartz inequality. the perturbation in BF design, the result of robust design is Ultimately, we prove the effectiveness of our method against better than non-robust design. The basic question arises in other existing methods by the use of simulation. In terms of comparison of non-robust design with equal power allocation power consumption, finally we also see that our design uses design. In general, due to considering the channel matrix by 1.2 dB less power than the other methods at an equal itself, the better performance is in favor of non-robust design, acceptable error probability. but in lower SNRs indicating channel failure against noise, Fig. 2. The simulation results in the absence of the channel perturbation. 171

2019 4th International Conference on Information Technology (InCIT2019) Fig. 3. The simulation results in the presence of the channel perturbation. Fig. 4. A comparison between our approach and the method of [2] in a perturbed MIMO channel REFERENCES [7] B. Mahboobi, M. Ardebilipour, and E. Soleimani-Nasab, “Robust cooperative relay beamforming,” IEEE Commun. Letters, vol. 2, p. [1] M. Kashiha, P. Azmi, and B. Mahboobi, “A worst-case robust 399402, 2013. combination method of beam-forming and orthogonal spacetime block coding,” Springer Wireless Personal Commun., pp. 1–10, [8] A. Padmanabhan, “Precoder design for multi-antenna transmission 2018. in mu-mimo with qos requirements,” Master’s thesis, University of OULU, Finland, 2016. [2] V. Mai and A. Paulraj, “Optimal linear precoders for mimo wireless correlated channels with nonzero mean in space-time coded [9] W. Jiaheng, M. Bengtsson, B. Ottersten, and D. Palomar, “Robust systems,” IEEE Trans. on Signal Processing, vol. 54, no. 6, pp. mimo precoding for several classes of channel uncertainty,” IEEE 2318–2332, 2006. Trans. Signal Process., vol. 61, no. 12, pp. 3056–3070, 2013. [3] B. Liu, L. Shi, and X. Xia, “Robust rank-two multicast beamforming [10] Z. Xi, D. Palomar, and B. Ottersten, “Statistically robust design of under a unified csi uncertainty model,” IEEE Signal Process. Lett., linear mimo transceivers,” IEEE Trans. Signal Process., vol. 56, no. vol. 23, pp. 1419–1423, 2016. 8, pp. 3678–3689, 2008. [4] Z. Luo, S. Leung, and X. Yu, “Robust precoder design for mimo [11] KW. Huang, HM. Wang, J. Hou, S. Jin, “Joint Spatial Division and relay networks over double correlated rician channels,” IEEE Trans. Diversity for Massive MIMO Systems,” IEEE Trans. on Comm., on Signal Processing, vol. 63, no. 9, p. 11, 2015. vol. 67, no. 1, pp. 258–272, 2019. [5] X. Wen, M. Pesavento, “Long-Term general rank multiuser [12] H. Shen, J. Wang, B. Levy, and C. Zhao, “Robust optimization for downlink beamforming with shaping constraints using QOSTBC,” amplify-and-forward mimo relaying from a worst-case perspective,” IEEE ICASSP Conference, China, 2016. IEEE Trans. Signal Process., vol. 61, no. 21, pp. 5458–5471, 2013. [6] J. Borwein and J. Vanderwerff, Convex Functions: Constructions, Characterizations and Counterexamples. Cambridge University Press, 2010. 172

2019 4th International Conference on Information Technology (InCIT2019) Class Attendance Recording using QR Code via Smartphone Amompan Chomlin Lalita Na Nongkhai Pak Padungpattanadis Faculty of Information Technology Faculty of Information Technology Faculty of Information Technology Thai-Nichi Institute of Technology Thai-Nichi Institute of Technology Thai-Nichi Institute of Technology Bangkok, Thailand Bangkok, Thailand Bangkok, Thailand [email protected] [email protected] [email protected] Abstract— This article presents auto classroom attendance II. LITERATURE REVIEW AND RELEVANT STUDIES recording by using QR code on smartphones. Both teachers and A. QR Code[2] students take responsible for their parts. The teachers evaluate the result of each subjects. Then, the codes for each will be generated. QR Code is a 2D bitmap sign which was invented in 1994 When the students receive the code, they can scan and the by Denco company-part of Toyota company. It was qualified attendance will be recorded automatically. This research is divided in ISO (ISO/IEC18004) standard. After that, the QR code into 3 modules; teacher module, generate module, and student standard was set by AIM International Standard (AIM-ITS module. This research found that this attendance record method 97/01) as an automatic information record used in industries. resulted in a positive way. The result was evaluated by three ISO/IEC JTC 1SC 31 was suggested to be a standard due to experts (mean = 3.78; S.D. = 0.48). Also, the satisfaction rate was its effectiveness. The QR code sign is matrix consisting of evaluated by ten users which resulted in ‘very satisfy’ (mean = cells in a square. The information record system consists of 4.45; S.D. = 0.27). Timing pattern, Alignment pattern, Version information as Fig. 1. Keywords—automatic attendance record; QR code technology; smartphone; framework Fig. 1. QR code structure I. INTRODUCTION The working system is matrix code recording large amount of data as 7,089 letters. The density of data is 100 time per Information technology faculty of Thai-Nichi Institute of vector The data can be read by the sensor Charge Coupled Technology records students’ attendance for every class in Device(CCD). The data is organized and recorded in the order to build responsibility and discipline for students to memory. Importantly, there are four level of lost data follow rules and regulations. Currently, the teachers have protection; 7%, 15%, 25% and 30% per one QR code. The different ways of recording students’ attendance e.g. calling correction function works immediately as the Solomon code. name and noting in paper form or Microsoft Excel form, Solomon code organizes and corrects the wrong data. The checking via class activities, etc. The information shows each function can still work properly until the data is all corrected. student’s’ absent and late. A classroom with a large number of Users can set the level of data correction. When users create students consumes a lot of time of attendance recording. Also, the QR code, the data incorrection should not be more than the information might be inaccurate. Late students could make 30% as Fig. 2. the recording process more complicated for the teacher. Fig. 2. Data protection Quick response (QR) code is a Matrix code; the QR codes were developed in Japan in 1994 by Toyota subsidiary, Denso B. Smartphone[3] Wave to help track automobile parts throughout production. Over the years, smartphones have gotten a little staid. This technology has been around for over a decade but has since become popular as a medium for marketers to reach Advances have generally come in the form of incremental smart phone users. Quick Response Codes, or QR Codes, are improvements to popular features that are now standard nothing new. In fact, in Japan and Europe they have been among manufacturers and models. Annual enhancements used in marketing as well as inventory control and such as faster processors, better cameras, and higher manufacturing for the last 10 years. The security of one resolution displays are fairly predictable to the point that dimensional (1D) barcodes is lower than 2D barcodes. 1D barcodes are very easy to read by scanning the lines and the spaces. However, 2D barcodes are not easy to read a symbol pattern by human eyes. With regard to readability, 1D barcodes must scan along a single directional. If the angle of a scan line does not fit within a range, the data would not be read correctly. However, 2D barcodes get wide ranges of angles for scanning. Thus, 2D barcodes are readability.[1] From the above problem, the researcher conducted a research about using QR code for class attendance recording via smartphone. Apart from the convenience, the information would be better organized and accurate. 173

2019 4th International Conference on Information Technology (InCIT2019) they’re come to be expected. While bigger screens, thinner Fig. 3: Theoretical framework designs, and longer lasting batteries are great, the smartphone market is badly in need of the kind of revolutionary leap that The framework consists of 3 modules as follows: the original iPhone represented when it was first introduced in 2007. 1) Teacher module refers to the module related to the instructors. In this regard, the instructors have to sign up and More importantly, though, there are other signs and log on with username and password. Furthermore, the murmurs that smartphones are about to undergo a second instructors have to create their own classrooms by setting date renaissance over the next few years as a number of startups and time of when the classrooms start and finish. After that, are working on a number of new smartphone features. Here the instructors will get a code and distribute to students to are some new technologies on the horizon that are worth enroll the class. keeping an eye on. 2) Generate module is divided into two parts. The first Francisco Liébana-Cabanillas[4] conducted a research one is a code for students to enroll the class while the second about The main goal of our research is to analyze users’ one is a QR code for students to scan in order to record their acceptance of Quick response (QR) code mobile payment study times for each subject. systems, considering the population's widespread use of mobile devices. A comprehensive review of the scientific 3) Student module refers to the process of students literature has justified the development of a behavioral model signing up and activating their accounts used username and that explains intention to use of mobile payment via QR. To password. Then, they can join the class used a given code from this end, the authors have carried out a study through an the instructor so that it is convenient for them to use a online survey to a national panel of Internet users. The results smartphone scanning a QR code and will be able to check the show that attitude, innovation and subjective norms are record from each class. determinants of the future intention to use this technology. The conclusions and implications for management provide B. Research procedure alternatives for companies to promote this new business by means of the new technical developments. This paper is a • Research procedure pioneer study of intention to use with mobile payment via QR. Classic variables from the Technology Acceptance The researcher reviewed the previous studies about Model (TAM), as well as variables from other recent studies, classroom record and collected the data from the were used as models for this research (compatibility, security, school of information technology in order to evaluate personal innovation and individual mobility). the effectiveness of the system for developing the classroom record system to be more effective. Yang, Jaeseok [5] studied as mobile computing technologies have been more powerful and inclusive in • Data synthesis people's daily life, the issue of mobile assisted language learning (MALL) has also been widely explored in The data is synthesized in order to use and develop the Computer-Assisted Language Learning(CALL) research. automatic classroom record system correctly and Many researches on MALL consider the emerging mobile validly as it is helpful for arranging and checking data technologies have considerable potentials for the effective immediately. language learning. This review study focuses on the investigation of newly emerging mobile technologies and • System design their pedagogical applications for language teachers and learners. Recent research or review on mobile assisted The researcher employed Unified Modelling language learning tends to focus on more detailed Language (UML) which consists of Use-Case applications of newly emerging mobile technology, rather Diagram, and Context Diagram to display the data as than has given a broader point focusing on types of mobile Figs. 4 and 5. device itself. In this paper, I thus reviewed recent research and conference papers for the last decade, which utilized newly emerging and integrated mobile technology. Its pedagogical benefits and challenges are discussed. III. METHODOLOGY A. Theoretical framework The researcher presented a framework of automatic classroom record using QR code technology from smart phone as Fig. 3. 174

2019 4th International Conference on Information Technology (InCIT2019) After registration; all users will login for use the system as Fig. 7, Fig. 4. Use – Case Diagram Fig. 7. Login page The program classifies the users in to two types. Part 1: Teachers 1.Teachers create a class and fill in details; subject, start and end date as Fig. 8. Fig. 5. Context Diagram Fig. 8. Classroom details IV. SYSTEM DEVELOPMENT 2. From Fig. 8, teachers can share this class via application such, as Line, Email, Facebook, etc. After that, students can All data is used in order to develop the QR code system on attend the class as Fig. 9. smartphone . It is convenient for students to join the classroom whether from computers or smartphone with new technology. Then, the system is developed for students to use easily and convenience because the system can be deployed across different smartphone (android, iOS, and windows). The system classifies the types of users; teachers and students. Firstly, register is required. Fig. 6. Register Fig. 9. Classroom schedule 175

2019 4th International Conference on Information Technology (InCIT2019) 3. Sharing success is shown as Fig. 10. Fig. 13. Class information and class code 2. The names of student are recorded as Fig. 14. Fig. 10. Code after class ended 4. After class sharing, students can scan the QR Code as Fig. 11. Fig. 11. QR Code Fig. 14. Reported data after class ended 5. Teachers can look at the information as Fig. 12. 3. If students scan QR code before the teacher allows, the result will be as Fig. 15. Fig. 12. Student attendance name list Fig. 15. Data are recorded after QR Code scanned. Part 2: Students 4. Students can check their attendance information as Fig. 1. After login, students enter the code and click + the 16. record the attendance as Fig. 13. Fig. 16. Results after QR code scanned 176