Proceedings of 2019 4th International Conference on Information Technology

2019 4th International Conference on Information Technology (InCIT2019) Region of Interest Identification on Low- Resolution Lateral Spine Radiography Image using Density-based and Ellipse-like Method Saowalak Thamnawat John Gatewood Ham Suwanna Rasmequan Burapha University Burapha University Burapha University [email protected] [email protected] [email protected] Abstract— X-ray images of the lateral spine are nature of the low-quality image, the identification of important for diagnosing spine problems such as both the actual bone area and the regenerative bone area osteoporosis, bone fractures, and spondylosis. In order to is difficult to distinguish. identify bone diseases, often a series of images is required. These are taken using a low level of X-ray Fig. 1. Vertebral bodies that connect to the spinal on low contrast radiation to reduce the risk of exposure to overshoot image. radiation. Dual Energy X-ray Absorptiometry is a standard medical tool used to diagnose bone diseases. In Spinal curve and vertebral body identification for addition, the spine alignment of each individual person is bone segmentation from X-ray images is challenging different others. Therefore, developing an approach that due to images lacking clarity. Jennarong Saenpaen et can identify the spine area is challenging. In this work, al. [2] compared approaches using preprocessing for the algorithm for automatic identification of spine and improving the quality of an image. Among the three vertebral bodies is proposed. The proposed method methods compared were Brightness Preserving consists of three main steps. The first step, Bi-Histogram Dynamic Fuzzy Histogram Equalization (BPDFHE), Equalization with adaptive sigmoid functions (BEASF), Histogram Equalization (HE ), and Contrast Limited is a technique used for enhancing the spinal and Adaptive Histogram Equalization (CLAHE). The result vertebral bodies. In the second step, Density-based and showed that Contrast Limited Adaptive Histogram Ellipse-like techniques are combined to locate the curve Equalization (CLAHE) provided the best results. of the spine. For the third step, object improvement Madha Christian Wibowo et al. [3] proposed an techniques are applied to predict the location of vertebral approach to estimate the curvature of the spine. For this bodies. The experimental results show that the approach approach, a Top-Hat Filter was used to adjust the image reached 79.67% of Area Overlap Ratio. 81.67% of the clarity. Then, Gradient Vector Flow was used to locate Precision value. parts of different colors and make them be more pronounced to enable finding the exact the edge of the Keywords-component; low-resolution radiography; objects to be used to find the total area of the spine. lateral spine; BEASF; Density-base; Ellipse-like; Curve- Bagus Adhi Kusuma [4] proposed an algorithm to Estimation, Object Improvement automatically define spinal curvature from digital X- ray images. The preprocessing was done using Canny I. INTRODUCTION Edge Detection. The K-means Clustering algorithm was used to detect the centroid point after the Treatment of many spinal diseases requires having segmenting. Then Polynomial Curve Fitting was used X-ray images to see the nature of the spine and for determining the spinal curve. From the spinal vertebral bodies. For example, in order to analyze curvature information, the scoliosis curves were osteoporosis, a medical specialist must calculate the classified into 4 conditions: Normal, Mild, Moderate, Bone Mineral Density (BMD). BMD can be calculated and Severe Scoliosis. using a piece of medical equipment called a Dual Energy X-ray Absorptiometry (DXA) scanner. DXA is In this paper, we present a new approach to popular because of the inexpensive machine cost and a improving the image quality to automatically locate the low doses of radiation used. For each DXA treatment, Region of Interest of the lateral spine and vertebral a patient will receive a low dose of radiation compared to other types of X-ray machines. Therefore, this equipment is suitable for generating a series of images for bone treatment. The images obtained from DXA machines often have a very low quality but are used anyway because they are safer than other scanners which use higher doses of radiation.[1] Due to this characteristic, the details of the objects in an image are low and the amount of noise is high. This often obscures the details of the bone structure as shown in Fig.1. Because of this 77

2019 4th International Conference on Information Technology (InCIT2019) bodies. From observation, we know the joints of each bone degeneration from this X-ray image, as shown in of the vertebral bodies are found to be in areas that have Fig.3. the highest bone density. Due to this property and the fact that the vertebral bodies are normally aligned with Fig.3. Lumbar vertebrae (a), Dual-energy X-ray absorptionmetry the spinal curve, the proposed method took these (b). properties into consideration. The proposed method consists of three steps. In the first enhancement step Bi- III. METHODOLOGY Histogram Equalization with Adaptive Sigmoid The main purpose of the proposed method is to Functions (BEASF) is used to recondition those low identify the osteophytes diseases. In this section, the contrast areas of the lateral spine image to highlight the description of the steps of the proposed approach for bone area. Secondly, as the shape of the curvature of lateral spine enhancement and Region of Interest the spine is the starting point to identify the vertebral Identification in low-resolution radiography images is bodies, an Ellipse-like method is applied to estimate the discussed. The overview of the proposed method is curve of the spine. Finally, the prediction of the shown in Fig.4. vertebral bodies is implemented using a Multi theta Gabor Filtering and Mean-range candidate selection from the local maximum horizontal projection. This allows identifying the area of the vertebral bodies. II. BACKGROUND KNOWLEDGE The basic background knowledge is described below. A. Anatomy of the Spine The vertebral column [5], also known as the backbone, is a part of the axial skeleton. For a normal human spine, there are 24 vertebrae. They are 7 cervical vertebrae, 12 thoracic vertebrae, and 5 lumbar vertebrae. They are separated from each other by intervertebral discs. One of the most common bone disease diagnosis techniques is inspecting the lumbar vertebrae [6] . Lumbar vertebrae are in the lower back as shown in Fig.2., between the thoracic vertebrae and sacrum. The five units of lumbar vertebrae are usually identified by the names L1, L2, L3, L4 and L5. L5 is the initial point of the lumbar region. L1 is next to the thoracic vertebrae (T12), and L5 is next to the sacrum. Lumbar vertebrae support the body weight and help the body move for almost any gesture. This area is the most commonly damaged in cases of osteoporosis, bone fractures, and spondylolisthesis. Fig.2. Anatomy of the Spine Fig. 4. The overview of the proposed approach B. Dual-energy X-ray absorptionmetry A. Contrast Enhancement A Dual-energy X-ray absorptiometry scanner [7] is 1) Bi-Histogram Equalization with adaptive one of the medical machines use for measuring bone sigmoid functions (BEASF) mineral density (BMD). Two X-ray beams, with Bi-Histogram Equalization with Adaptive Sigmoid different energy levels, are radiated at the patient's bones. When soft tissue absorption is subtracted out, Functions, [8] contains three modules: histogram the bone mineral density (BMD) can be determined splitting, sigmoid transform creation, and mapping. from the absorption of each beam. It is used to diagnose bone diseases. A doctor evaluates the symptoms of a) Histogram Splitting 78

2019 4th International Conference on Information Technology (InCIT2019) Let ‫ ܫ‬be an input image of size ‫ ܰݔܯ‬and let ݉ be the mean of intensity (Eq.1). ݉ ൌ  σೝಾసషబభ σ೎ಿసషబభ ூሺ௥ǡ௖ሻ (1) ெ௫ே The mean of intensity ݉ is employed as a splitting Fig.6. Area of the density of intensity of facet joint in the spinal. point to separate the histogram ‫ ܪ‬into two pieces. ‫ܪ‬௅ and ‫ܪ‬௎ are sub-histograms. (see Eq.2, 3 and 4). 2) Removing Noise In this step, because each bone in each person has where, ‫ ܪ‬ൌ ‫ܪ‬௅ ‫ܪ  ׫‬௎ (2) different density, we must eliminate the density of the ‫ܪ‬௅ ൌ ሼ‫ܪ‬଴ ൅ ‫ܪ‬ଵ ൅ ‫ ڮ‬൅ ‫ܪ‬௠ሽ (3) intensity in the dispersed image. Then, opening operation is employed for removing all small ‫ܪ‬௎ ൌ ሼ‫ܪ‬௠ାଵ ൅ ‫ܪ‬௠ାଶ ൅ ‫ ڮ‬൅ ‫ܪ‬ூିଵሽ (4) connected-component. The resulting image from removing noise with this After the splitting, we calculate the probability process are shown in Fig.7. density function of the two sub-histograms. Then we calculate the median and cumulative distribution Fig.7. Output Binary image from density based (Left image) and functions for both sub-histogram. The medians of Output from area opening Morphological transformation (Right ‫ܪ‬௅ܽ݊݀‫ܪ‬௎ denoted by ߤ௅and ߤ௎, respectively image) b) Sigmoid transform creation 3) Geometric Ellipse-like Object Detection Two parametric, non-linear sigmoid functions are created with their origins located on the medians of their corresponding sub-histograms. The target of this function is to increase noise robustness and cumulate the distribution function. c) mapping Mapping is done with histogram equalization and stretching. The mapping function is applied to each pixel of the image. The results of this image improvement are shown in Fig.5. Fig.5. Compare the original image with the improved image created Fig.8. Properties of an ellipse shape using BEASF. The shape of the object found in the binary image B. Density base to locate Spinal from the density base image is similar to an ellipse shape (Eq.6) and (Eq.7). Properties of the elliptical 1) Density of Intensity image are shown in Fig.8. The density of intensity of the facet joint is an important reference point. It indicates the shape of the ௫మ ൅ ௬మ ൌ ͳǢ ܽǡ ܾ ൐ Ͳ (6) spine curvature. The vertebral body is positioned at a ௔మ ௕మ right-angle to the curve. The data matrix ‫ ܫ‬is an intensity image. Values represent the intensities of the ሺ௫ି௫೎ሻమ ൅ ሺ௬ି௬೎ሻమ ൌ ͳǢ ܽǡ ܾ ൐ Ͳ (7) image (see Eq.5). The elements in the intensity matrix ௔మ ௕మ represent various intensities, where the intensity 0 represents black and the intensity 255 represents full ‫ݔ‬ǡ ‫ ݕ‬are coordinates of centroid. The length of the intensity or white in a grayscale image. major axis is ʹܽ and the length of the minor axis is ‫ ݊ܫݕݐ݅ݏ݊݁ܦ‬ൌ ܽ‫ݔܽ݉݃ݎ‬ሺ௫ǡ௬ሻ‫א‬ௌሺ‫ܫ‬ሺ‫ݔ‬ǡ ‫ݕ‬ሻሻ (5) ʹܾ. After small objects have been removed, Object The results of finding the image area of the density of intensity of the facet joint using the spinal Detection using Geometric Ellipse-like is done. Then in this method are shown in Fig.6. we calculate the centroid of each Ellipse. The resulting image from performing Ellipse-like Object Detection in this way are shown in Fig.9. 79

2019 4th International Conference on Information Technology (InCIT2019) Fig.9. Output from Ellipse-like Object Detection C. Prediction the vertebral body locations 4) Points estimation using linear-equation. 1) Object Improvement using Multi-theta Gabor This step will find the approximate object centroid Filtering on the spinal axis using a linear equation (Eq.8) and (Eq.9) use the centroid of the object that the ellipse can After obtaining the curvature of the spine, we will detect from bottom to top, respectively. The resulting use this curve to be the reference point to find the from image the Points estimation using those linear- boundary of the spine. Then, the results from the equations is shown in Fig.10. spinal area are determined to extract the edge of the spine. Multi- theta Gabor Filtering [9] is a technique to use extract objects in specific orientations. (Eq.11). ‫ܫ‬௠௚௕ሺߠଵǡ ߠଶሻ ൌ σఏ௜ୀమఏିభଵ൛‫ܫ‬௚௕ሺ݅ሻ  ‫ܫ  ׫‬௚௕ሺ݅ ൅ ͳሻൟ (11) Let ‫ܫ‬௠௚௕ be the output image of Multi- theta Gabor Filtering. ߠଵǡ ߠଶ are the initial and final theta values and ‫ܫ‬௚௕ is the output of the Gabor Filter function. The results of the prediction of the vertebral ݉ ൌ ሺ௬మି௬భሻ (8) body locations in the image processed with Multi- (9) ሺ௫మି௫భሻ theta Gabor Filtering in this way are shown in Fig.12. ܻ ൌ ݉ܺ ൅ ܾ Fig.10. Estimate interpolated point of the spine curve Fig.12. Output from Multi-theta Gabor Filtering 5) Lateral Spine Curve Estimation using 2) Candidate selection using Mean-range Polynomial Regression Candidate Selection The final step is estimating the shape of the After vertebrae poses’ pattern were revealed, curvature of the spine. After the interpolation horizontal projection is performed to find the local procedure, which yields the point for increasing the maximum peak. The results of the candidate selection point and direction of the correct curve of spinal image from applying Mean-range Candidate Selection position data, polynomial curve fitting is used. (see in this way are shown in Fig.13. Eq.10) The regression model determines the set of points for the region by assigning 5 to ݊ for fitting the to the data points. The generated Lateral Spine Curve Estimation image from estimating the location of the spine in this way are shown in Fig.11. ‫ ݕ‬ൌ  ߚ଴ ൅ ߚଵ‫ ݔ‬൅ ߚଶ‫ݔ‬ଶ ൅ ǥ ൅ ߚ௡‫ݔ‬௡ ൅ ߝ (10) Fig.11. The output image showing the estimated location of the Fig.13. Local maximum Horizontal Projection of Lateral Spine spine image Then, all the local peak that those points are expected positions that line the segment vertebral bodies are defined. The next step is a process of selecting candidates from all local points by finding the mean of the point in the range that is under the criteria. An algorithm for finding the new candidate for prediction of the boundary vertebral bodies is given in Fig. 14. 80

2019 4th International Conference on Information Technology (InCIT2019) ‫ ݌݈ܽݎ݁ݒܱܽ݁ݎܣ‬ൌ ቚ஺‫ת‬஻ቚ ൈ ͳͲͲ (12) ஺ Let ‫ ܣ‬be an output image and ‫ ܤ‬be a corresponding ground-truth images. ܲ‫ ݊݋݅ݏ݅ܿ݁ݎ‬ൌ  ்௉ (13) ்௉ାி௉ Fig.14. Mean-range Candidate Selection Method The experimental result shows that the performance of our proposed approach achieved 79.67% for Area After that, the new candidate will be used as a point Overlap and 81.67% for Precision as shown in Tables for the vertebrae. This can be seen in the output image I and II. in Fig.15. TABLE I. RESULT OF SPINAL IDENTIFICATION WITH AREA OVERLAP Fig.15. Result of Selected candidates to predict vertebral bodies IV. EXPERIMENTAL RESULTS The proposed approach used 30 X-ray images of lateral spines a from Dual-energy X-ray absorptiometry machine. The dataset was provided by a local hospital. They were obtained using a safe, low radiation imaging process. Each of the images has a variety of quality and contrast. A number of the images have very low quality due to patient physical conditions. In the experimental results, the automatic spine identification using our proposed method was compared with the ground-truth images. This is demonstrated in Fig.16 TABLE II. RESULT OF VERTEBRAL BODIES IDENTIFICATION WITH PRECISION (TP AND FP) (a) (b) V. DISCUSSION AND CONCLUSION Fig.16. (a) Input image with interpolated line, and (b) ground-truth In this research, a new approach to automatic spinal image with spine curve region. and vertebral body identification from Low- Resolution Lateral Spine Images was proposed. Our Each of the ground-truth images were manually approach consists of three main steps. First, the low- defined by the experts who were trained by a quality images are improved using Bi-Histogram physician. The effectiveness of the proposed algorithm was measured using Area Overlap for spinal curve identification. A Confusion matrix method was employed to measure the performance of the vertebral body identification. This test is demonstrated in Eq. 12 and Eq. 13. 81

2019 4th International Conference on Information Technology (InCIT2019) Equalization with adaptive sigmoid functions for [2] J. Saenpaen, S. Arwatchananukul and N. Aunsri, \"A enhancing the area of spine. In the second step, Density-based and Geometric Ellipse-like techniques Comparison of Image Enhancement Methods for Lumbar are combined to locate the curve of the spine. Finally, Multi-theta Gabor Filtering and Mean-range Spine X-ray Image,\" 2018 15th International Conference on Candidate Selection techniques were applied to predict vertebral body locations. The proposed method Electrical Engineering/Electronics, Computer, automatically locates lateral spine data. In the performance measurement, the experimental result Telecommunications and Information Technology (ECTI- shows that the approach reached 79.67% of Area Overlap Ratio. 81.67% of Precision value. In future CON), Chiang Rai, Thailand, 2018, pp. 798-801. work, region identification of osteophytes for support of the diagnosis of an orthopedic doctor will be [3] M. C. Wibowo and T. A. Sardjono, \"Spinal curvature attempted. determination from x-ray image using GVF snake,\" 2015 International Conference on Information & Communication VI. ACKNOWLEDGMENT Technology and Systems (ICTS), Surabaya, 2015, pp. 35-40. This work was financially supported by the [4] B. A. Kusuma, \"Determination of spinal curvature from Research Grant of Burapha University through the scoliosis X-ray images using K-means and curve fitting for National Research Council of Thailand (NRCT), fiscal early detection of scoliosis disease,\" 2017 2nd International year 2018, Faculty of Informatics, Burapha University, conferences on Information Technology, Information Systems Burapha University Hospital, and Dr.Alisara and Electrical Engineering (ICITISEE), Yogyakarta, 2017, pp. Wongsuttileart, MD. 159-164. [5] The Vertebral Column. (n.d.). Retrieved from https://teachmeanatomy.info/back/bones/vertebral-column/ [6] Lumbar Vertebrae. (n.d.). Retrieved from https://www.physio- pedia.com/Lumbar_Vertebrae [7] Radiological Society of North America, Rsna, & American College of Radiology. (n.d.). Bone Densitometry (DEXA , DXA). Retrieved from https://www.radiologyinfo.org/en/info.cfm?pg=dexa REFERENCES [8] E. F. Arriaga-Garcia, R. E. Sanchez-Yanez and M. G. Garcia- Hernandez, \"Image enhancement using Bi-Histogram Equalization with adaptive sigmoid functions,\" 2014 International Conference on Electronics, Communications and Computers (CONIELECOMP), Cholula, 2014, pp. 28-34. [1] Pietro, M. A. de. (n.d.). DEXA scan: Purpose, procedure, and [9] W. Yookwan, K. Chinnasarn and B. Jantarakongkul, \"Automated Vertebrae Pose Estimation in Low-Radiation results. Retrieved from Image using Modified Gabor Filter and Ellipse Analysis,\" 2018 5th International Conference on Advanced https://www.medicalnewstoday.com/articles/324553.php?fbcl Informatics: Concept Theory and Applications (ICAICTA), Krabi, 2018, pp. 141-146. id=IwAR0iwOo_mCZnirjl9uJFfpQHnXkMn0pPUtLg- Xkt2lrfhvoM0tr1tEkkpjU 82

2019 4th International Conference on Information Technology (InCIT2019) Tunica Media Localization in Intravascular Image with Shadow Artifact Constraint using Circular-like Estimating Techniques Jiraporn Wongwarn Suwanna Rasmequan Burapha University, Thailand Burapha University, Thailand [email protected] [email protected] Abstract—The advancement of medical technologies they also contain the shadow that blocks out data details allows diagnosis processes more convenient and time of the vascular structure. The shadows are the saving. Ultrasound technique is being used in a number specialists’ obstacles to diagnosing medical vascular of medical diagnosis processes. Vascular diseases are one diseases. In order to identify the diseases effectively, of the group of diseases that used ultrasound techniques the doctor must clearly localize the elements within the to diagnose. These types of diseases may cause serious vascular images. Tunica media is the main component illnesses such as heart failure, coronary artery disease, within the blood vessel which is essential to be and cardiomyopathy. Ultrasound technique often localized. As shown in Fig. 1, in case that the image provides a low-quality image due to the safety contains a lot of shadows that is overlap with the tunica precaution. Thus, Intravascular images produced by media, the doctor was unable to specify the exact area ultrasound techniques often have low contrast, high of the tunica media. noises, and few shadows. These features of the low- quality image made it difficult to identify each layer of Fig. 1 Intravascular image with shadow blocked data details. the Intra Vascular which is needed to segment the actual area of the plaque. In this paper, Circular-like estimating Recently, a number of research works are proposed techniques is proposed to locate the Tunica Media on in this area as follows. Hannah Sofian et al. [1] low-quality Intravascular image. The proposed method proposed a method to find the edge location of the provides an interesting result with the JI of 87.45% for median artery wall or Tunica Media from Intravascular images with shadow constraint and 92.06% for non- images using Otsu Thresholding, Empirical shadow constraint images in dataset A. In addition, the Thresholding, Binary and Morphological Operation. proposed method achieved JI of 63.90% for shadow Anusorn Wong-od et al. [2] proposed an automatic constraint images with various patterns in dataset B. method for detecting lumen and media-adventitia These results implied that the proposed method can boundaries from vascular ultrasound images by reasonably apply with both shadow and non-shadow applying a stretching equation to increase the clarity of artifacts while those earlier works have to attempt them the Intravascular image while adaptive K-mean is used with different methods. to reduce the complexity. The two separated areas are then segmented using the Convex Hull method. Mehdi Keywords-component; Tunica Media; Circular-like; Faraji et al. [3] proposed the Extremal Regions of Intravascular ultrasound; Shadow Artifact; Entropy; Extremum levels method to detect the areas and to draw the boundaries of the lumen and the tunica media INTRODUCTION vascular wall in the Intravascular Ultrasound images obtained from the 20 MHz ultrasound device. They also Atherosclerosis is a disease caused by the expanding of presented the region selection process to label the edge plaque inside the arteries. If the blood vessels have a of the vascular channel (Lumen) and the median artery plaque made up of fat, cholesterol, calcium, and other wall (Tunica Media). Ju Hwan Lee et al. [4] proposed a substances. which can be found inside the blood geometric deformable model-based segmentation vessels. Gradually, plaque becomes harder and it approach to segment the intima and media-adventitial narrows the arteries. This blocked the flow of oxygen- (Tunica Media) borders in the sequential intravascular rich blood to the organs and other parts of the body. ultrasound images. Anusorn Wong-od et al. [5] which be the cause of serious problems with proposed a new way to recover intravenous ultrasound atherosclerosis, including heart attack, stroke, or even death. So, the causes of failure identification within the circulation system are indispensable. Medical diagnosis technique of these types of diseases uses ultrasound technique. Intravascular images produced by ultrasound techniques may provide high-profile information in some cases. However, the latest ultrasound device innovations for the internal coronary artery examination sometimes fail to produce the high detail of the vascular structure. The ultrasound photos of those failed cases not only be low in quality with low contrast, high noises, but 83

2019 4th International Conference on Information Technology (InCIT2019) images and segmentation of lumens and media. A (b) circular analysis is proposed to estimate the lumen and media. The spreading technique was combined to Fig.3 (a) Intravascular ultrasound image and (b) Example of image recover damaged pixels. in each group In this paper, a method to localize the Tunica Media C. Circular-like in the Intravascular Ultrasound Image that contained shadows using Circular-like estimating techniques is For the circle shape of objects detection, the well- proposed. The localization of the Tunica Media can known algorithm known as the Hough transform is then lead to the process of evaluating the amount of the often used to detect the Circle-like structure [8]. The plaques accumulated within the arteries. The related purpose of the technique is to find circles in imperfect concepts and theories are presented in the following image inputs. That can be done in the following three section. important steps. BACKGROUND KNOWLEDGE - Accumulator Array Computation - Center Estimation A. Anatomy of Blood Vessel - Radius Estimation Blood Vessel [6] is a part of the circulatory blood system in the body that has the following elements: the outer covering of the artery (tunica adventitia), the actual wall of the artery (tunica media), the layer of endothelial and other cells that make direct contact with the blood inside the artery (tunica intima). By nature, arteries tunica intima is thin. The lumen is the actual open channel of the artery through which the blood flows shown in Fig.2. ������ = ������ + ������ ∗ scions ((������������)) 2 ������������������ℎ ������ ∈ (0,2������) (1), ������ = ������ + ������ ∗ Fig.2 Anatomy of Blood Vessel where (������, ������) is the center of the circle, and ������ is the radius, When the ������ varies from 0 to 2������, a complete B. Intravascular Ultrasound circle of radius ������ is generated. Intravascular images are used as input for medical D. Entropy imaging analysis methodologies. These types of images retrieved by using a specially designed catheter with a One of the important measure in information miniaturized ultrasound attached to the distal end of the theory is \"entropy\". Entropy is used to measure the catheter [7]. From Intravascular images in Fig. 3 (a), amount of uncertainty of data. That is, if the data(2i)s, there are elements as mentioned above in the anatomy highly chaotic, the value of entropy is high. structure such as Tunica Media, Lumen, Tunica Adventitia and Shadow as depicted in Fig. 2. The ?= characteristics of the images used in this experiment are 2 types: Shadow and Non-Shadow Images. The ������������������=>?(@A) = − C C ������E������>GH������������������K������E������>GH experiment was also done with two sets of data. Dataset A contained both Shadow Image and Non-shadow >LM GLM Image as shown in Fig. 3 (b). Dataset B contained only Shadow Image as shown in Fig. 3 (c). where ������ is the probability of intensity image, ������ and ������ are width and height of image and ������>G is the intensity of (a) the grayscale image. E. Euclidean distance Euclidean distance is a normal distance between two points in a straight line. In a segment ������������, if ������ = (������1, ������2, . . . , ������_������) and ������ = (������1, ������2, . . . , ������_������) in the cartesian coordinate system, is the two points on the Euclidean space, and ������ the distance between the point, then ������ and ������ is calculated from eq. ( 3). ������ = S(������1 − ������2)K + (������1 − ������2)K (3) F. Merging Merging is the operation of the set to create a new set that results from combining all the elements. The union of two sets ������ and ������ is the set of elements which are in ������ , in ������ , or both ������ and ������ is calculated from eq. (4). ������ ∪ ������ = {������ ∶ ������ ∈ ������ ������������ ������ ∈ ������} (4) 84

2019 4th International Conference on Information Technology (InCIT2019) METHODOLOGY § Tunica Media Localization In this work, a method to localize the Tunica Media From the observation, the appearance of the which is a component needed to identify the plaque area in the arteries of Intravascular Ultrasound Images Tunica Media edge is always larger than the circular is proposed. The proposed method can be used to locate shape of the lumen area. From the experiment result, the Tunica Media for both Shadow and Non-Shadow the proper radius size of the Tunica Media for the Constraint Imaged. The details description of the dataset A is range between 70 and 90 pixels as shown proposed method is described in the following steps. in Fig. 6. For dataset B, the proper radius size is range between 35 and 60 pixels. With these observations, the Tunica Media candidates are located using the minimum radius ( ������_������������������ ) and maximum radius (������_������������������). Fig.6 Define the scope of media § Mapping Tunica Media using Circular-like After the feature of the radius of Tunica Media was identified, the candidate objects that got a shape like a circle defined by eq. (1) is located. A huge number of circular-like objects is scope down using parameter ������, ������, and ������. That is, all the candidate circles got the ������ parameter that falls between ������_������������������ and ������_������������������ as shown in Fig. 7. Fig. 4 Diagram of the proposed method Fig.7 Set of all the possible circles A. Preprocessing § Centroid Initialization using Entropy The first step, the speckle noise is reduced using However, the result of the earlier step produced a the original image convolution with a median filter. big number of circular-like objects. Some of them The output of this process was a blur and smooth might be in the potential area of Tunica Media. Some image as shown in Fig. 5. of them might not. Therefore, there should be a way to distinguish them. As the property of the Tunica Media (a) (b) seems to have the same centroid with the lumen area. Thus, in this step, the seed point needed to locate the Fig.5 (a) original image and (b) result after convolution with a circle have to be in this area. Together with the color median filter. property of the lumen area which are completely black. So, the centroid point must be selected from the lumen B. Estimation area and have the lowest data change value. That value In this step, a noise reduction image from the pre- was obtained from the argument min of entropy as eq. (2). The centroid of the circle as a seed point in Fig. processing step is processed further to locate the 8 was led to the circle selection in the next step. Tunica Media using circular estimation to determine the candidate areas of the Tunica Media using the following steps: (a) 85

2019 4th International Conference on Information Technology (InCIT2019) C. Detection After all the possible candidates were combined, the area of the Tunica Media can be detected using contour extraction as shown in Fig. 11. (b) Fig.11 Contour extraction of Tunica Media Fig.8 Choosing the seed point from the value entropy EXPERIMENTAL RESULT In Fig. 8 (a) shown all the possible circles The dataset was obtained from the imaging system which got different properties as shown in Fig. 8 (b). used for the acquisition is a Si5(VolcanoCorporation), That is, the different value of entropy such as (b1) equipped with a 20MHz Eagle Eye monorail catheter 5.2051, (b2) 5.2197, (b3) 4.771, (b4) 6.8042, (b5) at Computer Vision Center, Bellaterra, Universitat de 5.9274, (b6) 6.8992 and (b7) 6.71. So, centroid which Barcelona, Barcelona, ESP. In this data set, the real selected to be the seed point is b3 which has the least shape or ground truth is drawn by hand by the expertise entropy value of 4.771. of the Computer Vision Center. The accuracy of the proposed method was measured using Jaccard index § Identify Candidate using Euclidean distance (JI), Hausdorff Distance (HD) and Percentage distance After the seed point is located, then the length of the area (PAD). seed point and the remaining centroids were identified • Jaccard index (JI) [9] is region-based as shown in Fig. 9 using Euclidean Distance method as accuracy measures as the following equation: depicted in eq. (3). For any circle object that have a distance of seed point and its centroid lesser than 36 ������������E������Z[\\]^_ , ������`_H = abcdefgh∩bjha (5), pixels, that particular circle was suspected to be a Tunica Media. abcdefgh∪bjha Fig.9 Finding the distance of the seed point with another centroid where ������Z[\\]^_ is the area of the resulting region, ������`_ is the area of ground truth, the Jaccard index measure of § Nearest Circular Merging From the above step, the circles that qualified to similarity for the two sets of data. be a possible Tunica Media area were then selected. From this example, there were 2 circles as shown in • Hausdorff Distance (HD) [5] is contour- Fig. 10 (a) which qualified as the area of Tunica media. based accuracy measures as the following equation: These two qualified circles were then union together using eq. (4). The result is the possible area of the ������������������������m (������, ������) = max {max min‖������ − ������‖, max min‖������ − ������‖} (6), Tunica Media as shown in Fig. 10 (b). ������ ∈ ������ ������ ∈ ������ ������ ∈ ������ ������ ∈ ������ (a) (b) where || || denotes the Euclidean distance, ������ the Fig.10 Selected circles by Euclidean distance ground truth contour, and ������ the resulting contour. The best HD should be lower since the value of zero means perfect result or isometric line. • Percentage distance area (PAD) is region- based accuracy measures as the following equation: ������������������E������Z[\\]^_, ������`_H = wbcdefbjghhxbjhw ∗ 100 (7) From the experimental result of the proposed method for estimating the edge of the Tunica Media. Dataset A is divided into 2 groups, in group one which consists of 10 shadow images as shown in TABLE I and 20 non-shadow images as shown in TABLE II. The experimental result of Active Contour, Threshold, Anusorn [2.] and the proposed method, were individually compared with Ground Truth. 86

2019 4th International Conference on Information Technology (InCIT2019) TABLE I. AVERAGE RESULT OF ACCURACY MEASURES OF TUNICA-MEDIA BOUNDARY ESTIMATION IN DATASET A WITH SHADOW TABLE II. AVERAGE RESULT OF ACCURACY MEASURES OF TUNICA-MEDIA BOUNDARY ESTIMATION IN DATASET A WITHOUT SHADOW From the summary of the TABLE I While the proposed method yielded 92.06% (JI), experiment in the dataset A image of the shadow 33.65% (HD), 4.66% (PAD). It was clearly seen group. We will find that the effective active that the proposed method has better performance contour method is 40.94% (JI), 51.93% (HD) and than the active contour and threshold in all 63.22% (PAD). The threshold method is effective category’s measurement. However, when compared at 57.14% (JI), 60.70% (HD) and 44.90% (PAD) with Anusorn [2], the proposed method has better and the methods we offer are effective at 87.45% performance than only in Hausdorff Distance (JI), 38.14% (HD) and 12.60% (PAD). We can see measurement. that in the dataset A of a shadowed image group, our method provides better results than the two TABLE IV. SUMMARY RESULT OF DATASET A methods above, you can see that result in TABLE WITHOUT SHADOW III. TABLE III. SUMMARY RESULT OF DATASET A WITH SHADOW From the result shown in TABLE IV, The experimental result of Dataset B as images of the dataset A do not have any shadow, or shown in Table V. is the comparison between the the images are quite perfect. The active contour proposed method and Anusorn's work in 2017. This method yielded 50.65% (JI), 51.95% (HD), 48.87% is due to the fact that Active Contour and Threshold (PAD). The threshold method yielded 69.16% (JI), cannot be applied with dataset B because this dataset 38.44% (HD), 29.49% (PAD). Anusorn methods has a large area of shadow and misshapen Tunica yield 95.47% (JI), 45.75% (HD), 4.70% (PAD). Media. Which the proposed method is lower performance as follows. Anusorn gets 88.53% (JI), 87

2019 4th International Conference on Information Technology (InCIT2019) 44.89% (HD), and 10.49% (PAD) while the DISCUSSION proposed method gets 63.90% (JI), 33.33% (HD) and 36.34% (PAD). The overall result of the proposed method was still could not outperform as compared to Anusorn [2] and TABLE.V SUMMARY RESULT OF DATASET B WITH [5]. However, the proposed method can be applied SHADOW well to both Shadow and Non-shadow constraint. The main reason for the wrong prediction was the fix of Fig. 12 Result of DATASET A: (a) Active Contour, (b) Threshold, the parameters in step IV, the selecting of a candidate (c) Proposed Method in shadow group. circle using Euclidean distance. In addition, for dataset B, the results were poor because of the centroid selection process and the seed point estimation in step III, using the lowest entropy method. When the centroid was defined in the area where the shadow occurs, as shown in Fig. 15 (a), the proposed algorithm will predict such an area as the area of lumen because it also has the lowest entropy. As a result, the algorithm may choose that centroid point as a seed point as shown in Fig. 15 (b). In future work, the proposed method needs to be modified to remove those drawbacks and make the model more generalize and more accurate. Fig. 13 Result of DATASET A: (d) Active contour, (e) Threshold, Fig. 15 (a) Circle search and (b) Result of selected circle (f) Anusorn [2] and (g) Proposed Method in non-shadow group. * the green line of ground truth and the red line of the estimate REFERENCES with all kinds of the methods. [1] H. Sofian, J. C. M. Than, N. Mohd Noor and H. Dao, Fig. 14 Result of DATASET B: (a) Anusorn [5], (b) Proposed \"Segmentation and detection of media adventitia coronary artery Method. * the green line of ground truth and the blue line of the boundary in medical imaging intravascular ultrasound using otsu estimate of the proposed methods. thresholding,\" 2015 International Conference on BioSignal Analysis, Processing and Systems (ICBAPS), Kuala Lumpur, CONCLUSION 2015, pp. 72-76. In this paper, an approach using Circular- like Estimating Techniques to locate the Tunica [2] A. Wong-od, A. Rodtook, S. Rasmequan and K. Chinnasarn, Media on low-quality Intravascular image is \"Automated segmentation of media-adventitia and lumen from proposed. The proposed methodology started by intravascular ultrasound images using non-parametric removing the noise using Median Filter method. thresholding,\" 2017 9th International Conference on Knowledge Then, determine the scope of the tunica media using and Smart Technology (KST), Chonburi, 2017, pp. 220-225. a method to find the circle shape in the given boundary. When getting centroids of all possible [3] Mehdi Faraji, Irene Cheng, Iris Naudin, Anup Basu, circles, the seed point was selected from the one with Segmentation of arterial walls in intravascular ultrasound cross- the lowest entropy from all centroids. The candidate sectional images using extremal region selection, Ultrasonics, objects were selected using Euclidean Distance Volume 84, 2018, Pages 356-365, ISSN 0041-624X. measure. After that, all the candidate objects were, and a contour exaction was applied to locate the [4] J. H. Lee, Y. N. Hwang, G. Y. Kim and K. Sung Min, Tunica Media as shown in Fig. 12 -14. \"Segmentation of the lumen and media-adventitial borders in intravascular ultrasound images using a geometric deformable model,\" in IET Image Processing, vol. 12, no. 10, pp. 1881-1891, 10 2018. [5] A. Wong-od, A. Rodtook, S. Rasmequan and K. Chinnasarn, \"Intravascular ultrasound image recovery and segmentation based on circular analysis,\" 2017 9th International Conference on Information Technology and Electrical Engineering (ICITEE), Phuket, 2017, pp. 1-6 [6] Angioplasty.Org staff. “Intravascular Ultrasound (IVUS).” Retrieved from http://www.ptca.org/ivus/ivus.html [7] Radiological Society of North America, Rsna, & American College of Radiology. Intravascular Ultrasound. Retrieved from https://www.radiologyinfo.org/en/info.cfm?pg=ultrasound- intravascular [8] Sinha, Utkarsh. “Circle Hough Transform.” AI Shack, www.aishack.in/tutorials/circle-hough-transform [9] “Jaccard Index / Similarity Coefficient.” Statistics How To, 18 Mar. 2018, www.statisticshowto.datasciencecentral.com/jaccard- index/. 88

2019 4th International Conference on Information Technology (InCIT2019) Medical Device Authentication and Authorization Protocol in Indonesian Telemedicine Systems Setianto, YB Dwi Wahyuningrum, Shinta Estri Faculty of Computer Science Faculty of Computer Science Soegijapranata Catholic University Soegijapranata Catholic University Semarang, Indonesia Semarang, Indonesia [email protected] [email protected] Abstract— Indonesia is an archipelago comprising Nowadays, however, the existing telemedicine security approximately 13.000 islands. This geographical condition is systems have mainly focused on data and data one of the challenges in distributing equal health development communication. There are only very few telemedicine services throughout Indonesia. Telemedicine is one way to security systems that concern about authentication, overcome spreading it as a type of medical service that can authorization and standartization / certification of medical overcome geographical distance constraints. devices. Medical devices are vital parts of the telemedicine system; The Indonesian government is committed to protect its they need to be standardized so that they will not provide invalid citizens from the use of unstandardized medical devices by data. This is in accordance with the applicable law in Indonesia. issuing a law in 2009. It is called UNDANG-UNDANG No.36 Every medical device used in health services must be tested and / 2009 Pasal 103 ayat 1 which states \"Safeguarding calibrated periodically by the Health Facilities Testing Center pharmaceutical preparations and medical devices is held to (BPFK) and / or an institution of testing of authorized health protect the public from harm caused by the use of facilities. pharmaceutical preparations and medical devices who do not meet the requirements for quality and / or security and / or This paper proposes a new authentication and authorization efficacy / benefit \". Technically, every medical device used in protocol, which enables authentication and authorization of health services must be periodically tested and calibrated by medical devices digitally from the authorities (BPFK). With this the Health Facility Testing Center (BPFK) and / or institution protocol, information about the validity of test and calibration for testing health facilities in accordance with another law that of medical devices used can be conveyed well in the telemedicine is Undang Undang No. 44/2009 Pasal 16 ayat 2. system. Periodic testing evidence and calibration of medical The protocol mechanism provides guarantees for devices by BPFK are represented by official stickers or control authentication and authorization of medical devices in the sheets affixed to each medical device that has been tested and telemedicine system. This protocol also guarantees calibrated as in Figure 1. This official sticker or control sheet confidentiality and freshness of the data sent from medical is used as a guide for medical personnel to ascertain whether devices to health facilities. medical devices function properly. Keywords—telemedicine, security protocol, device security, e- Fig. 1. Proof of Periodic Testing and Calibration of BPFK health, health device. I. INTRODUCTION Telemedicine is the use of information and communication technology in the clinical health field to overcome distance constraints [3]. The concept of Telemedicine began in 1950 with the idea of Gershon-Cohen through his article \"Telonogsis\", Telonogsis (Teleo, Roentgen, and Diagnosis) using facsimile technology to send roentgen results from Pennsylvania to Philadelphia [10]. Telemedicine in the course of time continued to develop and evolve into modern Telemedicine today. Modern telemedicine is now increasingly accepted by health services and by patients. Telemedicine is also one method of reducing patient health costs [3]. The telemedicine pilot project in Indonesia has been initiated by the government since 2012 with a single feature of teleradiology, and it has continued to develop until 2016. Currently Indonesian government telemedicine includes tele- radiology, tele-electrocardiogram and tele-ultrasonography [8]. Data security and tools are important issues in the development of telemedicine, since poor telemedicine security can adversely affect the health services provided [4]. 89

2019 4th International Conference on Information Technology (InCIT2019) In the telemedicine system, medical personnel are far from IV. PROPOSED PROTOCOL the medical devices, so the sticker method and official control A. System Assumptions sheet are irrelevant, because they visually cannot be observed by health workers on duty. A mechanism is needed that Several assumptions that must be fulfilled so that the enables the delivery of information and validation of medical proposed protocol can run according to the design: devices integrated in the telemedicine system. 1. Every sensor, sensor box and site gateway produced This paper proposes a new authentication and by the manufacturer must have an identity attached authorization protocol, which enables authentication and to the hardware. Like the MAC address on the authorization of medical devices digitally from the authorities Network Interface Card. (BPFK), so that only routine tested and calibrated tools can be included in the telemedicine system. 2. Every sensor, sensor box and site gateway produced by the manufacturer must provide key store II. TELEMEDICINE AS AN ALTERNATIVE SOLUTION FOR mechanism. The mechanism must save the key in PROVIDING HEALTH SERVICES THROUGHOUT INDONESIA encrypted format. Only authenticated and authorized person can change the key with specific Indonesia is the largest archipelagic country in the world password. consisting approximately 13,000 islands [6]. This geographical condition has been one of the challenges 3. Sensors, sensor boxes, and sites gateway belong to / distributing equal health development services throughout the under management from the Health Facility. country. In addition, the number of specialist doctors in Indonesia is not as many as needed for the population. 4. Each sensor, sensor box, and site gateway is checked Specialist doctors mostly work in big city cities, 65% of and periodically calibrated by the medical device specialist doctors only want to practice on the island of Java authority. Check and calibration results and validity & the island of Bali, so 29% of Type C hospitals do not have period of the tools should be stored on the eHealth basic 4 specialist doctors [7]. Device Authority server This uneven distribution of specialist doctors certainly 5. This proposed protocol does not block the use of makes it difficult to provide advanced medical services for unauthorized devices, unregistered devices, or areas that are geographically far from urban areas. validity checks and calibration devices. This proposed protocol will only provide information on Telemedicine is one solution as it is a way to provide the validity of tools from the eHealth Device medical services that are able to overcome geographical Authority to the Health Facility, as additional distance constraints [1, 3, 10]. Telemedicine is also an information to produce the correct diagnosis. alternative way to reduce the medical costs of patients in the area [3]. Telemedicine in Indonesia began to be implemented B. Protocol Architecture in 2012 and up to 2016 already has 3 service categories The Proposed protocol architecture has 4 (four) main namely teleradiology, tele-EKG and tele-USG [8]. components, namely: eHealth Sensorbox, Site Gateway, III. TELEMEDICINE AUTHENTICATION & AUTHORIZATION Health Facility Application, and eHealth Device Authority CHALLENGES Server. The relationship between components can be illustrated in the figure below: The main issues that should be considered in implementing telemedicine in developing countries is Fig. 2. Proposed Protocol Architecture security, privacy and confidentiality [1, 9]. Various efforts were made to improve the security of telemedicine, starting from the use of biometrics [9], the use of watermarks [5], to the use of RFID technology [2]. Until now, there are not many studies done on the security of existing telemedicine, in other words special studies on this area are still considered lacking. From 1994 to 2009 there were only 66 articles in 14 Journal of Telemedicine, which discussed the safety of telemedicine. Very few of the 66 articles presented solutions to the telemedicine security system that holistically discussed the authentication and authorization of medical devices [4]. Medical devices are a vital part of the telemedicine system. Medical devices that are not standardized will provide invalid data. A holistic telemedicine security system that includes medical device authentication and authorization has become an urgent need in the telemedicine, especially the implementation of telemedicine in accordance to Indonesian legal regulations. 90

2019 4th International Conference on Information Technology (InCIT2019) the functions of the main component in this protocol as SàB : SID | DATA follows: eHealth Sensor sends Sensor ID (SID) and sensor 1. SensorBox: Is a pool of the sensor (s) that the eHealth reading data to SensorBox. This message is in plain text, used. One SensorBox can be connected to more than not encrypted. The message is not encrypted because the one eHealth sensors. The task of SensorBox is to communication path from the sensor and the box is a forward sensor data and health data from the sensor relatively safe path, and also because of the limited readings to the Site Gateway component. One patient computing resources of the sensor module. will need 1 Sensor Box. b) Sensor Box to Site Gateway Communication 2. Site Gateway: Is a pool of SensorBox (s). Site Gateway is a representation of the area where the Fig. 4. Sensor Box to Site Gateway Communication patient is cared for, for example in the patient's home. In one area more than one patient is possible. For that BàG : {{SID | BID | NB}KAB | SID | BID | DATA}KBF one Site Gateway can be connected with more than one SensorBox. The first component of the message from Sensor Box to the Site Gateway is the ticket {SID | BID | Note} 3. eHealth Device Authority Server : Is the major KAB. Tickets are encrypted with a KAB key, which is component in this protocol. In this component a a long term shared key between SensorBox (B) and database of medical devices which have obtained eHealth Device Authority (A). This ticket will be used usage permits is stored. In this component the valid by Health Facility Application to request Device period from the results of the test and periodic Validity Information from eHealth Device Authority calibration by the authorities is also stored. Data / (A). Information from these components will be needed by Health Facility Application as a reference for the The second component of the message from the validity / accuracy of medical data produced by Sensor Box to the Site Gateway is the Sensor ID (SID), medical devices. Sensor Box ID (BID) and sensory read data. All message components are encrypted with the KBF key, 4. Health Facility Application : Is the final component which is a long term shared key between the Health of the protocol that receives data from medical devices, Facility Application and the Sensor Box. and also data on the validity of medical devices from Device Authority Server. Users of this component are c) Site Gateway to Health Facility Application medical applications that doctors use to determine Communication diagnoses. This component will display a warning or information about the validity of medical devices Fig. 5. Site Gateway to Health Facility Application Communication located in the remote area of the telemedicine system. This warning or information can help the doctor to GàF:{{{SID|BID|NB}KAB |SID|BID|DATA}KBF|GID}KGF make a diagnosis. In general, Site Gateway only forwards messages C. Protocol Messages from the sensor box to the Health Facility Application. The message component is the same as the message 1) Notation received from the sensor box, plus the Site Gateway ID Messages used in the proposed protocol use the (GID). All message components are encrypted with long term shared keys between site gateway and health following notations : facility application. S : eHealth Sensor B : Sensor Box d) Health Facility Application and eHealth Device G : Site Gateway Authority Sever Communication F : Health Facility Application A : eHealth Device Authority NX : Timestamp made by X V : eHealth Device Validity Information KXY : Long Term Shared Key between X&Y XàB : Communication Message from X to B X | Y : Concatenate Message Y to X {X}Y : Message X encrypted using key Y SID : Sensor Module Identity BID : Sensor Box Identity GID : Site Gateway Identity FID : Health Facility Application Identity 2) Message Format a) eHealth Sensor to Sensor Box Communication Fig. 3. Sensor to Sensor Box Communication Fig. 6. Health Facility Application and eHealth Device Authority Sever Communication 91

2019 4th International Conference on Information Technology (InCIT2019) FàA : {{SID | BID | NB}KAB | FID | NF}KAF chosen with the consideration of the limited resource of AàF : {SID | BID | FID | NF | V}KAF medical device hardware. The message from the health facility application to the Guaranteeing message freshness is done by including the health device authority server consists of a ticket from the timestamp in the encrypted message sent. With the use of sensor box, and Health Facility Application Identity. All timestamp, messages resending which exceeds valid time message components are encrypted with KAF, long term frames will be detected as expired message. Expired shared key between Health Facility Application and eHealth messages will be ignored by the system. Server Authority Device. C. Symetric Key Management A reply message from the eHealth Device Authority Server, consists of a Sensor ID, Sensor Box ID, Health The main issue of symmetric cryptography is shared key Facility Application ID and the requested device validity secrecy. To increase the confidentiality of symmetric keys, information. All message components are encrypted with this protocol uses a long term key that is stored in eHealth KAF, long term shared key between Health Facility device storage. Application and eHealth Server Authority Device. As stated in System Assumption IV.A.2, that keys stored V. DISCUSSION in eHealth Devices cannot be read and changed by unauthenticated and unauthorized persons. It can only be Evaluation of the protocol mechanism in guaranteeing changed by eHealth Device Authority staff. This long term authentication and authorization of medical devices in the key is changed periodically by eHealth Device Authority staff telemedicine system, as well as guaranteed confidentiality during periodic checking and calibration. and freshness for messages sent are explained as follows: VI. CONCLUSSION A. Authentication & Authorization Assurance This Proposed protocol makes the functional validity of As described in the assumption system, sensor modules, medical devices as an inseparable part of the telemedicine sensor boxes, and site gateways have an identity (ID) attached system. Control of the functional validity of medical devices to the hardware. The same system is applied in the MAC by authorities is carried out digitally and integrated with the address on the Network Interface Card (NIC). telemedicine system. In addition to hardware ID sensor boxes and site gateways The mechanism in the proposed protocol guarantees have a long term key that is stored in their storage system. authentication and authorization of medical devices in the This long term key is set by e-health device authority when telemedicine system, and also guarantees confidentiality and checking and calibrating periodically. Long term keys are freshness for messages sent. also given by the health facility when registering medical devices to their system since the sensor module, sensor box ACKNOWLEDGMENT and site gateway are under management / ownership of the health facility. This research was funded by Direktorat Riset dan Pengabdian Masyarakat Direktorat Jendral Penguatan Riset This combination of hardware ID and long term key is dan Pengembangan, Kementerian Riset, Teknologi, dan used as an authentication method that is quite strong when the Pendidikan Tinggi. Research Contract No: protocol works. Each data transmission is always 010/L6/AK/SP2H.1/PENELITIAN/2019 Fiscal Year 2019. accompanied by a hardware ID that is encrypted with a different long term key. This long term key adjusted to the REFERENCES communication opponent so that only a valid entity is capable to send a hardware ID that matches the long term key. The [1] Alajmi, D., Almansour, S., & Househ, M. S. (2013, June). opponent must also confirm the encryption and decryption Recommendations for implementing telemedicine in the developing methods used. world. In ICIMTH (pp. 118-120). Authorization in this protocol is carried out by the entity [2] Bouet, M., & Pujolle, G. (2010). RFID in eHealth systems: eHealth Device Authority. The main authorization applications, challenges, and perspectives. Annals of parameters are valid periods from the results of periodic tests Telecommunications-annales des télécommunications, 65(9-10), 497- and calibration of medical devices. Medical devices that are 503. still in valid test periods and periodic calibrations are authorized devices in this protocol. Valid periods will always [3] Breen, G. M., & Matusitz, J. (2010). An evolutionary examination of be updated when every test and periodic calibration is done telemedicine: A health and computer-mediated communication to medical devices. perspective. Social work in public health, 25(1), 59-71. B. Message Confidentiality & Freshness Assurance [4] Garg, V., & Brewer, J. (2011). Telemedicine security: a systematic review. Journal of diabetes science and technology, 5(3), 768-777. In order to perform a guarantee of Message Confidentiality, every message sent through a public network [5] Guo, X., & Zhuang, T. G. (2009). A region-based lossless is always encrypted. The encryption uses encryption methods watermarking scheme for enhancing security of medical data. Journal that have been agreed upon by entities that are involved in the of digital imaging, 22(1), 53-64. communication. The encryption uses a long term key that is given and stored in the storage system of every medical [6] Hill, H., Resosudarmo, B. P., & Vidyattama, Y. (2009). Economic device. This type of cryptographic method for this protocol is geography of Indonesia: location, connectivity, and resources. a symmetric method. Symmetric cryptography method is Reshaping economic geography in East Asia, 115-134. 92

2019 4th International Conference on Information Technology (InCIT2019) [7] Mustikowati, S. M. S. (2006). Faktor-faktor yang Mempengaruhi Penerimaan Penempatan Dokter Spesialis Ikatan Dinas. Jurnal Manajemen Pelayanan Kesehatan, 9(02). [8] Yankes, Kemkes. (2017). Dirjen Pelayanan Kesehatan Serahkan Perangkat Telemedicine Kepada Pihak Pengampu Dan Diampu. http://www.yankes.kemkes.go.id/read-dirjen- pelayanan-kesehatan- serahkan-perangkat-telemedicine-kepada-pihak-pengampu-dan- diampu-3040.html diakses 23 Agustus 2018 [9] Zhang, G. H., Poon, C. C., Li, Y., & Zhang, Y. T. (2009, September). A biometric method to secure telemedicine systems. In Engineering in Medicine and Biology Society, 2009. EMBC 2009. Annual International Conference of the IEEE (pp. 701-704). IEEE. [10] Zundel, K. M. (1996). Telemedicine: history, applications, and impact on librarianship. Bulletin of the Medical Library Association, 84(1), 71. 93

2019 4th International Conference on Information Technology (InCIT2019) iDahon: An Android Based Terrestrial Plant Disease Detection Mobile Application Through Digital Image Processing Using Deep Learning Neural Network Algorithm John C. Valdoria Arlene R. Caballeo Brian Irvin D. Fernandez College of Technology College of Technology College of Technology Lyceum of the Philippines University Lyceum of the Philippines University Lyceum of the Philippines University Manila, Philippines Manila, Philippines Manila, Philippines [email protected] [email protected] [email protected] John Marco M. Condino College of Technology Lyceum of the Philippines University Manila, Philippines [email protected] Abstract—This research focuses on the detection of common the plant diseases. Further, this study was intended to educate diseases on terrestrial plants found in the Philippines through the people who are beginners or have little knowledge about use of image processing through Deep Learning Neural Network. horticulture. Through the developed application, the users can Android-based smartphones were used to capture images of the simply capture the image of the leaf and then help them terrestrial plant to detect the disease of the plant while Deep identify the disease on plant. The users can save time and Learning Neural Network Algorithm is utilized to distinguish the money, in finding or hiring a botanical expert, and allows the disease of the terrestrial plants. The results were trained using users to be more aware of the common terrestrial plant classification models that could identify the diseases at certain diseases. rate and accuracy considering the number of images used. Therefore, the Deep Learning Neural Network classification II. RELATED LITERATURE AND STUDIES demonstrates the identification of the common disease terrestrial A. Digital Image Processing plants found in the Philippines Digital image processing was used to perceive and bunch Keywords— Digital Image Processing, Deep Learning infectious disease reactions impacted on different Neural Network, Terrestrial Plant, Plant Pathology cultivating/horticulture crops. Personal Computers (PC) have been used to mechanization, and to make decision candidly I. INTRODUCTION strong system for taking the fundamental decision on the country's age and security to inspect. The plant affliction end The agricultural landmass is more than just feeding is limited by the human visual limits because a substantial bit sourcing in today's world. But despite the predominant benefits of the principal signs are minute. As plant prosperity watching that agriculture has offered to the world, it faces various is as yet done by individuals due to the visual thought of the challenges to support a vast population [1]. Several existential plant checking undertaking. PC vision techniques have all the problems threaten agriculture worldwide. The growth of the earmarks of being particularly balanced. The sum and nature human population and the decline of arable land are significant of plant things get decreased by plant diseases. Spotlight has threats. The increasing output has been the major priority of been done in the early area of infectious disease reliant on the modern agriculture in order for the world to have a secure food reaction [3]. supply, this simple goal has been said to promote excessive B. Deep Learning Neural Network usage of resources such as agrochemicals, one of the sources of unsustainable agricultural environments, which in turn, There are numerous techniques in computerized or PC makes plants more vulnerable to pathogen attack and plant vision plant ailment identification and grouping process, yet at diseases harder to control [2]. the same time, this exploration field is deficient [4]. Also, there are still no business arrangements available, aside from those In the present time, the current method of detecting these managing plant species acknowledgment dependent on the plant diseases are only made through the naked eye. For this to leaves pictures. In this paper, another methodology of utilizing be done, a team of experts must be required to monitor a plant profound learning strategy was investigated to naturally as required which then costs very high if done on a larger characterize and identify plant infections from leaf pictures. scale[1]. The created model could identify leaf nearness and Through these realizations in past studies done by various recognize solid leaves and diverse illnesses, which can be researchers, this study aims to develop a method of plant outwardly the creators want to accomplish a significant effect disease detection using image processing integrated in an on economic advancement, influencing crop quality for who Android application that can detect common diseases on and what is to come. The technique depicted in the framework terrestrial plant found in the Philippines. This study further is another methodology in distinguishing plant illnesses sought to develop an application that can serve as a tool to help utilizing the profound convolutional neural system prepared to lessen plant deaths by identifying the diseases at an early stage without spending significant amount of time recognizing 94

2019 4th International Conference on Information Technology (InCIT2019) and adjusted to fit precisely to the database of a plant's leaves Figure 1 shows the common terrestrial plant diseases that was assembled freely for different plant ailments [4]. explored in the study namely [a] Bacterial Blight, [b] Leaf Spot, [c] Chlorosis, [d] Powdery Mildew, [e] Sooty Mold, [f] C. TensorFlow Leaf Rust, [g] Fusarium Wilt, [h] Leaf Scald, [i] Leaf Blister, and [j] Leaf Scorch diseases. Machine learning, particularly the flavor known as significant learning, controls Google's ability to decipher The [a] Bacterial Blight is the major bacterial disease in the talked look for requests and to see a face in a photo. By cassava belt worldwide. Cassava production is inclined to releasing their in-house-developed significant learning numerous limitations all through the production cycle, framework TensorFlow as open-source programming. Google including biotic, abiotic, and executive limitations [8]. The has given a consistent stage to significant learning image [b] shows the Cercospora leaf spots caused by investigation and applications. The possible results for Cercospora arachidicola and Cercosporidium personatum [9]. TensorFlow in natural research are enticing, at any rate This plant disease is seen as a small brown spots develop with offering computational researchers a more regulated structure a reddish edge border varies from a small circular spots up to that is most likely going to improve ease of use and 4mm wide with a gray-ash center. This disease is caused by reproducibility of significant learning techniques [5]. the fungi Cercospora beticola, C. capsici, C. nasturtii, C. canescens and C. coffeicola. D. Plant Pathology The images [c], [d], and [e] shows the Chlorosis disease A plant disease is generally characterized as a strange from the plant, the Powdery Mildew present in a terrestrial development and additionally brokenness of a plant. Diseases plant, and the Sooty mold that is a charcoal dark growth in the are the consequence of some unsettling influence in the plant, respectively. The Chlorosis disease infers a general ordinary life procedure of the plant. Biotic diseases are drying of the leaves. Dapper formed dull shaded spots may brought about by living beings (e.g., organisms, microscopic make among veins and leaf edges may scorch (dim hued along organisms, and infections). Abiotic diseases are brought about the edge). Iron is basic for the improvement of chlorophyll, by non-living ecological conditions, (e.g., soil compaction, which is accountable for the green shading in plants and key wind, ice, soil salt harm, and supporting roots) [6]. for photosynthesis (sugar age in plants) [10]. While the Powdery Mildew disease can influence various plants, This section discussed the common terrestrial plant including natural product, vegetable, and agronomic harvests, diseases explored in this study. The following presents the just as woody and herbaceous ornamentals. Powdery mildew images of plant diseases present in plants. The images of these is effectively distinguished by the nearness of white, tan, or plant with plant diseases were captured and process through dark powdery contagious development (mycelium and spores) training model utilized in this study. that is available essentially on surfaces of contaminated plant parts [11]. On the other hand, the Sooty mold shows up as a [a] Bacterial Blight [b] Leaf Spot dark covering on the outside of leaves, natural products, twigs also, parts of numerous deciduous and evergreen bushes and [c] Chlorosis [d] Powdery Mildew trees. This growth isn't pathogenic to plants yet acquires its sustenance from creepy crawly honeydew. When spores grow, [e] Sooty Mold [f] Leaf Rust they convey dark parasitic strands (mycelium) that cover the plant tissue and cause the staining [12]. [g] Fusarium Wilt [h] Leaf Scald Further, the images [f], [g], and [h] presents the Leaf Rust, [i] Leaf Blister [j] Leaf Scorch Fusarium Wilt, and Leaf Scald diseases which are commonly seen in terrestrial plants. The Leaf Rust is commonly known Fig. 1. Common Terrestrial Plant Diseases as “coffee leaf disease” which was first revealed by an English traveler on wild Coffea species in the Lake Victoria locale of East Africa in 1891 [13][15]. While the Fusarium were accounted for as the most widely recognized soil borne parasite pathogen in causing spoiled foundation of roselle [14]. The Leaf Scald on the other hand is a disease caused by bacterium Xanthomonas albilineans, characterized by creamy or grayish streaking and later withering of the leaves [15]. Moreover, the images [i] Leaf Blister and [j] Leaf Scorch were the last two terrestrial pant diseases explored in this study. The Leaf Blister or also called Tarphrina blister is a disease caused by the parasite Tarphrina caerulescens which is typical sickness influencing numerous types of oaks. Individuals from the red oak aggregate are especially vulnerable to contamination. Substantial diseases of red oaks hinder their appearance however don't jeopardize the tree wellbeing dark-colored with age. A few blisters may 95

2019 4th International Conference on Information Technology (InCIT2019) consolidation and cause the leaves to twist [16]. While the plant has a detected disease, the application will also show the Leaf Scorch plant disease is a sickness that tissue between the possible remedy for that specific plant disease. principle veins may be influenced causing leaf spots. In B. System Design serious cases, whole leaves will turn dark colored or dark and pass on. If the condition continues, needles will turn dark- Load Training Process Datasets using Untrained Neural colored or create darker groups and may drop off [17]. Datasets TensorFlow Library Network Model E. F1 Score F1 Score is used to get the harmonic mean of precision and recall giving both metrics equal weights as illustrated in Fig. 2. The goal of the F1 score is to minimize both the false positive and false negative of a good model. F1 = 2 * precision * recall precision + recall Fig. 2. F1 Score Test Accuracy of Configured Neural Network Configure Training Training Model III. METHODOLOGY Training Model using Model using A. Conceptual Framework Python Python Tested Training Model Plant disease analysis using Deep Learning Implemented Model on Neural Network Algorithm iDahon Application Fig. 4. System Design Using Deep Learning Neural Network Training Diagram Detected Disease iDahon App Fig. 4 shows the system design using deep learning neural Results network algorithm. This shows how the training model was Capture plant image implemented in the iDahon application developed in the study. using iDahon App The images of the healthy plants and the terrestrial plant with diseases were gathered which served as the datasets for training iDahon Users model. The datasets were processed through the Tensorflow library. Once processed, the training model was configured Fig. 3. Conceptual Framework of the Study using python to be able to detect the terrestrial plant diseases via mobile based application. This deep learning neural Fig. 3 depicts the conceptual framework used in this study. network trained model was simulated through Docker software The iDahon application is used by the user to capture the leaf to test its accuracy [7]. This was done to determine if the model of the terrestrial plant to determine its health condition. The can already detect the terrestrial plant diseases or whether it captured image will then process and analyze using Deep can already distinguish between healthy plant and plant with Learning Neural Network Algorithm implemented in the disease. The last stage was the implementation of the trained developed iDahon application to determine the condition of the model or the inference stage where the new data model is plant. This is based on the Cross Eentropy, Train Aaccuracy applied using the android studio. In this inference stage, the and Validation accuracy of the pixels from the sample image iDahon application can now be utilized to detect whether the or photo. The output will show the result based on the analysis terrestrial plant has a disease or a healthy plant. using the trained model. The result will display whether the terrestrial plant is healthy or in poor condition. If the terrestrial 96

2019 4th International Conference on Information Technology (InCIT2019) C. System Architecture Table 1 shows the dataset used. Overall, it consists of 900 images of different terrestrial plant abnormalities. Plant with a health condition that consists of 90 images is also included as a negative model among others to help the model accurately identify the terrestrial plant with the disease. User TABLE II. F1 SCORE Android Smart Phone Class Precision Recall Bacterial Blight 0.854 0.900 Camera Module Leaf Spot 0.970 0.990 Chlorosis 0.980 0.910 Detection of Terrestrial Gallery Module Powdery Mildew 0.976 0.978 Plant Disease Sooty Mold 0.976 0.983 Leaf Rust 0.976 0.975 Fusarium Wilt 0.976 0.985 Leaf Scald 0.980 0.980 Leaf Blister 0.980 0.980 Leaf Scorch 0.980 0.980 *Healthy Plant 0.984 0.997 Total 0.965 0.969 Fig. 5. System Architecture Table 2 shows the F1 score result that indicates the accuracy of the detection of health conditions and diseases of Fig. 5 shows the system architecture of the iDahon terrestrial plants. The F1 score which is calculated using the application. It also shows the interaction between the user, given precision and recall from the confusion matrix. The table software application and the hardware. The architecture shows shows that class Bacterial blight obtained the lowest prediction that the hardware which is the android based smartphone uses rate due to the common characteristics observed on bacterial a camera for the detection of terrestrial plant disease which is blight also seen on other plant diseases. On the other hand, the the main module of the application. It also shows that disease Healthy Plant obtained the highest number of correctly detection has two modules where the user can determine the predicted data. The gathered data from the confusion matrix disease of the terrestrial plant. The Camera Module is used as was used to get the precision and recall of each class. a tool to take a picture of the leaf of the plant. The Gallery module serves as the repository of the captured photo of the Plant TABLE III. CONFUSION MATRIX % leaves of the plant stored in the gallery section of the Diseases TPa % FPa % FNa smartphone. Bacterial Blight 109 72.67% 25 16.67% 16 10.67% Leaf Spot 122 81.30% 17 11.30% 11 7.30% IV. RESULTS Chlorosis 134 89.30% 7 4.67% 9 6.00% Powdery 128 85.30% 7 4.67% 15 10.00% A. Datasets Mildew 129 86.00% 9 6.00% 12 8.00% The proponents used 1650 images as datasets in the Sooty Mold 118 78.67% 14 9.33% 18 12.00% 123 82.00% 13 8.67% 14 9.33% training model of the Deep Learning Neural Network Leaf Rust 136 90.67% 5 3.33% 9 6.00% Algorithm implemented in the iDahon Application. The size of Fusarium 135 90.00% 7 4.67% 8 5.33% the images used is 500 x500 pixels and in jpeg format. 60% of Wilt 131 87.30% 6 4.00% 13 8.67% the images which consists of 1650 images were used to train Leaf Scald 138 92.00% 3 2.00% 9 6.00% the model. 20% of the images were used as a test image which consists of 330 images, and the remaining 20% is used to Leaf Blister validate the output of the training model that consists of 330 images. Leaf Scorch Healthy TABLE I. DATASETS Plant Plant Diseases Trained Test Validation Total a. Legend: TP – True Positive, FP – False Positive, FN – False Negative. Images Images Bacterial Blight 30 150 Table 3 shows that the Healthy plant training model Leaf Spot 90 30 30 150 obtained the highest number of the predicted conditions of Chlorosis 90 30 30 150 plants. Conversely, the bacterial blight training model obtained Powdery Mildew 90 30 30 150 the lowest number of detected disease. Additionally, the Leaf Sooty Mold 90 30 30 150 Blister obtained the least number of False Negative. Leaf Rust 90 30 30 150 Fusarium Wilt 90 30 30 150 Leaf Scald 90 30 30 150 Leaf Blister 90 30 30 150 Leaf Scorch 90 30 30 150 *Healthy Plant 90 30 30 150 Total 90 30 330 1650 990 330 97

2019 4th International Conference on Information Technology (InCIT2019) B. System Performance Result REFERENCES TABLE IV. IDAHON SOFTWARE APPLICATION PERFORMANCE [1] Singh, V. & Misra, A. K. (2017). Detection of Plant Leaf Diseases EVALUATION BY THE EXPERTS Using Image Segmentation and Soft Computing Techniques, Information Processing in Agriculture. Criteria Mean (x̄ ) Verbal http://dx.doi.org/10.1016/j.inpa.2016.10.005 Interpretation Functionality 4.68 [2] Zhan, J., Thrall, P., & Burdon, J.(2014).Crop pathogen emergence and Reliability 4.47 Excellent evolution in agro ‐ ecological landscapes. Usability 4.46 Very Good https://doi.org/10.1111/eva.12251 Efficiency 4.56 Very Good Accuracy 4.72 Excellent [3] Pujari, J. D., Yakkundimath, R., & Byadgi, A. S. (2015). Image Portability 4.63 Excellent Processing Based Detection of Fungal Diseases in Plants. Procedia Total 4.59 Excellent Computer Science, 46, 1802–1808. https://doi.org/10.1016/J.PROCS.2015.02.137 Excellent [4] Sladojevic, S., Arsenovic, M., Anderla, A., Culibrk, D., and Stefanovic, Table 4 shows the iDahon software application D. (2016) Deep Neural Networks Based Recognition of Plant Diseases. performance evaluation made by the experts. As gleaned from Leaf Image Classification, Department of Industrial Engineering and the table, it revealed that two of the criteria in the performance Management, Faculty of Technical Sciences, University of Novi Sad, evaluation of iDahon App was rated “Very Good” and Trg Dositeja Obradovica. majority of the criteria was rated “Excellent”. As shown in the table, “Reliability” and “Usability” criteria were rated of [5] Rampasek, L., & Goldenberg, A. (2016). TensorFlow: Biology’s “Very Good” with weighted means of 4.47 and 4.46, Gateway to Deep Learning? Cell Systems, 2(1), 12–14. respectively. As the data disclosed, the “Functionality”, https://doi.org/10.1016/J.CELS.2016.01.009 “Efficiency”, “Accuracy”, and “Portability” were rated “Excellent” with weighted means of 4.68, 4.56, 4.42, and [6] Diseases, B., & Plants, P. (2017). Plant Pathology, 1–19. 4.63, respectively. It was noted that the experts rated the iDahon App positively since the overall mean of the iDahon [7] Ferentinos, K. P. (2018). Deep learning models for plant disease software performance evaluation is 4.59 with a verbal detection and diagnosis. Computers and Electronics in Agriculture, 145, interpretation of “Excellent”. The experts commended the 311–318. https://doi.org/10.1016/J.COMPAG.2018.01.009 application in producing accurate results in detecting terrestrial plant diseases. [8] Harris, K. P., Martin, A., Novak, S., Kim, S., Reynolds, T., & Anderson, C. L. (2015). Prepared for the Agricultural Development On the other hand, the experts suggested minor Team of the Bill & Melinda Gates Foundation, (298). improvement in terms of the descriptions in terrestrial plant diseases. [9] Debele, S., & Ayalew, A. (2015). Integrated management of Cercospora leaf spots of groundnut (Arachis hypogaea L .) through host resistance V. CONCLUSION and fungicides in Eastern Ethiopia,), 82–89. https://doi.org/10.5897/AJPS2014.1260 This study explored the Deep Learning Neural Network Algorithm which has the capability in detecting terrestrial plant [10] Roots, T. G. (2015). Iron Chlorosis of Woody Plants, 1–7. disease with accuracy based on F1 score testing method and implementation of the application under the evaluation of [11] Pfeufer, E., Gauthier, N. W., & Bradley, C. A. (2017). Powdery Mildew, experts in the field of botany and agriculture. Through the use 4–7. of publicly images gathered which served as the datasets in creating the training model in detecting terrestrial plant [12] Strategies, M. (2015). Sooty Molds : various, 1–2. diseases, the result showed that the developed application has an 80% accuracy rate; thus supporting the functionality, [13] Arneson, P. A. (2011). Coffee rust, 1–10. https://doi.org/10.1094/PHI- reliability, usability, efficiency, accuracy, and portability I-2000-0718-02 performance of the application. This shows that the android based mobile application has an excellent capacity in detecting [14] Ng, L. C., Ismail, W. A., & Jusoh, M. (2017). Research Article In vitro common terrestrial plant diseases in the Philippines. This Biocontrol Potential of Agro-waste Compost to Suppress Fusarium application can be utilized and can be applied as a tool to oxysporum , the Causal Pathogen of Vascular Wilt Disease of Roselle. prevent and raise awareness on common diseases found in https://doi.org/10.3923/ppj.2017.12.18 terrestrial plants. [15] Dalbó, M. A., Bruna, E. Dela, & Luiz, A. (2018). SCS 438 Zafira – a ACKNOWLEDGMENT new plum cultivar resistant to leaf scald ( Xylella fastidiosa ), 229–233. The researchers deeply acknowledge the academic [16] Cycle, D. (2015). Oak Leaf Blister: Taphrina caerulescens, Plant institution—Lyceum of the Philippines University—for the Pathology and Plant‐Microbe Biology Section 1–2. support throughout the development of this study, and above all, our honor and praise to the Lord God Almighty, for He [17] Rose S. & Swift C.E. (2014). Leaf Scorch, Gardening Series: Deseases deserves all the glory. Colorado State University Extension Fact Sheet No. 2.911. https://extension.colostate.edu/docs/pubs/garden/02911.pdf 98

2019 4th International Conference on Information Technology (InCIT2019) Coin Recovery from Inaccessible Cryptocurrency Wallet Using Unspent Transaction Output Pongpat Rakdej Nanta Janpitak Faculty of Information Science and Technology Faculty of Engineering at Sriracha, Mahanakorn University of Technology Kasetsart University Sri Racha Campus, Bangkok, Thailand [email protected] Sri Racha, Chonburi, Thailand [email protected] Maykin Warasart Woraphon Lilakiatsakun Digital Innovation Center, Digital Government Faculty of Information Science and Technology Development Agency (Public Organization) (DGA) Mahanakorn University of Technology Bangkok, Thailand Bangkok, Thailand [email protected], [email protected] [email protected] Abstract — In 2008, the blockchain technology was invented block according to mining algorithm such as proof of work in to create a modern electronic cash transaction protected bitcoin to find the right current block’s hash value. After that, through cryptographic mechanisms in peer to peer or the verified block will be distributed and stored in database decentralized network. The first success cryptocurrency based which linked to previous block together with previous block’s on blockchain technology is Bitcoin, which core processes are hash value inside the block. The next block will also include based on public key cryptography. While the public key is current block’s hash value. The basic theory behind transformed and used as an address for payment, the private blockchain is hashing. Hashing or cryptographic hash function key is used to claim ownership to access the funds stored in the is a one-way function that takes data of any size as input and correspondent address. The processes of generating addresses, produces an identical fixed length output. Hash value has signing transactions, and checking the balance are parts of the properties such that it will be the same value for the same input cryptocurrency core processes. With these activities, the wallet and if input has changed just a little, hash value will differ. application was created to assist and handle the execution of Hash computation is very fast, but the process to find the input those processes. is nearly impossible. Hence the relationship of linking block with previous block’s hash number in blockchain makes it The private key plays the most important component to extremely difficult for an attacker to tamper the information interact with assets inside cryptocurrency blockchain database. in a block because all subsequent blocks will have to be Without the private key, the asset in blockchain will not be regenerated, which will be detected because the hash wouldn’t managed or controlled, thus all cryptocurrency coins in match. correspondent wallet will be lost. This paper provides a solution to recovery cryptocurrency coin that lost according to inability The main task of cryptocurrency is transferring the to login to wallet application. The main component used to ownership of assets (or coins). The basic transaction consists recover coins in this paper is the unspent transaction output. of two counterparties, sender and receiver, such that they exchange the information to each other as shown in Figure Keywords—Cryptocurrency, Blockchain, Wallet, Private key, 1[2]. The input of the transaction is transferable coins with Coin Recovery digital signature proves that the sender has ownership in those coins. While the output of transaction is coin receivers (in I. INTRODUCTION form of cryptocurrency address) with condition (if any) and the amount of coins to be sent. A transferable coin in Cryptocurrency is a digital money that has been cryptocurrency is represented in the form of previous implemented by using blockchain technology with peer to transactions and has not yet spent or unspent transaction peer networks. As a distributed database feature in blockchain, output (UTxO). To use the UTxO as an input of transaction, there is no central storage and no governmental body to sender must certify his identity by signing the digital signature control overall implementation among financial institutes, which employs public-key cryptography. With public-key corporates or even government itself. In 2008, the concept of cryptography, the signature signed by the sender's private key blockchain was incorporated into other computational concept can be verified by anyone who has access to the sender's technologies to create cryptocurrency. The pioneer of Bitcoin public key. The public-key cryptography algorithm for which concept is the anonymous named \"Satoshi Nakamoto\" most of cryptocurrencies are used is elliptic curve digital published Bitcoin: A Peer-to-Peer Electronic Cash System [1] signature algorithm (ECDSA) as standardized by NIST [3]. in which electronic money (Digital Money or Electronic Cash The private key is used to generate a public key and System) is protected through an encryption mechanism cryptocurrency address. Then Sender must produce his instead of central data collection or authority, later this is the signature with private key corresponding to receiver address foundation of blockchain and other cryptocurrencies ever of previous transaction output to prove his ownership over since. UTxO. And the private key must be kept secretly and carefully. The blockchain technology in cryptocurrency is a distributed public ledger of transactions. Verified transaction As a cryptocurrency user, the must-have item in the first data will be accumulated and stored as a block. The block place is a wallet which its main function is used to generate consists of several transactions, the block also contains and keep both the private key and the address. The wallet also metadata such as the previous block’s hash value, timestamp, has other functions that facilitate users to interact with block size, transaction counter, and current block’s hash value. The block will be created (mined) by miner to verify the right 99

2019 4th International Conference on Information Technology (InCIT2019) Fig.1 Input-ouput of transsactions and UTxO. Fig.2 Bitcoin regular transaction structure and output reference network such as create a transaction. It is important that user III. RELATED WORK must keep the wallet away from the access by others. And then Without the private key, it was no way to redeem a UTxO there are many forms of authentication and many types of or to brute force for exactly private key corresponding to the software and hardware for wallet application. However, if the address is merely possible by current technology. The related wallet cannot be accessed by the owner, the coins according works in this proposed technique are the components or to the addresses in the wallet will not be used anymore. This applications of cryptocurrency that may facilitate to offer the is terrible to lose all of coins. second chance to use the UTxO. A. Transaction II. MOTIVATION A transaction in cryptocurrency is a data structure that According to the research from blockchain forensics firm stores the transfer of tokens between two parties. From the Chainalysis in November 2017 [4], about 3.79 million BTC Bitcoin regular transaction structure (as shown in Figure 2) have been lost forever. And fact sheet created by Crypto [8], there are some parts that can facilitate to make a Research GmbH in February 2019 [5], told that the number of transaction in advance. First, there is an nLocktime in the Bitcoin lost reached 3 million up from 6 million BTC, transaction which indicates the earliest time or earliest block meaning that 14.3% up to 28.6% of the maximum supply is when that transaction may be added to the block. It was lost permanently and the magnitude of such loss was worth contained in Bitcoin since beginning. There is an explanation around 20 to 45 billion USD for loss of Bitcoin that could not about nLocktime as – “An unrecorded open transaction which be circulated in market anymore. This is a huge loss and can be replaced until nLockTime.” [9]. This means we can became a pain for many people involved. Basically, there is make a transaction in advance and can replace any time before no private key to redeem the coins of the existing UTxO. nLocktime. To make transaction in advance has some The reasons for the inability to access the wallet include difficulties that it will not be included in mining block. Then the unintentional loss of password and the disappearance of if it is broadcasted before nLocktime longer than 48 hours, it the owner. In January 2019, CEO of the largest crypto will go out from network if rebroadcast is not made. exchange in Canada, QuadrigaCX, died suddenly [6]. Second, the scriptPubkey field in each transaction’s output Although QuadrigaCX keeps cryptocurrency in cold storage can imply a condition to holding time (OP_CHECKLOCK- wallet, no one can access. This caused QuadrigaCX to loss TIMEVERIFY) for receiver to redeem it. In this case, the 190 million USD and went to bankrupt in few weeks, also transaction can be included in a block, but it cannot be there are many addresses which are used to be visited for a changed or use the UTxO until the condition in scriptPubkey while, but later there is no movement for longtime such as is met. The condition in scriptPubkey is not flexible and Bitcoin address: - unable to change later on. For some other cryptocurrencies such as Ethereum, there 1HQ3Go3ggs8pFnXuHVHRytPCq5fGG8Hbhx are not these options at the current time. B. Wallet and Key management This address was accessed to send 101 BTC on Apr. 23, The distinctive feature of cryptocurrency is the private 2015[7], and still there is 69,370.082201 BTC (worth key. Cryptocurrency infrastructure did not provide private key approximately 500 million USD) balanced as a UTxO from management directly. Wallet is a user interface application to that transaction and there is no movement after that date. facilitate the key management for generating and broadcasting the transactions and later the key will be under user’s Considering these handicaps of unable to access UTxO due to the loss of private key is to be our primary objective of this paper. 100

2019 4th International Conference on Information Technology (InCIT2019) Resistant Kept OfNfloinTerusted Third Party to PhRyseisciasltaTnht etoft PhRyseiscialileOntbtsoerPvaasRtsioewsnoilridenLt otossKeIymCmheudrinate AccNesosNtoewFuUndsesr SCorfotwssa-dreevice Portability Key(s) Resistant Category Example Malware Keys in Local Storage Bitcoin Core ● ●●●● Password-protected Wallets MultiBit ○ ●○ ● ●● Offline Storage Bitaddress ○●● ● ● Air-gapped Storage Armory ○●● ●●● Password-derived Keys Brainwallet ●●○ ●●●● Hosted Wallet (Hot) ●●●●● Hosted Wallet (Cold) ○● ●● ●● Hosted Wallet (Hybrid) Blockchain.info ○○ ●●●●● Cash ●●● ●●●●●● Online Banking ●●●●● Table 1 A Comparison of key management technique [10] responsibility. S. Eskandari et al. [10] summarizes the key such as bio wallet [12]. But the problem will occur when management in varying levels of security and convenience as owner who has permission to access forgets his credential or in Table 1. We can categorize the methodology to manage the absents (or died), the wallet will not be able to use anymore. private key into 6 techniques: - Only hosted wallets which are third party service may provide 1. Keep the key as the file in local device’s storage – This other means of recovering the private key in this case. technique makes key easy to be accessed by wallet C. Recovering Private Key application and can keep unlimited private keys, but it has To recover credentials, many applications have some no protection against malicious attacks such as malware, etc. techniques such as sending a reset webpage through an e-mail 2. Password-protected (Encrypted) Wallets - locally store the or enquiring some personal questions that users had provided. key in wallet file and encrypt with a key derived from a But for highly secured system like cryptocurrency, it may lead user-chosen password or passphrase. to some threats that can be attacked by malicious hackers, or 3. Offline Storage of Keys - store the key offline on some in the case such that authorized owner may also forget, or in form of portable media such as paper or USB. The wallet the extreme case when the owner passes away. It seems like shall interact with storage to get the key for signing and we have a black box that no key can open it. There are still 2 broadcasting the transaction. possible ways to recover the private key. The first method is 4. Air-gapped Key Storage – store the key offline on a brute force technique. The private key should be completely portable device that can create transactions by itself, but it random, and there are 2^256 (256 bits for most of needs wallet application on online device to broadcast the cryptocurrencies) different keys that can be generated, so the transaction. chance of matching a private key to a public key is 1.55 5. Password-derived Keys – this technique is reproduced quattuorvigintillion (1.55 x10^77). If we use a PC that has a private key from the password or passphrase. The wallet performance of brute-forcing at 1 Gkeys/sec, it will take up to may have a file but does not have the key inside. 3.67x10^60 years. But if private key is made from memory of 6. Hosted Wallets – use web service to store the key such as the owner (brain wallet) [13] or from a poor random number the coin exchange; however, considering the amount of generator [14] or from guessing [15], it may increase little coins, most of the host wallet operators also manage their possible to attack for private key. Today 2,048 wordlists in coins in 2 forms – hot wallets for daily use or circulated BIP-39 [16] are recommended to use 12 to 24 words randomly and cold wallet to keep the coins safely from attacker. compound together as mnemonic seeds for generating From the six techniques of key management above, if we deterministic keys. Then it becomes harder for the attacker to do not keep the private key in plaintext file or paper, it means recover the key. that we must have some way to authenticate the wallet application or storage device to derive the key for The second method is to find some crumbs inside the authenticating the transaction. Up to this present time, many wallet owner’s device through forensic methodology. The techniques have been proposed which make storing and wallet application shall load and decrypt the private key into signing process more secure and protect from others memory when signing a transaction. Then if the computer is unauthorized access, the technique of using both passphrase not shut down, it may leave some crumbs to find the private and key seed together with hashing technique to derive a key in memory, but this is barely possibly as no user ever use private key for transaction signing, etc. [11]. Another factor a computer without shutting it down. There are a few of efforts authentication which has been implemented for long time to seeking some forensic artifacts in local storage such as access wallet file and became more sophisticated as time pass Doran’s research [16], he devised up his own testing environment (use multibit wallet application which is obsoleted on desktop PC) to survey what could get from the experiment and found the “wallet.dat” file which contains the user’s private keys and transactions. But further examination 101

2019 4th International Conference on Information Technology (InCIT2019) of this file revealed large amounts of unreadable Base64 text. Fig.3 The retire option working flowchart In Jan 2015, Cohen’s [17] published his article and his code BTCscan.py to seek for address or private key that may be left The first case, Alice opens the wallet application and inside a device. There are a few open sources of software that selects wallet file as normal. The application will check that can extract private key from encoded wallet.dat file such as the file has been set the retire option or not. If not, the pywalllet.py but not for encrypted file. application will ask Alice to set it. After that (even not to set the options) Alice can use the wallet as usual. IV. PROPOSED WORK The second case, after open wallet application and select A. Use case scenario wallet file that has been set the retire option, the application We start from a holistic view as typical cryptocurrency will check the time from the last login, calculate and compare against the latest Bitcoin’s bock header. If the difference in users, A year ago, Alice who has 100 BTC and 1,000 ETH in time is less than 6 months, Alice is still able to use the wallet. her wallet. She used a “desktop wallet application” installed The wallet application will automatically reset the last login on a computer at her office. For her safety, she configured her time and update the advance transaction when Alice logoff or wallet software to encrypt her “wallet file” as a general quit the application. practice and used long passphrase that never to tell anybody. One day she went to join a conference in Las Vegas and The last but not least, in our scenario, we assume that Alice unfortunately, she was robbed and hurt by someone and had set the retire option, set Bob’s address as the receiver became sleeping beauty for six months. Her husband, Bob had address, and set the time period to retire the wallet as 6 to pay for everything including her apartment that under months. Alice came back to her desk but cannot remember renovation. After a year pass, Alice became better and can go passphrase to log in her wallet file. It is already one year since to work but she cannot remember her passphrase to open her her last login. It is fortunately that she set the retire option she wallet file, thus wallet file was used last time one year ago. can tell Bob prepare to receive her transfer from the retire The situation like this means she may lose all coins which option and transfer back to her later. After she opens the wallet worth more than one million USD at today's price. application and selects her wallet file, the wallet application checks the wallet file last login time and compares against the B. A Chance to recover unspent coins. latest Bitcoin’s bock header. The difference of time is more As mention before, the coins left in Alice’s wallet are the than 6 months, then the retire options are activated immediately. The wallet application will broadcast the output of the previous transaction which is correlated to her predetermined transaction and transfer all of UTxOs from the address that can derived from private key that kept in the last login time (the time that made this transaction). Now the wallet file. But if Alice cannot open her wallet file, she cannot coins inside her wallet do not lose forever. Alice could recover create any transaction to transfer her coins. However, if there her fortune again. is an option that can make a transaction in advance every time C. Technical behind this proposed work. for every UTxOs in every address to transfer all of her coins 1. In Bitcoin, there is an nLocktime field, which is the part of to reserved address before she quit the wallet application and re-create it whenever the wallet file was logged in to keep it a transaction, indicates the earliest time or earliest block updated the advance transaction, then she can use all her coins when that transaction may be added to the blockchain. This again even she cannot login to the wallet file. We call this field can let us create like future contract and keep it until option as retire options as it will activate on inaccessible wallet the time is reached then broadcast. Some cryptocurrencies file and we will not use that wallet file again. such as Ethereum do not have this field, but we can use smart contract instead. The retire option can be elaborated in the flowchart shown 2. The process to redeem UTxOs with the private key, there in Figure 3. Before setting, the retire option needs two is an option to sign the whole transaction then the content important requirements. First, we must have a reserved in any field in signed transaction script cannot be changed address that maybe our own reserved address or our descendants’ address that will be a receiver of the retire option, in above scenario Alice may set Bob’s address as receiver of the retire option. Bob does not need to know Alice’s private key, but he must have his private key that can access to receiver address. By the way, after setting the retire option, Alice still can use her coins as normal because wallet application will update every time when it is logged out or quitted. Second, we must specify the period of no movement in which no transaction has ever been made, thus this circumstance will also trigger retire option to be effective. We use Bitcoin timestamp and block number as reference because it is on distributed network and is surely verified by consensus protocol (means very difficult to fake). For example, Alice set the period for 6 months which means the wallet application will check Bitcoin timestamp with wallet file’s last login if it is more than 6 months, the retire options will be effective and will transfer all UTxOs on last login to receiver address. The flowchart in Figure 3, illustrates the process into three cases. 102

2019 4th International Conference on Information Technology (InCIT2019) later. This option call Signature Hash Type or SIGHASH http://fortune.com/2017/11/25/lost-bitcoins/, [Accessed May flag in Bitcoin. 8, 2019]. [5] Crypto Research, “Factsheet: Full Analysis of BTC”, 2019, 3. The rest of mechanism to make this retire option work is [Online] Available: https://crypto-research.at/wp-content/ adding the process to create and handle the retire script. uploads/2019/02/250219_BTC.pdf, [Accessed May 8, 2019]. The retire script shall make in one script and update in [6] Joseph Young,” $190 Million in Crypto Gone Forever, How every login, logout, and any changes in UTxOs in the Canada’s Biggest Bitcoin Exchange Lost it All”, Feb 1,2019, wallet (every time after received or sent coins). The latest [Online] Available: https://www.ccn.com/190m-gone-how- retire script shall replace the prior one. Then it can be canada-biggest-bitcoin-exchange-lost-it, [Accessed May 8, effective after the presetting time past (6 months in prior 2019]. scenario) since latest access or last login to the wallet. The [7] “Explorer of Bitcoin Address : 1HQ3Go3ggs8pFnXuHVHRyt- other thing is to communicate with the user to inform the PCq5fGG8Hbhx” [Online] Available: https://www.block- current status of the option and its current configuration. chain.com/btc/address/1HQ3Go3ggs8pFnXuHVHRytPCq5fG G8Hbhx?filter=1, [Accessed May 8, 2019]. D. Advantage of retire option [8] Krzysztof Okupski, “Bitcoin Developer Reference Working From our example, we can summarize the advantage of Paper”, Jul 30, 2016, Available: https://github.com/minium/ Bitcoin-Spec/blob/master/Bitcoin.pdf this option as follow: - [9] Mike Hearn, “Anti DoS for tx replacement”, in bitcoin-dev mailing list, Apr 17, 2013, Available: https://lists.linuxfound- 1. We have a chance to get all coins back from inaccessible ation.org/pipermail/bitcoin-dev/2013-April/002417.html, wallet file. It will not lose forever. [Accessed May 8, 2019]. [10] S.Eskandari, D.Barrera, E.Stobert, J.Clark, “A First Look at the 2. If the owner of the wallet is disappeared or dead, the right Usability of Bitcoin Key Management”, Workshop on Usable descendant can receive the coins even they cannot open the Security, Jan. 2015 wallet file. The retire option is made in advanced then [11] Y.Lui et al., “An efficient method to enhance Bitcoin wallet other people cannot change the receiver address because it security”, 11th IEEE International Conference, Oct. 2017 needs to be signed by private key that no one can access. [12] E. Benli et al., “BioWallet: A Biometric Digital Wallet”, The retire option can be activated by anyone as long as he ICONS 2017 The 12th Int.l Conf. on Systems, Apr. 2017 or she cannot access the wallet within predetermined [13] N. Courtois, G. Song, R. Castelluc, “Speed Optimizations in period. Bitcoin Key Recovery Attacks”, Tatra Mountains Mathematical Pub., Dec., 2016 3. If a bad thing does not happen, we are still able to use the [14] N. Courtois, P. Emirdag, F. Valsorda,“Private Key Recovery wallet, spending and receiving the coins as normal. The Combination Attacks: On Extreme Fragility of Popular Bitcoin retire options will not be activated even the system time or Key Management, Wallet and Cold Storage Solutions in network time is changed by malicious action because it Presence of Poor RNG Events”, Oct. 16, 2014 uses the time calculated from block header which well [15] W.J. Buchanan, “Guessable Private Keys: How To Make A verified. Million in Cryptocurrency”, preprinted version, Apr., 2019, Avail.: https://www.researchgate.net/publication/332683948 V. CONCLUSION AND FUTURE WORK [16] M.Doran, “A Forensic Look at Bitcoin Cryptocurrency”, The The good practice to use the wallet in cryptocurrency is to Sans Institute, Oct 21, 2015 encrypt and protect the private key with sophisticated [17] Chris Cohen, “Forensics and Bitcoin”, [Online] Available: authentication. However, the more secure method is https://articles.forensicfocus.com/2015/01/16/forensics- implemented, the more chance that the owner is unable to bitcoin/ , [Accessed May 8, 2019]. access and lost all related coins forever. Today, the amount of coins that have not circulated in the market is a big deal and increases every day. In this paper, we propose a solution that uses a normal transaction and imposes the advanced sign-in technique and uses block header compared with last login time to determine the triggered time to recover all coins from an inaccessible wallet to a preconfigured address. Considering the future work, we will consider other causes that get cryptocurrency lost and try to recover it to mitigate the unfavored repercussion from unintentional lost by combine with other techniques. REFERENCES [1] Satoshi Nakamoto, “Bitcoin: A peer-to-peer electronic cash system”, 2009, Available: https://bitcoin.org/bitcoin.pdf, [Accessed on May 8, 2019]. [2] bitcoin.org, “Block Chain Guide”, [Online] Available: https://- bitcoin.org/en/blockchain-guide, [Accessed May 8, 2019]. [3] P. Gallagher and C. Kerry, “Federal Information Processing Standards Publication : Digital Signature Standard (DSS)”, June, 2019, FIPS PUB 186-4, Available: https://nvlpubs.- nist.gov/nistpubs/FIPS/NIST.FIPS.186-4.pdf [4] J.J.Roberts and N.Rapp, “Exclusive: Nearly 4 Million Bitcoins Lost Forever, New Study Says”, Nov 2017, [Online] Available: 103

2019 4th International Conference on Information Technology (InCIT2019) Implementation of Voice User Interfaces to Enhance Users’ Activities on Moodle Toshihiro Kita Chikako Nagaoka Naoshi Hiraoka Research Center for Instructional Systems Research Center for Instructional Systems Research Center for Instructional Systems Kumamoto University Kumamoto University Kumamoto University Kumamoto, Japan Kumamoto, Japan Kumamoto, Japan [email protected] [email protected] [email protected] Martin Dougiamas Moodle HQ Perth, Australia [email protected] Abstract—Voice user interfaces (VUIs) are effective and Additionally, even if an LMS itself is not equipped with a intuitive to improve user interaction with various types of VUI, searching the help documentation of an LMS by voice information-technology-based systems. VUIs are increasingly be- commands will be useful when operating the LMS for creating coming suitable for use in various practical applications, e.g., teaching materials, leading to improved operability of the voice-operated smartphones or smart speakers such as Amazon LMS. Echo or Google Home. If VUIs, which enable hands-free and intuitive use, become available to a learning management system II. DEVELOPMENT OF APPS FOR SMART SPEAKERS (LMS) such as Moodle, learning activities on the LMS can be made easier and help motivate the users. In addition, if it is In this research, two popular services for smart speakers possible to search LMS help documentation by speech via a are utilized for development: Amazon and Google. Tools and VUI and listen to the search results, the work efficiency of LMS documentation for building the VUIs are found for “Amazon content creators may be improved. In this research, we have Alexa” [4] and for “Actions on Google” [5], which contain developed a voice app for learners to attempt quizzes on Moodle several examples and templates for developers to get started sites, and a voice app for users to search MoodleDocs (Moodle quickly with VUI development, including integration of the online documents). VUIs with external systems. Keywords—VUI, LMS, VLE, MoodleDocs, Smart Speakers, With Amazon Alexa, a cloud-based voice service, develop- Voice Command, Hands Free Speaker ers can build and publish Alexa Skills1 that can be used with Alexa devices such as Amazon Echo smart speakers. I. INTRODUCTION Actions on Google service, in contrast, are for building Voice user interfaces (VUIs) allow people to use voice VUI applications for Google Assistant devices such as Google input to control computers and devices. VUIs have advan- Home smart speakers. tages over typing, such as faster input speed, hands-free use, and intuitive use [1]. Thanks to recent improvements in the In this paper, Alexa Skills that can be used with Alexa accuracy of speech recognition and synthesis through machine devices such as Amazon Echo smart speakers, and applications learning approaches, VUIs are rapidly becoming suitable for for Google Assistant devices such as Google Home smart various practical purposes in devices such as voice-operated speakers, are collectively referred to as ”voice apps”. smartphones and smart speakers like Amazon Echo or Google Home. Google’s Dialogflow [6] provides a voice app develop- ment environment based on “Actions on Google” service. Moodle [2] is a popular Learning Management System An ”intent” is a unit which defines how to respond when (LMS) used around the world. Moodle offers various fea- a user speaks. To define the response, a response string tures to perform learning activities, such as Quizzes and can be directly specified on Dialogflow. Alternatively, the Assignments. Making these features available through a VUI user’s utterance data can be sent to an external service using could enhance the functionality of Moodle. Of course, VUIs ”webhook” [7] (Fig.1) for processing in order to obtain a are essential for blind learners. Apps using VUIs have been response. developed for several LMSs such as Moodle, which show the submission due dates of assignments or the score of users The development environment called Alexa Developer Con- in courses. However, there are no apps that can be used to sole [8] provided by Amazon Alexa has a structure similar perform the learning activity itself via a VUI with an LMS, to Dialogflow, but responses at an intent are not specified as far as the authors have examined [3]. 1VUI applications for Amazon products are called Skills. 104

2019 4th International Conference on Information Technology (InCIT2019) Fig. 1. Smart speakers connected to a Moodle site (Both Google Home and Amazon Echo are supported) directly on the Alexa Developer Console. An external service TABLE I (endpoint) must always be called to transmit a user’s utterance NAMES OF MOODLE WEB SERVICES FUNCTIONS USED BY THE VOICE APP data and to obtain the result that becomes the response at the intent (Fig. 1). mod quiz get quizzes by courses mod quiz start attempt III. ATTEMPTING QUIZZES THROUGH A VUI mod quiz get attempt summary mod quiz save attempt If oral quizzes on an LMS were to be available using mod quiz process attempt a smart speaker placed at home, the learner could attempt mod quiz get attempt review the quiz questions more easily than with personal computers mod quiz get user attempts and smartphones, possibly triggering the user’s willingness to learn. For example, before enrolling in a course or in the early services. These functions are provided by a vanilla version of stages of learning in the course, if a ”trial quiz” (a quiz that Moodle, thus users do not need to install any special Moodle can evaluate the skills to be learned and attract the attention plugins to use the voice app. One of the difficulties we faced in of learners) is provided to learners through smart speakers, the voice app development was that the data returned by each learners could quickly grasp what they will learn and what they web service function are intended for displaying purposes, and will be able to do when they complete the course, stimulating were not directly usable as the voice app response. To retrieve their desire to learn. question texts, or question answers, the question data in HTML format, whose structure was not explicitly documented, needed The voice app developed in this research, which allows to be analyzed. students to take quizzes2 on Moodle using a VUI, is publicly available as a demonstration command [9]. Any users of Fig. 2 demonstrates the behavior of a Skill under devel- Google Home or Google Assistant app can try the voice app by opment on Alexa Developer Console. The learner (right) is saying ”OK Google, talk to Moodle quiz.” For Amazon Echo, shown taking a quiz on Moodle. The score is mentioned and users can try the voice app by saying ”Alexa, open Moodle feedback is provided based on the learner’s answers. quiz” after enabling the ”Moodle quiz” Skill. Currently the supported languages are English and Japanese. The publicly available version of our voice app works with The user’s language can be set in the smart speaker settings a fixed user logged in to the demo Moodle course, but the and the Moodle preferences settings. If a quiz needs to support development version uses a feature to link accounts [10]–[12], multiple languages based on the users’ language settings, quiz enabling a connection to any Moodle site and the ability to questions must be prepared using the format of the multi- take quizzes on it by storing the access token for the Moodle language content filter [14]. site. IV. SEARCHING HELP DOCUMENTS BY VOICE COMMANDS The connection between the voice app and Moodle is made possible using the Moodle web services API [13] as shown in From the viewpoint of online teaching material creators, Fig. 1. Users’ history of quiz attempts is stored on the Moodle LMSs like Moodle have many features. When creating teach- site as usual and can be reviewed by each user. To allow ing materials, it is often necessary to search and browse learners to attempt quizes, the voice app uses the functions documentation about the operation methods and functions, shown in Table I to communicate with Moodle via REST web which is bothersome and can hinder the utilization of LMSs. 2Currently multiple choice questions are supported. In this research, we developed a voice app with which users can search the help documentation of Moodle functions and 105

2019 4th International Conference on Information Technology (InCIT2019) Fig. 3. Alexa Skill development on Developer Console Fig. 2. Example of running Alexa skill ”Moodle quiz” by Moodle HQ. It covers all the features of Moodle and is used by many as a basic manual to refer to when using Moodle. can listen to the result using only voice commands. The voice app is designed to search Moodle help documentation created Although there is already a Japanese version of Moodle- by translating English MoodleDocs [15] into Japanese3 using Docs, like many other versions in other languages than En- the Google Cloud Translation API (mistakes and deficiencies glish, the Japanse version is not as rich as the English version were corrected manually afterwards). MoodleDocs is one of and does not cover, in particular, the new features of the latest the most popular Moodle online manuals, which is maintained version of Moodle. For this reason, it was necessary to trans- late all the pages of English version of MoodleDocs using the 3Japanese users are the target users in this research. Other languages can machine translation service. The original English MoodleDocs also be supported by extending the system in a straightforward manner. is built on the MediaWiki system, but the archived files [16] retrieved from the system were used for this translation. Voice apps are typically designed assuming that possible word candidates (personal names, date, city names, etc.) are used in a user’s utterance to obtain a part of the user’s utterance content as a parameter4. However, this voice app needed search terms for searching a large number of documents, meaning that any words or phrases must be obtainable. Therefore, for the Alexa Skill, AMAZON.SearchQuery was used as the slot type (Fig. 3), and for the intent on Dialogflow, sys.any entity was used (Fig. 4). Responses must be limited to the amount of information that is necessary and sufficient in order to accurately convey the search results by voice. In addition, a function capable of displaying images and text information in a visual format can also be implemented. From this standpoint, additional 4It is also called a slot. 106

2019 4th International Conference on Information Technology (InCIT2019) Our future plan includes extending the voice app to be able to notify learners and teachers of important activities (such as due or overdue assignments) to support their learning and teaching on Moodle. One possible method is to add web service API support to the Timeline block [17] and a notification feature to the voice app. In addition, more intelligent and proactive support for users can be implemented by importing insight results of Moodle Analytics engine [18]. The authors have joined another research project on im- plementing a safety education system based on Moodle that is aimed at preventing serious accidents. The VUIs we are developing are expected to play an important role in achieving these aims. Fig. 4. Agent (voice app) development on Dialogflow REFERENCES improvements could be made to produce an even more user- [1] Pearl, C. : Designing Voice User Interfaces. O’Reilly Media. (2016). friendly voice app. [2] Moodle.org: Moodle - Open-source learning platform, https://moodle.org/ (2019). [3] Kita,T., Nagaoka,C., Hiraoka,N., Suzuki,K., and Dougiamas, M., : A Discussion on Effective Implementation and Prototyping of Voice User Interfaces for Learning Activities on Moodle, CSEDU 2018, https://doi.org/10.5220/0006782603980404 (2018) [4] Amazon.com: Amazon Alexa, https://developer.amazon.com/alexa (2019). [5] Google Developers: Actions on Google, https://developers.google.com/actions/ (2019). [6] Dialogflow, https://dialogflow.com/docs (2019). [7] Create fulfillment using webhook, https://dialogflow.com/docs/tutorial-build-an-agent/ create-fulfillment-using-webhook (2019). [8] Alexa Developer Console, https://developer.amazon.com/alexa/console/ask (2019). [9] Kita : Moodle Quiz (Actions on Google, Alexa Skill) https://tkita.net/ai/moodlequizvui.html (2019) [10] Understand Account Linking, https://developer.amazon.com/docs/account-linking/ understand-account-linking.html (2019) [11] Account linking, https://developers.google.com/actions/identity/ (2019). [12] An Alexa Skill for the MMBBS, https://github.com/jtuttas/alexa (2019). [13] Web services API - MoodleDocs, https://docs.moodle.org/dev/Web services API (2019). [14] Multi-language content filter - MoodleDocs, https://docs.moodle.org/37/en/Multi-language content filter (2019) [15] MoodleDocs, https://docs.moodle.org/ (2019). [16] Wimski: Download offline Moodle Docs packages, https://wimski.org/docs/ (2019). [17] Timeline block - MoodleDocs, https://docs.moodle.org/37/en/Timeline block (2019) [18] Analytics - MoodleDocs, https://docs.moodle.org/37/en/Analytics (2019) V. CONCLUSIONS In this research, we developed functions to take quizzes on a Moodle site and to search LMS manual documentation via a VUI in order to enhance user activities. Although there are currently few examples of LMSs equipped with VUIs, it is considered worthwhile to prac- tically explore the use value of VUIs as a new channel that strengthens the interaction between LMSs and the users. Implementing a VUI as a user interface requires consideration of how to capitalize on the advantages while being aware of the limitations and drawbacks of a VUI. 107

2019 4th International Conference on Information Technology (InCIT2019) Development and Evaluation of Smartphone Learning Material for Blended Language Learning Yuichi Ohkawa Masaaki Kodama Yuta Konno Graduate School of Education Graduate School of Educational Informatics Graduate School of Educational Informatics Tohoku University Sendai, Japan Tohoku University Tohoku University [email protected] Sendai, Japan Sendai, Japan [email protected] [email protected] Xiumin Zhao Takashi Mitsuishi Institute for Excellence in Higher Education Center for Information Technology in Education Tohoku University Tohoku University Sendai, Japan Sendai, Japan [email protected] [email protected] Abstract—This study evaluated a newly developed smart- progress and that being unable to review easily according to phone-based review material application (in blended learning) class progress caused them anxiety. The inability to conduct that enables learners to learn continuously and intermittently continuous but intermittent review indicated that learners can by providing visualization of learning status in the smartphone learn in a short time only by dividing learning material application. In beginning Chinese language classes, our blended into short questions. Therefore, we have developed the learning practice consists of both face-to-face classroom and smartphone learning material application KoToToMo Plus, home assignments in e-learning. For blended learning, we implemented as blended learning, with a user interface (UI) provided a prototype of smartphone-based learning material in that enables students to review according to class progress, 2017 but found some issues that might have prevented learn- to select learning content according to their recent learning ing’s continuance. Therefore, we developed the smartphone- status, and to resume interrupted learning easily through their based learning material application KoToToMo Plus. This visualized learning status [3]. We aimed to make it possible version of the application enables students to learn easily in to conduct continuous but intermittent review based on micro a little spare time by having achieved a user interface that learning. Since April 2018, we have practiced blended learn- provides visualization of learning status and makes it easy to ing with KoToToMo Plus and compared its evaluation with resume interrupted learning and to select the next content. that of the previous KoToToMo application from 2017. This Since April 2018, we have practiced blended learning using and paper outlines the design and implementation of KoToToMo evaluating this application in actual classes. Results showed Plus and reports evaluation results obtained through practice. increased numbers of learning times and decreased feelings about learning as burdensome. II. RELATED WORK Keywords—Blended learning, smartphone, learning material, A. Smartphone-Based Learning Materials micro learning, language learning With the spread of smartphones in recent years, many I. INTRODUCTION applications for language learning have been proposed. Beginning university classes in foreign languages limited Rosell-Aguilar et al. [4] have developed and evaluated a class time, so to acquire language skills, learners need to re- smartphone application for learning Chinese by reading and view after class. Therefore, we have proposed and practiced writing Chinese characters, responding with words’ mean- three steps of blended learning that combine face-to-face ings, and distinguishing pronunciations. Kim et al. [5] have classroom work with e-learning review [1]. Additionally, developed and provided a mobile learning application for with the recent spread of smartphones, we have developed Korean certification, in order to clarify that mobile learning is the smartphone learning material application KoToToMo that effective for solo study outside class and for preparing for a learners can use intermittently outside the classroom. The certification examination. The commercial service Duolingo application is based on micro learning and has been used [6] offers smartphone language learning application for mul- for actual classes’ blended learning since 2017. As a result tiple languages and aims to maintain learning motivation by of practice with the application, we have found increased showing learning history that incorporates gamification. learning time compared with e-learning review accessed through a web browser [2]. These systems target self-contained study completed only on a smartphone, but using mobile learning material in From a questionnaire conducted at the end of the first a face-to-face class is inappropriate. Even in class, these semester, however, we found some learner dissatisfaction, specifically that learners could not check their learning 108

2019 4th International Conference on Information Technology (InCIT2019) systems were used in a form not directly linked to the content (a) Home Screen (b) Selection of Questions of the face-to-face class and could not be used for preparation or review of content learned in class. Fig. 1. Example of a KoToToMo Plus screen. (iOS version) B. Learning Material for Blended Learning 2) Visualization of Learning Status for Each Question: We decided that correct or incorrect answers, degree of Baldauf et al. [7] developed a prototype system of smart- memory, and amount of learning should be visualized as phone learning materials and practiced with it in a class a learning status for each question. Here, “correct” or “in- of 13- and 14-year-olds. Too, they examined the learning correct” indicates the learner’s most recent answer. Degree materials’ necessary conditions for use in blended learning. of memory means the high recall value immediately after learning and the decrease in inverse proportion to elapsed Results indicated that content for blended learning roughly time; if learning occurs again, degree of memory rises, and requires relation to the face-to-face class and quick access its rate of decrease moderates. Amount of learning means to any learning material application, so these are proposed. the duration required to answer or the number of times a However, only limited content, such as English words for question is answered. So that learners can judge intuitively, secondary education, are learning targets. Too, the UI design colors and icons indicate correctness or incorrectness. The and learning status visualization have not been included in correct answer is blue with a check mark, the incorrect a balanced way for the four language skills, for example, answer is red with a cross icon, and those questions as yet speech and grammar. Therefore, we believed that it could unanswered are gray. To remind learners to work again with not be appropriately used for beginning second language a question when degree of memory has decreased and so they learning in the university targeted in our study. Instead, can check simultaneously each question’s (in)correctness, with a smartphone learning material application, our study degree of memory is indicated in shades of blue (correct) aims to enable review of content in face-to-face classes. of red (incorrect). The color is dark immediately after the Therefore, for students to acquire language skills, we thought answer and lightens according to degree of memory. Amount it necessary for them to learn in a balanced and continuous of learning is represented by the number of blocks, and way through different types of questions. In addition, we learning time by the length of a block. thought it necessary for students to be able to check their actual learning status in order to advance their learning According to color, learners can grasp at a glance ques- concurrent with face-to-face classroom progress. tions not yet answered or questions answered correctly or incorrectly. Since learners can know how much they remem- III. KOTOTOMO PLUS ber by checking each question’s degree of memory, they can decide which questions to repeat. Being able to check their To make intermittent learning based on micro learning amount of learning provides a sense of accomplishment and sustainable for review in blended learning, students must satisfaction in learning, thus motivating further learning. be able to select content according to their recent progress. Learners must also be able to progress despite repeated inter- 3) Visualization for Each Type of Material: Language ruptions and restarts. Therefore, we designed the smartphone learning material that learners should review consists of learning material application KoToToMo Plus to achieve multiple types of questions. Too, learning evenly through such learning activities [3]. all types is desirable. Learners need to grasp their progress on each type of question and should concentrate on the A. Design of KoToToMo Plus The design of KoToToMo Plus allows students simultane- ously to check their learning status and select questions on the same screen, instead of switching to another screen. This makes it possible to select questions according to learners’ current status, and they can easily determine the learning content to be tackled next and then begin immediately. To achieve this, we visualized and presented their current learning status appropriately. The design’s details about visualized learning status are described below. 1) Review According to Class Progress: To support learn- ers who cannot recall which chapter should be reviewed, the number of classmates who worked intensively on a certain chapter during a certain week are presented. As a result, students can find the chapter other learners are working on and roughly predict class progress. Additionally, the ability to see their classmates’ learning activities might sustain learners’ motivation. 109

2019 4th International Conference on Information Technology (InCIT2019) TABLE I USAGE STATISTICS ACQUIRED FROM LOG DATA OF KOTOTOMO (2017) AND KOTOTOMO PLUS (2018) Common materials All materials Items 1st semester 2nd semester 1st semester 2nd semester Average number of times of learning per week 2017 2018 2017 2018 2017 2018 2017 2018 Average of time of learning at once (minutes) Average of time of learning per week (minutes) 0.85 1.13∗∗ 0.80 1.05∗∗ 0.86 1.24∗∗ 0.81 1.13∗∗ Average number of correct answers per question 4.70 5.75 ∗ 6.56 5.81 ∗ 4.23 3.73 5.67 4.34 ∗∗ 4.03 6.23 ∗∗ 5.35 6.21 ∗ 3.61 3.82 4.58 4.33 0.82 0.95 1.03 1.00 0.79 0.92 1.00 0.99 ∗∗p < .01, ∗p < .05 type not yet adequately mastered. Therefore, our design IV. EVALUATION THROUGH PRACTICE shows accumulated learning status according to each type of question. Again, learners can grasp at a glance questions A. Outline of Practice and Evaluation not yet worked on already answered correctly or incorrectly. Since April 2018, we have provided the KoToToMo Plus Progress for each type of learning can be visualized application to beginner Chinese classes at our university, via a progress bar that accumulates color according to specifically to seven classes (in which one co-author teaches) (in)correctness and degree of memory for each question. By with 218 learners. Class hours run 90 minutes once a week pressing the open/close button on the screen of a chapter, and 45 hours a year. In these classes, blended learning has learners see progress in all the chapter’s learning types, along been introduced, and after face-to-face classroom meetings, with progress for each of chapters. As a result, learners a review using this application and Quizlet has been assigned can easily grasp which types have been sufficiently or as homework. insufficiently mastered in each chapter. This application logs learning activity and sends it to a 4) Visualization for Each Chapter: The class and text- server. By examining each log, we know who, when, which book are composed of a plurality of chapters, and, in the question, and how long. In addition, the previous KoToToMo application, learners need to complete review of the chapter application, provided in the 2017 classes, also recorded corresponding to class progress. For that purpose, learners learning logs. Furthermore, we administered a questionnaire need to grasp their progress in each chapter. Therefore, accu- to learners at the end of the first semester both in 2017 and mulated progress in all chapter’s learning types is presented 2018. In this questionnaire, we asked about blended learning, in the visualization of each chapter’s learning status. Thus, the textbook, and the application used in blended learning. learners can grasp their progress in each chapter and easily In addition to this questionnaire, we administered another discover a chapter that needs more attention. Learners are questionnaire only in 2018 on the visualization functions of expected to proceed so that all progress bars turn blue as the KoToToMo Plus. class continues. In the following, we describe the application’s evaluation 5) Resuming Interrupted Learning: To resume easily their based on log data results and the two questionnaires men- learning from the last session, learners see first the question tioned above. they worked on last, and the chapter that contains the ques- tion, presented with the orange label “previous learning.” In B. Evaluation Based on Log Data addition, when the application opens, the screen automati- cally scrolls to the chapter position where this label appears. 1) Evaluation Procedure: We analyzed learning log data When learners select that chapter, the screen automatically to clarify the application’s basic usage status and to evaluate scrolls to the question on which the label appears. As a result, the application according to changes in levels of learning’s learners can confirm the question interrupted during their last continuance, time, and amount. We acquired averages for the session and easily resume their interrupted learning. number of learning times per week, the duration of learning at once, the duration of learning per week, and the number of B. Implementation of KoToToMo Plus correct answers per question. We also acquired rate changes of learners using each application each week. Based on the design previously described, we imple- mented the new smartphone learning material application Here, one duration in the average time of learning is a KoToToMo Plus for iOS and Android. To prevent differences value obtained by treating as a different duration if there is an in ease of use caused by the differences in operating systems interval of more than 15 minutes between one answer and the (OSs), we developed a native application of each OS while next when learners answer multiple questions continuously. referring to their design guidelines. Fig. 1 shows examples Learning time indicates the interval between the learning of the iOS version’s execution screen for KoToToMo Plus. screen’s display and answer’s completion; time for selecting We have confirmed that the implemented application oper- a question from the menu is not included. ates properly according to our design. Because each blended learning class is conducted on a different weekday, the seven classes’ progress varies, depending on holidays. Therefore, we selected tabulation periods based on the number of class meetings. Since we 110

2019 4th International Conference on Information Technology (InCIT2019) 100 Summer vacation Second semester First semester Learners’ rate in using the application each week (%) 80 60 40 20 2018 (KoToToMo Plus) 2017 (KoToToMo) 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 Number of weeks from the first Monday in April Fig. 2. Changes in rate of students who learned by using applications one or more times a week, acquired from log data. TABLE II RESULTS OF QUESTIONNAIRE ON SMARTPHONE APPLICATION FOR REVIEWING ANSWERED AT END OF 1ST SEMESTER (5-POINT RATING SCALE EXCEPTING Q10–12; 5: AGREE–1: DISAGREE) Index Questions 2017 (N = 282) 2018 (N = 202) (KoToToMo) (KoToToMo Plus) Q6 Q7 Layout of screen UI was comprehensible. 4.00 4.39∗∗ Q8 Operation of the application was easy. 3.89 4.52∗∗ Q9 By using the application, you could learn for reviewing continuously. 3.59 3.94∗∗ Q10(1) You have worked on reviewing with enthusiasm. 3.56 3.86∗∗ Q10(2) Q11a How often did you learn for reviewing a week? (the number of times) 1.91 1.80 Q12a How long did you spend for reviewing once? (minutes) 23.09 18.81 ∗∗ Q13 Q14 How often did you read aloud when training of reading? 3.21 3.30 Q17 How often did you use the functions for recording and playing voice? 2.45 2.92∗∗ Q18 You have not hesitated to conduct “exercise” training for judgment of pronounce. 3.66 3.92∗ You felt fun when you conduct “grammar training” like a game. 3.87 4.21∗∗ You felt that to learn for reviewing using this application was good. 3.94 4.39∗∗ You want to continue to use learning material such as this application into the future. 3.87 4.36∗∗ ∗∗p < .01, ∗p < .05, a. 4-point rating scale (4: Almost every time–1: Not at all) had recorded learning logs only since June 7 in 2017, only for common materials. On the other hand, no significant classes conducted after that were included in the tabulation. difference was found in the number of times answers were Additionally, since learners’ reviews might differ (especially, correct per question in all cases. increase) from ordinary reviews shortly before semester examinations, we decided to exclude the week prior to Fig. 2 shows learners’ rate changes in using the application semester examinations from tabulation. Thus, the tabulation each week in 2017 and 2018. The horizontal axis shows the targets for the first semester were after the 10th class until number of weeks from the first Monday in April. Since, as the 14th class and, for the second semester, after the 16th mentioned above, log recording started on June 8 in 2017, class until the 25th class. Because rate changes of learners there are no plots before June 7 (Week 9) in the 2017 graph. each week did not have an established tabulation period, we analyzed all periods for which we could acquire logs. Fig. 2 shows that, compared to 2017, 2018 had a higher rate of learners who worked on learning at least once a week. KoToToMo and KoToToMo Plus differ in some questions We also found that the graph’s variation each week in 2017 and in total numbers of questions. Therefore, when we was greater than in 2018. compared tallied log results for 2017 and 2018, we tallied separately and compared logs for learning materials common C. Evaluation Based on Questionnaire at End of Semester to both applications (common materials) and logs for all learning materials (all materials). 1) Evaluation Procedure: To examine students’ impres- sions of (blended) learning and motivations to learn with the 2) Results: TABLE I shows the tabulation results. The two applications, we administered a questionnaire at the end average number of learning times per week in the first of the first semester in 2017 and 2018. and second semesters for both common materials and all materials significantly increased from 2017 to 2018. The In 2018, the questionnaire had 22 questions, of which average learning time per week significantly increased for all five are on the textbook, 13 on the smartphone application materials, while no significant difference could be confirmed for reviewing, and four on the short quizzes. Of these, we compared only questions related to the smartphone applica- tion and common to 2017 and 2018. Q 10 is a description 111

2019 4th International Conference on Information Technology (InCIT2019) TABLE III LIST OF QUESTIONS IN THE QUESTIONNAIRE ON VISUALIZATION FUNCTIONS Index Description of Question Q2 Progress bars of each chapter were felt to be helpful to grasp progress status of your learning of each chapter. Q3 Progress bars of each chapter were felt to be helpful to select question to be tackled next in the application. Q4 Progress bars of type-by-type of questions were felt to be helpful to grasp progress status of your learning of each type. Q5 Progress bars of type-by-type of questions were felt to be helpful to select question to be tackled next in the application. Q6 Signs for right or wrong of each question were felt to be helpful to select question to be tackled next in the application. Q7 Displays of measure of memory were felt to be helpful to select question to be tackled next in the application. Q8 Displays of amount of learning were felt to be helpful to select question to be tackled next in the application. Q9 It made you feel satisfaction or achievement that status of right or wrong, measure of memory and amount of learning were visible. Q10 Displays of number of recent learners were felt to be helpful to know which chapter you should learn in each week. Q11 Displays of number of recent learners were felt to be a trigger to begin reviewing. Q12 Displays of number of recent learners were felt to be helpful to select question to be tackled next in the application. Q13 Display of the previous learning was felt to be helpful to resume learning from the location learned last time. Q14 Display of the previous learning was felt to be helpful to select question to be tackled next in the application. Agree Agree a little Neither agree nor disagree Disagree a little Disagree Not use Dummy 1 44 47 63 5 10 Q2 105 50 37 9 Q3 Q4 74 22 22 5 11 Q5 0 Q6 Q7 98 13 35 Q8 1 35 Q9 73 Q10 12 1 Q11 Q12 119 49 10 7 7 Q13 Q14 71 48 29 1 33 49 11 0% 85 89 25 32 82 13 86 19 6 7 45 85 36 50 20 6 21 55 66 70% 23 9 21 57 43 35 26 12 20 40% 1 24 99 25 10 4 78 58 1 60% 22 4 10% 20% 30% 50% 80% 90% 100% Fig. 3. Results of questionnaire on visualization functions. (N = 193) question. Qs 11 and 12 are 4-point rating scale questions, and 2) Results: From October 22 to October 30 in 2018, we the other questions are 5-point rating scale questions. four- administered the questionnaire on visualization functions to point rating included: 4. Almost every time; 3. Sometimes; each class during class time. We received responses from 2. Not much; and 1. Not at all. Five-point rating included: 195 learners (except two who never used the application), 5. Agree; 4. Agree a little; 3. Neither; 2. Disagree a little; whose results are shown in Fig. 3. 1. Disagree. Results revealed that 50% to 85% of learners answered 2) Results: TABLE II shows the questionnaire’s results. that they thought all the visualization functions were helpful Of learners in the 2017 classes, 286 of 294 provided answers; in knowing their individual learning status and deciding in 2018, 202 of 218 provided answers. Results revealed which content to learn next. significant differences in mean values for all questions except Q 10(1) and 11. Of these, the average value for all questions E. Discussion except Q 10(2) increased favorably. 1) Continuous Learning: From learners’ rate changes per D. Evaluation Based on Questionnaire on Visualization week, shown in Fig. 2, we confirmed that about 45% to 65% of learners each week in 2017 and 55% to 75% in 2018 1) Evaluation Procedure: To investigate whether the worked on learning during the class term. Classes targeted functions developed for visualizing learning status and for for analysis teach one chapter every 2 weeks. The 2017 graph assisting learners to resume interrupted learning were helpful shows that the fluctuation rate repeats every two weeks. We for review activities, we administered a questionnaire on believe that a certain number of learners performed a review visualization functions. only once after they finished a chapter. On the other hand, the 2018 graph shows use of KoToToMo Plus more continuously TABLE III displays the list of 13 questions, and each per week because the fluctuation is smaller than in 2017. was rated according to 6 choices: a 5-point Likert scale Additionally, according to the usage statistics of KoToToMo and the additional “I did not use or check this visualization and KoToToMo Plus in TABLE I, we believe that the number function.” of learners weekly increased, since the average number of 112

2019 4th International Conference on Information Technology (InCIT2019) learning sessions per week also increased significantly from REFERENCES 2017 to 2018, and average values in 2018 exceeded 1.0. [1] X. Zhao, N. Tomita, F. Konno, J. Zhu, T. Inagaki, Y. Ohkawa, and From the questionnaire results in TABLE II, we also T. Mistuishi, “Development and Practice of Design Guidelines of e- confirmed that results of subjective assessments suggested Learning Materials in Blended Learning for Chinese as a Second that working on learning with enthusiasm and learning Foreign Language,” Trans. of Japanese Soc. for Information and continuously improved significantly. Therefore, we consider Systems in Education, Vol.31, No.1, pp. 132–146, 2014. (in Japanese) that KoToToMo Plus has achieved the continuous review at which we aimed in this study. [2] X. Zhao, N. Tomita, F. Konno, Y. Ohkawa, and T. Mitsuishi, “De- velopment and Practice of Review Material KoToToMo for Use on 2) Motivation: As TABLE I shows, the average learning Smartphones in Blended Learning by Beginning Learners of Chinese time per week for all materials significantly increased in in University,” Trans. of Japanese Soc. for Information and Systems from 2017 to 2018. On the other hand, no significant in Education, Vol.36, No.2, pp. 131–142, 2019. (in Japanese) difference appeared in the average learning time per week for common materials. The cause of increased learning time [3] Y. Ohkawa, M. Kodama, Y. Konno, X. Zhao, and T. Mitsuishi, “A for all materials might be that KoToToMo Plus has additional Study on UI Design of Smartphone App for Continuous Blended types of questions compared with KoToToMo. However, Language Learning,” Proceedings of 5th International Conference on from the end-of-semester questionnaire results in TABLE II, Business and Industrial Research, pp. 584–589, 2018. learners confirmed that in 2018, they took significantly less time to learn than in 2017. [4] F. Rosell-Aguilar, and K. Qian, “Design and user evaluation of a mobile application to teach Chinese characters,” The JALT CALL We surmise that learners need not spend time selecting Journal, Vol.11, No.1, pp. 19–40, 2015. questions to learn and that they feel less burdened by learning than their 2017 predecessors since they became able to [5] E. Kim, and H. Kim, “Development and Evaluate of the Self-study confirm easily through the improved UI in KoToToMo Plus Mobile Learning Materials for a Korean Certification Examination,” the content they should learn. Results in TABLE II also Trans. of Japanese Soc. for Information and Systems in Education, show that since responses to ease of use and to future use Vol.30, No.4, pp. 248-253, 2013. (in Japanese) of the application were improved significantly, we believe that earlier learners’ avoidance of review was caused by the [6] R. Vesselinov, and J. Grego, “Duolingo Effectiveness Study,” Re- learning materials’ design and that problems in KoToToMo trieved May 8, 2019 from https://static.duolingo.com/s3/DuolingoRep have been remedied in KoToToMo Plus. ort Final.pdf, 2012. V. CONCLUSIONS [7] M. Baldauf, A. Brandner, and C. Wimmer, “Mobile and Gamified Blended Learning for Language Teaching – Studying Requirements To make it possible for learners to review blended learn- and Acceptance by Students, Parents and Teachers in the Wild,” ing, based on micro learning, intermittently and continu- Proc. of 16th International Conference on Mobile and Ubiquitous ously, we designed a UI through which learners can easily Multimedia, pp. 13–24, 2017. resume interrupted sessions by checking their visualized learning status, and we developed the smartphone learning material application KoToToMo Plus. We have conducted a practice of blended learning using the developed application in beginners’ Chinese language classes since April 2018 and evaluated the application. Compared to the earlier KoTo- ToMo, used in 2017, KoToToMo Plus promotes intermittent learning, sustains higher frequency of review, improves sat- isfaction with review activity, and motivates further learning, all while reducing the burden of learning. We also confirmed that learners found the new visualization functions generally useful. However, we have not been able to confirm increased learning time and amounts. Therefore, we believe it nec- essary to investigate further a methodology that increases learning durations in addition to promoting intermittent learning. ACKNOWLEDGMENT Part of this work was supported by JSPS KAKENHI Grant Numbers 15K01012, 15K02709 and 17K01070. 113

2019 4th International Conference on Information Technology (InCIT2019) E-Commerce for the Preservation of Traditional Thai Craftsmanship Prisana Mutchima Nattha Phiwma Vipavee Valeepitakdej Faculty of Humanities and Social Sciences Faculty of Science and Technology Faculty of Management Science Suan Dusit University Suan Dusit University Suan Dusit University Bangkok, Thailand Bangkok, Thailand Bangkok, Thailand prisana [email protected] nattha [email protected] vipavee [email protected] Abstract—Traditional Thai craftsmanship is in danger of [4]. Therefore, e-commerce and internet technology has a going extinct owing to the pressures of modern development. significant social impact. E-commerce applications support Therefore, the objectives of this study were to develop the the interaction between various parties participating in a e-commerce system of traditional Thai craftsmanship. The commerce transaction as well as the management of the instruments research used for data gathering were in-depth data involved in the process via the network or the internet. interviews and questionnaire. The traditional SDLC was used Which electronic commerce is being accepted and used. for system development. The main results are summarized as The process of buying products on the web is normal [5]. follows: (1) The content of traditional Thai craftsmanship And is a way to conduct business on the internet that has that has been registered as a national cultural heritage in the potential to change the pattern of traditional economic 2009-2015 which can be developed into a creative economy activities [6]. It has internationalized, and buying products consists of registration year, history, production, production online across national borders has become straightforward area, special characteristics or identity, value and utility; (2) and convenient for consumers, providing new business The developed e-commerce system is divided the work into 4 opportunities for both domestic and international online groups: non-member users, buyers, sellers and administrators stores [7]. In India, e-commerce is an IT tool that can to support comprehensive work. This system is considered do the economic wonders of India by increasing cyber as a way to help preserve the traditional craftsmanship of entrepreneurs [8]. Thailand to be able to remain in Thai society. In addition, the linkages of production, communication Keywords— e-commerce, traditional Thai craftsmanship, and technology are related to economic and cultural ac- Thai craft preservation, handicraft products, Thailand tivities. Many markets are becoming more competitive and more international. Technological advances in logistics and I. INTRODUCTION distribution enable almost every business to trade and work together around the world. The element that makes e- Today, the Thai government focuses on its “Thailand commerce business successful is technology and logistics 4.0” policy. As part of the Thailand 4.0 strategy of a [9]. In the digital age, customer satisfaction has changed, technology-driven economy, the Thai Government’s aim giving priority to excellent service for the character’s care, is to build a “Digital Economy and Society” to enable quality of service and restoration offered not only about Thailand to become a digital leader and compete within the products or services. But also pay attention to customers ASEAN economic community. E-commerce is one way to by themselves. Most customers in this era want comfort achieve strategies [1]. In Thailand, the e-commerce market and responsiveness [10]. is the second largest in Southeast Asia and is expected to grow around 22% annually until 2020. With support from This article identifies, through a retrospective study of the Thai government, a “Digital Thailand” initiative that literature, the transitional dynamics of traditional crafts started in 2016 has brought about a wave of opportunities of South and South East Asia. Many researchers have for businesses across different industries to digitize their recorded the reasons behind the trafficking and subsequent operations and services [2]. craft changes that have occurred, whether through tourism or the expansion of the export market. Some countries have At present, internet technology is the focus of most entered commercial production mode to develop poorly countries in the world and is used in business, which is crafted production communities, while some countries are e-commerce. In general, e-commerce is not only about trying to restore the degraded tradition. However, tradi- business and technology, but also directly and indirectly, tional crafts have a tendency to change in a lively market such as generating income for traditional craft products [11]. E-commerce can provide such artisans an opportunity or wisdom products, leading to the conservation of tradi- to reap the advantage of widened markets beyond the limits tional craft products. E-commerce can stimulate economic of geographical boundaries [12]. The handicraft sector in growth, facilitate exports, stimulate rural development, India is one of them that received large international mar- generate government revenue and create new jobs [3]. E- kets and increases sales. It also helps the craftsmen to have commerce has opened a gateway of new opportunities for micro, small, and medium-sized enterprises (MSMEs) to access international markets, find new sources of demand, and build value through exposure to new technologies 114

2019 4th International Conference on Information Technology (InCIT2019) more content. Craftsman development is a prerequisite for Fig. 1. The conceptual framework based on research procedures. all e-commerce development by reducing marketing chan- nels helps to fulfill such requisite [13]. Cultural heritage traditional Thai craftsmanship that can be developed into a embodied in traditional crafts is an important part of any creative economy and 3) sales of traditional Thai craftsman- country that reflects the culture and traditions of a specific ship products through electronic commerce systems. After region. The handicraft sector is important in generating that, analyze data from in-depth interviews using content income and employment and is also recognized worldwide analysis from the answers obtained by using the triangular as a tool to reduce poverty. It is a way of conserving and technique to find the integrity of all data. promoting culture and art, traditions such as techniques and skills of traditional crafts that are transmitted from For the e-commerce system development process, it generation to generation. For many countries, outstanding consists of five steps as follows: Firstly, we inquire from cultural heritage is still important in their craft [14]. In manufacturers and users of e-commerce systems in the addition, Bhutan also uses e-commerce with craft shops requirement analysis phase. Consequently, we analyze and and is feasible in the market because online services are design system by context diagram and data flow diagram popular in the market and most people like to buy online (DFD). After that, the e-commerce system is developed by with convenient options at a price [15]. MySQL, phpMyAdmin, Apache and Sublime Text. This system consists of the main work systems including mem- Thailand has a traditional Thai craftsmanship inherited ber system, searching and listing products system, shopping from generations to generations in communities that are cart system and management system for administrators. some centuries old. Which is valuable, should be con- Next step, test and install the system, it is a unit test pro- served. Therefore, traditional Thai crafts products should gram, system integration test, and bug fixes. Afterwards, the be supported in the use of e-commerce in order to create researcher will test and improve the system. If the system opportunities for these manufacturers to be entrepreneurs in is complete then it will be distributed via the internet. the online business. Our goal is to create a fair income for Finally, we evaluated the efficiency of the system with manufacturers of Thai handicraft products. The use of e- questionnaires about information quality, system quality commerce for Thai handicraft products has resulted in Thai and service quality. Moreover, we have to maintain this handicraft products that remain, as it increases revenue and system by monitors system performance, rectifies bugs and distribution channels for manufacturers. requested changes from users. Therefore, we consider that it is very necessary to C. Data analysis develop the e-commerce for the preservation of traditional Thai craftsmanship which will start a new business (start The data analysis of the traditional Thai craftsmanship up) for the traditional Thai craftsmanship manufacturer and e-commerce system is as follows. (1) Analyze data from in- to preserve the traditional Thai craftsmanship. This paper is depth interviews about the traditional Thai Craftsmanship organized as follows: Describing the research methodology, using content analysis from the answers obtained by using including population and sample, data collection method the triangular technique to find the integrity of all data. and procedure and data analysis in Section II. In Section (2) Analyze data from questionnaires for evaluating the III, results are presented. Finally, conclusion is in Section efficiency of e-commerce systems by basic statistics, such VI. as median and standard deviation. The mean score of each subscale or item were coded as shown in TABLE I. II. RESEARCH METHODOLOGY III. RESULTS In this section, the research methodology is described. In our approach consists of two steps: in-depth interviews The results of the study were presented in in-depth inter- about traditional Thai craftsmanship and developing of e- views results, system development and system assessment. commerce systems. We propose our framework as shown in Fig. 1. A. Population and sample The population and the sample group are the theoret- ical key informants, namely, experts with knowledge of the traditional craftsmen work of 15 persons and experts with knowledge, ability and expertise in e-commerce of 3 persons. B. Data collection method and procedure Our framework starts from an in-depth interview with the manufacturer of the traditional craftsman’s products which can be developed into a creative economy. The questions in- clude content of traditional Thai craftsmanship and product trading channels. The issues of in-depth interviews are as follows: 1) knowledge of traditional Thai craftsmanship 2) 115

2019 4th International Conference on Information Technology (InCIT2019) TABLE I SCORE MEANING OF SUBSCALE Levels Scores Very High 4.50-5.00 3.50-4.49 High 2.50-3.49 Moderate 1.50-2.49 1.00-1.49 Lower Lowest A. In-depth Interviews Results Fig. 2. Examples of traditional Thai craftsmanship. Results from in-depth interviews found that products Fig. 3. Content of traditional Thai craftsmanship. sold through the e-commerce system should be produced on time and have a clear pattern. But traditional Thai B. System Development craftsmanship products made by hand, which may be The developed e-commerce system can be accessed at inaccurate from the information presented. Therefore, the seller should specify various restrictions for customers the website www.ich-thaicraft.com. This system divides the to know as well. For trading channels, traditional Thai work into 4 groups: non-member users, buyers, sellers and craftsmanship products are usually sold through Facebook, administrators. Line and the market that the government has organized, such as OTOP. Although some products are not suitable to • Non-member users: They can only view product in- be sold through e-commerce, it can increase the distribution formation. If they want to buy or sell products, they channels of such products.If more products are sold, it must apply for membership only, as shown in Fig. 4. will help preserve the traditional Thai craftsmanship from another channel. • Buyers: The system for buyers consists of a mem- bership management (Fig. 5), product search (Fig. 6), Therefore, traditional Thai craftsmanship that has been shopping cart (Fig. 7) and payment (Fig. 8). registered as a national cultural heritage in 2009-2015 which can be developed into a creative economy includes: • Sellers: The system for sellers consists of a prod- uct management, payment management, transportation • Fabrics and products from the fabric, such as Sin Tin management, notification and customer order manage- Chok tube skirt, Thai loincloth/Pha Kao Ma, Thai ment, as shown in Fig. 9. Hill Tribe fabric, Koh Yor weaving fabric, Tai Khrang weaving fabric, Tai-Phuan’s weaving fabric, Tai-Yuan • Administrators: The system for administrators consists weaving fabric, Tai Lue weaving fabric, Na Muen Si of membership management (Fig. 10) and trading textile, Ubon Ratchathani textile, Phrae Wa cloth, Mat Mi cloth, Yok cloth and Indigo Dyed cloth. • Wicker, such as bamboo wicker and Yan Li Phao wicker. • Lacquerware, such as Thai mother of pearl inlay and Thai lacquerware. • Pottery, such as Wiang Kalong pottery and Ratchaburi dragon jars. • Metal, such as Khan Long Hin or Thai bronzeware of Baan Bu community, Thai silver, Ban Pa-Ao brass, alms bowl of Ban Batra community, Phaguamngoen or silver beads of Surin province and Aranyik knife. • Wood, such as Thai fiddle carving. • Leather, such as Nang Talung and Nang Yai. • Decoration, such as antique gold jewelry of Phetch- aburi. • Traditional Thai art, such as Lanna lantern, gold leaf beating and Khon head mask. Examples of traditional Thai craftsmanship which can be developed into a creative economy shown in Fig. 2. The content of traditional Thai craftsmanship consists of registration year, history, production, production area, special characteristics or identity, value and utility, as shown in Fig. 3. 116

2019 4th International Conference on Information Technology (InCIT2019) checks (Fig. 11). Fig. 7. The screen shows the products ordered. Fig. 4. Login screen to subscribe for buyers. Fig. 8. Payment status notification screen. Fig. 5. Fill-in information screen to subscribe for buyers. Fig. 9. Product management screen for sellers. Fig. 6. Product search by category. Fig. 10. Screen for managing members. C. System Assessment The statistics used to evaluate the system are median (X˜ ) and standard deviation (S.D.).The overall of the effi- ciency of e-commerce systems is at a high level (X˜ =4.00, S.D.=0.67). The efficiency of information quality and sys- tem quality is at a high level (X˜ =4.00, S.D.=0.63 and X˜ =4.00, S.D.=0.82, respectively) while the efficiency of 117

2019 4th International Conference on Information Technology (InCIT2019) TABLE II PERFORMANCE LEVEL OF THE E-COMMERCE SYSTEM Performance Issues (X˜ ) S.D. Level Data Quality 1. The data is clear, complete, 5.00 0.70 Highest useful and without duplication. 2. The data is accurate and reliable. 4.50 0.92 High 3. The link is correct according to the link and goes 4.00 0.88 High to the correct content. 4. The method of presenting Fig. 11. Screen shows the details of the order information. information is clear, accurate, 4.00 0.94 High service quality is at a moderate level (X˜ =3.00, S.D.=0.71), reliable and interesting. as shown in TABLE II. 5.Content classification is easy 5.00 0.85 High IV. CONCLUSIONS to find and understand. In this paper, we proposed the developed e-commerce 6. The order of content is system for the preservation of traditional Thai craftsman- ship. In our approach consists of two steps: in-depth inter- step by step and continuous 3.50 0.92 High views the traditional Thai Craftsmanship and developing of e-commerce systems. Research results were as follows: and easy to understand. 1) The content of traditional Thai craftsmanship that has been registered as a national cultural heritage in 2009-2015 7. Data display matches 3.50 0.88 High which can be developed into a creative economy consists the needs of users. of registration year, history, production, production area, special characteristics or identity, value and utility. More- 8. The use of images and over, manufacturers should produce products in time and a clear pattern. Additional conditions should be specified that video media is appropriate 4.00 0.70 High the image and color may be inaccurate due to being made by hand. 2) The developed e-commerce system using the and able to convey meaning. traditional SDLC is divided the work into 4 groups: non- member users, buyers, sellers and administrators to support 9. Images and text on the comprehensive work. When selling products through the e- commerce system, it can increase the distribution channels screen are appropriate and 3.50 0.82 High for manufacturers directly and increase the public relations consistent and orderly and channels. Therefore, this system is considered as a way to help preserve the traditional craftsmanship of Thailand to easy to understand. be able to remain in Thai society. Data quality overall 4.00 0.63 High The limitations on the sale of traditional Thai craftsman- ship products in the electronic commerce system found that System Quality some products are not suitable for sale in the e-commerce system. Since traditional Thai craftsmanship products are 1. The layout of the system 4.00 0.74 High hand-made products. Therefore, there are limitations such structure is very good. as product production may take a long time or cannot be produced in time. In addition, product styles, colors, 2. Menu covers all processes. 3.00 0.84 Moderate patterns, and details may be mistaken for images displayed in the e-commerce system, which may cause problems later. 3. The time spent downloading 3.00 0.97 Moderate Because the buyer expects that the product ordered from the data is appropriate. e-commerce system must be the same as the image seen in the system in all respects. But when seeing the real thing, 4. The management of the it may not be like the photos in the e-commerce system. Therefore, selling traditional Thai craftsmanship products membership system is 4.00 0.74 High in e-commerce systems should consider these factors in order to succeed in selling products through electronic a step and convenient. commerce systems. 5. The management of the In the future, the system may be developed as a web portal or a central platform that is a center to gather search system and product listing 4.00 0.82 High are steps and convenient. 6. The management of the shopping cart system is 4.00 0.67 High easy and convenient. 7. The management of the administrators is easy 4.00 0.88 High and convenient. 8. The management of the business support system is 3.50 0.70 High easy and convenient. 9. The system is secure and 3.50 0.82 High reliable. System quality overall 4.00 0.82 High Service Quality 1. The system responds quickly 4.00 0.82 High to users. 2. The system can be used 4.00 0.67 High efficiently. 3. The instructions for use 3.00 0.71 High are clear. 4. Publicity of information to 3.00 0.71 Moderate members has many channels. 5. Contact channels with 3.50 0.97 Moderate operators are clear. 6. Search channels for 3.50 0.82 High relevant information are easy. 7. The system achieves the 4.50 0.70 High planned goals. 8. The system is easily 4.00 0.67 High accessible and easy to use. 9. There is a system to help 3.00 0.70 Moderate multi-channel users. Service quality overall 3.00 0.71 Moderate System overall 4.00 0.67 High 118

2019 4th International Conference on Information Technology (InCIT2019) distribution channels for traditional Thai craftsmanship products. In addition, users can purchase a full range of products on all channels through a single website for more public benefits. ACKNOWLEDGMENT This work was assisted by Suan Dusit University for supporting the scholarship. Additionally, the invaluable recommendation and supervision from the anonymous re- viewers are much appreciated. REFERENCES [1] R. Bukht and R. Heeks, “Digital Economy Policy: The Case Example of Thailand,” in Development Implications of Digital Economies. Centre for Development Informatics: UK, 2018. [2] International Trade Administration, “Thailand–eCommerce” Re- trieved May 22, 2019, from https://www.export.gov/article?id= Thailand-ecommerce, 2018. [3] S. N. Sirimanne, “A National E-commerce Strategy for Egypt” Retrieved May 28, 2019, from https://https://unctad.org/meetings/en/ Presentation/dtl eWeek2018p57 ShamikaSirimanne en.pdf, 2018. [4] International Trade Centre, New Pathways to E-commerce: A Global MSME Competitiveness Survey, Geneva, 2017. [5] G. S. Kumar and J. T. Jose, “Developing an electronic commerce platform,” 2017 IEEE International Conference on Power, Control, Signals and Instrumentation Engineering (ICPCSI). Chennai, India, 2017, pp.1309–1313. [6] S. Shahriari, M. Shahriari and S. Gheiji, “E-commerce and IT impacts on global trend and market,” INTERNATIONAL JOURNAL of RESEARCH –GRANTHAALAYAH A knowledge Repository, Vol.3, Iss.4, 2015, pp. 49–55. [7] H. Hallikainen and T. Laukkanen, “National culture and consumer trust in e-commerce,” in International Journal of Information Man- agement, vol. 38, 2018, pp. 97–106. [8] S. J. Wahee and B. R. Bhardwaj, “Role of Ecommerce in enhancing Cyber entrepreneurship in India: Implications through caselets,” 2015 2nd International Conference on Computing for Sustainable Global Development (INDIACom), 2015, 2146–2153. [9] I. M. Krishna and G. V. Chalam, “E-Commerce Explosion and Indian Consumer: An Explanatory Study,” The IUP Journal of Information Technology, vol. XII, no. 4, pp. 7-20, December 2016. [10] Hendra, Rini, E. S., Ginting, P. and Sembiring, B. K. F., ”Impact of eCommerce Service Quality, Recovery Service Quality, and Satisfac- tion in Indonesia,” in 2017 International Conference on Sustainable Information Engineering and Technology (SIET), 2017, pp. 35-40. [11] L. J. Chutia and Sarma, M. K., “Commercialization of Traditional Crafts of South and South East Asia: A Conceptual Model based on Review of Literature,” IIM Kozhikode Society & Management Review, vol. 5, no. 2, 2016, pp. 107-119. [12] A. Shah and Patel, R., “E-Commerce and Rural Handicraft Artisans,” Voice of Research, Vol. 5, Issue 3, 2016, pp. 24-29. [13] J. A. Bhat and P. Yadav, “Relevance of E-Commerce in the Hand- icraft Marketing,” in New Man International Journal of Multidisci- plinary Studies, vol.4, no.9, 2017, pp. 44–54. [14] Y. Yang, M. Shafi, X. Song, and R. Yang, “Preservation of Cultural Heritage Embodied in Traditional Crafts in the Developing Countries. A Case Study of Pakistani Handicraft Industry,” in Sustainability, MDPI, Open Access Journal, vol. 10(5), 2018, pp. 1–18. [15] Ministry of Labour and Human Resources, Royal Govern- ment of Bhutan. “E-Commerce Handicraft,” Retrieved May 22, 2019, from http://www.molhr.gov.bt/molhr/wp-content/uploads/2018/ 05/E-Commerce fv.pdf, 2017. 119

2019 4th International Conference on Information Technology (InCIT2019) Rehabilitation Exercise Prescription on Android System Yotanut Maliwan Tanabodin Chiencharoentanakij Faculty of Information and Communication Technology Faculty of Information and Communication Technology Mahidol University Mahidol University Nakhon Pathom, Thailand Nakhon Pathom, Thailand [email protected] [email protected] Nuttapon Sornanunkul Thitinan Tantidham Faculty of Information and Communication Technology Faculty of Information and Communication Technology Mahidol University Mahidol University Nakhon Pathom, Thailand Nakhon Pathom, Thailand [email protected] [email protected] Abstract— The rehabilitation exercise prescription widely used in Thailand. Doctors can use this application to (Rehab-X) aims to develop Android mobile applications create exercises prescription for patients with a menu to dividing into Rehab-X doctor for manipulating patients' choose the form of exercise poses relying on body part or visiting records and exercise prescriptions and Rehab-X symptom, stretch or strengthen and left or right body side. patient for viewing of the exercise prescription and watching Doctors can be able to identify the number of times to hold exercise video. each pose, the number of times to repeat and the number of times to do per day. To prescribe an exercise prescription, exercise pose information must be added to the system. The exercise poses Patients can receive the exercise prescription in a PDF are separated into two types: symptom and body part and each file via their email as well as the printed paper from the type is classified into stretching and strengthening. Each Rehab-X doctor application. In case, the patients have an exercise also has an associated video for a demonstration to Android smartphone, they can use the Rehab-X patient patients. The exercise prescription includes the exercise application on their smartphone by scanning the QR (Quick photos, step by step descriptions, and the QR code that links to Response) code from the Rehab-X doctor application to the demonstration video for each exercise pose. Moreover, the receive the exercise prescription. The system has QR code doctor can specify each exercise pose with a duration period containing a link to a video for each exercise pose so that for each time and the number of times to do per day. After all, patients can review and watch the video from their own the system creates the exercise prescription in a PDF file and smartphone at home. saves it in a particular patient's medical record as well as generates a QR (Quick Response) code for the patient to scan The remainder of the paper is organized as follows. and view the exercise prescription via his/her own Android Section 2 reviews and compares related work with our smartphone. This paper describes Rehab-X and programming project. Section 3 and 4 describes Rehab-X system and techniques which can be applied for issuing any further implementation techniques. Section 5 illustrates results and medical prescriptions on Android smartphone system. evaluations. Finally, Section 6 concludes our project work and provides future work. Keywords— Rehabilitation, Exercise, Prescription, Android, Mobile Application II. RELATED WORK I. INTRODUCTION x EndoRush [4] EndoRush is an exercise application designed for both Exercise is a part of rehabilitation therapy for body restoration and therapeutic means by strengthening or iOS and Android platforms. This application contains over stretching in order to enhance muscle flexibility and bone 800 exercises and categorizes more than 800 physiotherapies joints to have better movement and also to help the internal such as knee OA (Osteoarthritis), ACL (Anterior Cruciate systems of the body such as the heart and lung. [1-3] Ligament) injury, low back pain, and shoulder pain. Doctors usually use rehabilitation exercise to treat the x Rehab Guru [5] patients who suffer painfully or have movement problem at Rehab Guru is a software tool for doctor and patient various parts of their body. Currently, doctors prescribe a set of exercises in the form of paper where these exercises are which can be operated on both website and mobile demonstrated by a physical therapist at the clinic. Patients application. This application contains over 4500 exercises perform the exercises prescribed in the paper at home. and includes many features like feedback, progress tracking, However, the patients may not be able to remember all the and search. This application also has high definition video details of the exercise poses. Moreover, this paper can be and images to help the doctor and also provides exercise lost. As a consequence, the treatment may not be successful descriptions to their patient. and the patients still have chronic pain. x SimpleSet [6] Therefore, this project has developed a rehabilitation SimpleSet is the exercise prescription software running exercise prescription called Rehab-X based on the requirements and guidelines of the Department of on both mobile application and website to help the doctor Rehabilitation Medicine at the Faculty of Medicine with the exercise prescription process. This application was Ramathibodi Hospital. This system has been developed for designed by healthcare professionals for doctors to deliver Android-based smartphone system which is open source and an effective exercise course and therapy to their patients. 120

2019 4th International Conference on Information Technology (InCIT2019) x Exercise Prescriber [7] Figure 1 illustrates the flow of the rehabilitation exercise prescription (Rehab-X), which separates into two main parts. Exercise Prescriber is a clinical tool for health The first part is an exercise prescription process which professionals running on the website. The purpose of this allows a doctor to edit and create an exercise prescription for website is to help doctors to provide home exercise a patient. The second part is for the patient to view exercise information and advice for their patients. This online prescription in PDF file via email, a printed paper, or his or exercise is also an ideal software for physiotherapist, her smartphone. osteopaths, chiropractors, sports therapists, occupational therapists, athletic trainers, doctors, personal trainers, A. Rehab-X Doctor strength and conditioning coaches as well as exercise physiologists. This application allows doctors to do the followings: For our project, the rehabilitation exercise prescription - Create exercise prescription for patients, system (Rehab-X) is designed based on an Android smartphone. We deploy the SQLite as a local database to - Select proper exercise poses for each visit of patients save the exercise descriptions and details as well as doctor and patient data. We use Android Studio to develop the user - Edit patient records interface. Rehab-X doctor application is to allow doctors to provide the exercise prescription for patients in printed PDF - Create links to an exercise video demonstration file which can be sent to patient’s email and stored as medical records for each visit. Rehab-X patient application B. Database is for the patient to view the exercise prescription and watch the exercise videos via youtube. Table I compares the 1) SQLite Database System features of our project and related work. SQLite database system is used for collecting doctor, TABLE I. THE COMPARISON OF EXERCISE THERAPY APPLICATIONS patient and exercise data from Rehab-X Doctor as a record of doctor profile, patient profile and exercise prescription Rehab-X EndoRush Rehab Guru SimpleSet Exercise and save them to the local storage under the mobile [4] [5] [6] Prescriber [7] application. It also maintains the old prescription records for the doctor to re-prescribe the same prescription for the Users Doctor, Doctor, Doctor, Doctor Doctor patient. Patient Patient Patient 2) ER Diagram Platform Android Android, Android, Android, Website iOS iOS, Website iOS, Website Figure 2 and 3 describe data dictionary and ER diagram for Rehab-X doctor respectively. Rehab-X patient Patient ‫ض‬ ‫ض‬ ‫ض‬ ‫ض‬ ‫ض‬ contains only PRESCRIPTION EXERCISE TABLE and Selection EXERCISE TABLE which allows users to view the prescription while Rehab-X doctor allows users to add and Favorite ‫ض‬ x ‫ضض‬ ‫ض‬ edit doctor and patient information as well as create exercise Management prescriptions. Prescribe ‫ض‬ ‫ض‬ ‫ض‬ ‫ض‬ ‫ض‬ Exercise Categories ‫ض‬ ‫ض‬ ‫ض‬ ‫ض‬ ‫ض‬ Exercise ‫ض‬ x ‫ض‬x x Search Medical ‫ض‬ x ‫ضض‬ ‫ض‬ History Customize ‫ض‬ ‫ض‬ ‫ض‬ ‫ض‬ ‫ض‬ Instruction Prescription x x Export in ‫ض‬ ‫ضض‬ PDF Offline ‫ض‬ x x x x Operation III. SYSTEM ARCHITECTURE Fig. 2. Data Dictionary for EXERCISE TABLE Fig. 1. System Flow of the Rehabilitation Exercise Prescription (Rehab-X) 121

2019 4th International Conference on Information Technology (InCIT2019) Fig. 3. ER Diagram Use in Rehab-X Doctor 3) Tools Remarks: every Development Tools button in the list can - Android Studio for develop an Android be select to view an application - Visual Studio Code for source-code editor exercise detail - Notepad++ for HTML editor - Eclipse for exercise prescription editor Fig. 4. View and Edit Exercise Flow of Rehab-X Doctor Graphic Tools - Adobe Premiere Pro for video editor 2) Search Box - Adobe Photoshop for picture editor This function appears as a search box at the top of - Microsoft Paint for screenshot picture editor the screen in Figure 5 in the flow of B and Figure 6 in C. Rehab-X Patient the flow of E. This application allows patients to do exercises as the The main function of a search box is to let the user doctor has prescribed. They can view the prescription type in the keyword to find the data which will be shown digitally on their own smartphone to watch video for each in a list view. prescribed exercise pose. This function can be used by selecting the search IV. IMPLEMENTATION TECHNIQUES box. After that, the cursor will appear in the search box and an on-screen keyboard will be popped up from the A. System Functions bottom of the screen. The user needs to type the keyword In order to make such a system to support the prescribing that they want to search. function on the doctor smartphone and to view an exercise The search box function is activated when the user prescription on the patient smartphone, the eight methods selects the search box on the screen. As the user types are need and described as follows. the words into the search box. The application will send 1) Segmented Control Button This function appears as a button at the top of the screen in Figure 4 in the flows of F and H. The main function of them is to allow users to swap and switch the exercise categories between the left body side and the right body side and between strengthening and stretching types of exercise. The user can use this function by pressing or selecting the button at the top of the screen. The list of exercises be shown according to the keyword on the button. When the button is selected. The application will send the keyword to the SQLite. After that, the SQLite server returns the list of exercises. 122

2019 4th International Conference on Information Technology (InCIT2019) the word to the list view query. The list view handler creating the prescription, then the application will add will filter the result regarding to the provided keyword. those checked exercises for the prescription. When the star button is selected in each exercise. The application will send the favorite exercise to the SQLite. For the checkbox button, when the user selects the checkbox. The application will save the exercise to the prescription file. Fig. 5. Edit Patient Information Flow Fig. 8. Doctor Registration Flow Fig. 6. Create New Prescription Flow Fig. 7. Create Prescription from Old Record Flow Fig. 9. Overall of Patient Application Flow 3) Custom Buttons in a List View 4) Database Initialization This function appears as a star button in the list This function runs when the application starts for view in each exercise from Figure 4 in the flows of B, the first time at screen as shown in Figure 8 in the flows D, F and H as well as appears as a star button coupling of B and Figure 9 in the flow of B. with a checkbox in each exercise from Figure 6 in the flows of E, G, I, J and K and Figure 7 in the flows of F, The main function of it is to copy the master H, J and K. exercise database consist of images and exercise details to the working environment. The main function of a star button is to let the user select the favorite exercise which will appear in the The application will take care of this function in favorite exercise list and the checkbox button is to let the the background when the application installed and open user select the exercises for creating a prescription. for the first time. The user can use this function by selecting the The application initializes the exercise database button that appears in each exercise. If the user selects when the splash screen appears on the screen. By copy, the star at the exercise in the list view, then those the master database file is contained in APK setup files exercises will appear in the favorite list. If the user to the application cache folder. selects the checkbox at the exercise in the list view while 5) Sending Prescription to the Patient via Email This function appears as a button at the bottom of the screen named “ส่งอีเมลให้ผูป้ ่ วย” or in English “Send an Email to the Patient” in Figure 6 in the flows of L, Q and R and Figure 7 in the flows of M, R and S. 123

2019 4th International Conference on Information Technology (InCIT2019) The main function of this button is to send a B. Data Preparation prescription in PDF format via an email to the patient. In order to create an exercise prescription. We need to The doctors can use this function after they fill the prepare the exercise data including images, video patient’s email address at the screens shown in Figure 5 in the flow D and F before creating the prescription. demonstration, and description. The methods of data preparation can be done as follows. When the button is selected. The application will retrieve a recipient email address from the patient 1) Video Editing profile. After that, the Java Mail API will attach the We use software Adobe Premiere Pro CC 2018 and prescription in a PDF file to the email and send email using the built-in email address. Google Chrome to add copyright watermark, remove 6) HTML Prescription Generator sound and remove background from the video. The steps can be proceeded as follows. This function appears as a button at the bottom of 1. Import video to Adobe Premiere Pro. the screen named “ออกใบสังท่าใหม่” or “Create new 2. Use “Razor Tool” to trim the first part of the video prescription” as shown in Figure 6 in the flows of K, L, 3. Use “Crop Tool” to crop left and right of video that and N and in Figure 7 in the flows of L, M, and O. is not necessary for video The main function is to create the prescription in 4. Use “Color Key” to remove the background of the an HTML file format before converting them to PDF file using Google Chrome WebView engine. video 5. Import background photo and add watermark After the user finishes choosing the exercises from 6. Export Video in MP4 file format and Upload on multiple categories. The user can select the button “ออก YouTube ใบสงั ท่าใหม”่ . 7. Go to YouTube studio, set the link to unlisted so the The application will retrieve the exercises list and video will not publicly available and copy the link its details consist of freeze time, repeat, number of times, of video the weight of dumbbell and weight of sandbag send to the HTML editing engine. 8. Put video link in QR Code Generator and save QR 7) QR Code Generator Code This function appears as a button at the bottom of 2) Photo Editing the screen named “QR CODE” as shown in Figure 6 in the flows of L, O and P and in Figure 7 in the flows of We use software Adobe Photoshop CC and Movies M, P and Q. and TV as Windows 10’s built-in applications. This step The main function is to create a QR Code contains is to remove the bark. The steps can be proceeded as the exercise prescription and its details for sending the follows. data to the patient’s application 1. Capture the pose from video file 2. Open Adobe Photoshop and create new document This function can be used by selecting the “QR CODE” button at the bottom of the screen. The screen size 500 x 500 px will show the generated QR Code for the patient’s 3. Import picture that capture into the document application to scan and receive the prescription. 4. Use “Magic Eraser Tool” to remove background When the button is selected. The application will from the picture retrieve the exercise details into a string and convert them to QR Code image by using ZXing QR generator 5. Add Number of actions to picture and save it library. 3) Create IFrame 8) QR Code Reader An IFrame (Inline Frame) is a HTML document This function appears as a button at the bottom of embedded inside another HTML document. The IFrame the screen named “รับใบสังทา่ ใหม่” or “Receive the new HTML element is used to insert block of exercise that users select in the prescription process to the main prescription” as shown in Figure 9 in the flows of C, E, HTML file. This data is appeared as in Figure 6 in the F, G and H. flows of L and N and in Figure 7 in the flows of M and The main function is to read the QR Code O. generated in a doctor’s application. After that convert, V. RESULTS AND EVALUATION the prescription encoded in the QR code and save to the patient’s exercise prescription on a patient’s application. A. Performance Evaluation The purpose of this testing is to evaluate the The user can use this function by selecting the button at the bottom of the screen. The application will performance of the system to create a PDF file as described show the choices “ยอ้ นกลบั ”, “เลอื กจากรูปภาพ” and “ใชก้ ลอ้ ง” or in Table II with different smartphone systems [8-11]. As the results, the elapsed time is varied to the number of exercise “Back”, “Select from Image” and “Use Camera”. poses ranging from 3 to 15 seconds for 5 to 30 poses When the QR image has been retrieved using the respectively. choices mentioned above. The system will decode the prescription in the QR Code generated by the doctor’s TABLE II. THE ELAPSED TIME FOR CREATING PDF FILE IN SECONDS application and save the exercise prescription to the patient’s application. Model Sony Xperia XA1 Ultra [8] Vivo V7+ [9] Samsung Galaxy A7 (2017) [10][11] OS Android 8.0 (Oreo) Android 8.1 (Oreo) Android 8.0 (Oreo) MediaTek MT 6757 Helio Qualcomm Snapdragon Exynos 7880 Octa Specification P20 Octa Core (2.3 GHz) 450 Octa Core (1.8 Core (1.9 GHz) GHz) RAM 4 GB RAM 4 GB RAM 3 GB 5 poses 4 34 10 poses 6 5 5. 20 poses 9 88 30 poses 15 12 11 124

2019 4th International Conference on Information Technology (InCIT2019) B. User Evaluation manipulate exercise information including images, videos The purpose of this testing is to evaluate user and exercise details on Rehab-X Doctor as well as to use cloud database system instead of the SQLite database which satisfaction and user experience about the application. Figure 10 and 11 show the evaluation results. The rightmost, can facilitate a doctor to transfer patient information to the the middle, and the leftmost bars are represented for the others. Furthermore, Rehab-X Patient application might quality of the application, the design, and the overall attach a function coupling with wearable sensors to detect application. The scores are from 5 to 1 where 5 means very and measure the range of movement in order to track good and 1 means very poor or need to be improved. exercise-rehabilitation progress. 1) Rehab-X Doctor This evaluation is performed by 4 doctors and 2 VII. ACKNOWLEDGEMENT physical therapists. As described in Figure 10, the The authors would like to thank the National Software results indicate that they are quite satisfied with the Contest (NSC2019) for funding this project number application. 21p13w0028. We genuinely thank Dr. Poramed Chayaratanasin, MD, Department of Rehabilitation Fig. 10. Evaluation Results of Rehab-X Doctor Medicine, the Faculty of Medicine Ramathibodi Hospital, Mahidol University as an expert in rehabilitation medicine 2) Rehab-X Patient who initiated this project and Assoc. Prof. Dr. Sudsanguan This evaluation results are performed by 11 users. Ngamsuriyaroj, Faculty of ICT, Mahidol University for valuable advices. As described in Figure 11, the results indicate that most are quite satisfied except one needs to have a modernize REFERENCES screen design. [1] Arora, S., Erosa, S., and Danesh, H. Physical Medicine and Fig. 11. Evaluation Results of Rehab-X Patient Rehabilitation. In: Khelemsky Y., Malhotra A., Gritsenko K. (eds) C. Problems and Limitations Academic Pain Medicine, Springer, July 2019. [Online]. Available: - Smartphone must use Android operating system https://link.springer.com/content/pdf/10.1007%2F978-3-030-18005- version 8.0 (Oreo) or above for Rehab-X doctor application and Android operating system version 8_23.pdf. [Accessed on 31-August-2019]. 4.0 (Ice Cream Sandwich) or above for Rehab-X patient application. [2] Heard, M., Buchko, G., Hiemstra, L., and Kopka, M. Rehabilitation - The number of exercises in the prescription can be Principles for Anterior Knee Pain. Banff Sport Medicine, May 2019. created less than 30 poses at the same time. - The Internet connection is required for watching [3] Leonard, J.W. Exercise Therapy. In: Abd-Elsayed A. Pain, Springer, video. May 2019. [Online]. Available: - The database is not allowed to edit on smartphone. https://link.springer.com/content/pdf/10.1007%2F978-3-319-99124- VI. CONCLUSIONS AND FUTURE WORK 5_225.pdf. [Accessed on 31-August-2019]. This paper presented the Rehabilitation Exercise Prescription (Rehab-X) system for a doctor to issue the [4] “EndoRush Physiotherapy app -,” EndoRush Physiotherapy app -. exercise prescription and for Thai patients to view the exercise prescription. The Rehab-X Doctor Android mobile [Online]. Available: https://www.endorushapp.com/. [Accessed: 29- application enables the doctor to create a local database of Apr-2019]. patient information and patients’ exercise prescription. The Rehab-X Patient Android mobile application enables the [5] “Rehab Guru | Powerful Exercise Prescription Software,” Rehab patient to view the prescription and watch each prescribed Guru | Powerful Exercise Prescription Software, 13-Mar-2019. exercise video. We believe that our system can be applied to issue other prescriptions on an Android system. In future [Online]. Available: https://www.rehabguru.com/. [Accessed: 29- work, we plan to add a function that allows the doctor to Apr-2019]. [6] “Simply Effective,” SimpleSet. [Online]. Available: https://www.simpleset.net/. [Accessed: 29-Apr-2019]. [7] “Exercise Prescriber - Making Clinical Lives Easier, Patient's Lives Better.,” Exercise Prescriber - Making Clinical Lives Easier, Patient's Lives Better.[Online]. Available: https://exerciseprescriber.com/. [Accessed: 29-Apr-2019]. [8] Siamphone Dot Com Co., Ltd. “SONY Xperia XA1 Ultra สมาร์ทโฟนรอง รับ 2 ซิมการ์ด หนา้ จอ 6 นิว ราคา 9,990 บาท - สยามโฟน.คอม,” Siamphone. [Online]. Available: https://www.siamphone.com/spec/sony/xperia_xa1_ultra.htm. [Accessed: 16-Apr-2019]. [9] Siamphone Dot Com Co., Ltd. “Vivo V7 สมาร์ทโฟนรองรบั 2 ซิมการด์ หนา้ จอ 5.99 นิว ราคา 7,990 บาท - สยามโฟน.คอม,” Siamphone. [Online]. Available: https://www.siamphone.com/spec/vivo/v7 .htm. [Accessed: 16-Apr- 2019]. [10] Siamphone Dot Com Co., Ltd. “Samsung Galaxy A7 (2017) สมาร์ทโฟนร องรับ 2 ซิมการ์ด หนา้ จอ 5.7 นิว ราคา 12,990 บาท - สยามโฟน.คอม,” Siamphone. [Online]. Available: https://www.siamphone.com/spec/samsung/galaxy_a7_(2017).htm. [Accessed: 16-Apr-2019]. [11] “Samsung Galaxy A7 (2017),” Samsung Galaxy A7 (2017) - Full phone specifications. [Online]. Available: https://www.gsmarena.com/samsung_galaxy_a7_(2017)-8335.php. [Accessed: 16-Apr-2019]. 125

2019 4th International Conference on Information Technology (InCIT2019) Online Examination System with Cheating Prevention Using Question Bank Randomization and Tab Locking Samuel S. Chua Joshuel B. Bondad College of Technology College of Technology Lyceum of the Philippines University Lyceum of the Philippines University Manila, Philippines Manila, Philippines [email protected] Joven DL. Garcia Zechariah R. Lumapas College of Technology College of Technology Lyceum of the Philippines University Lyceum of the Philippines University Manila, Philippines Manila, Philippines Abstract— Online examination system is used by Exposing the long established method might prove to educational institutions to improve the quality of be unsuccessful to fully prevent academic malpractice instruction by having a supervised measure of outcomes for during examinations. E-learning has its vital and integral self-paced learning environments of their students. The assessment component using online examination [2]. reason E-learning became so popular is because of its fast Submitting exams in E-learning has already been done feedback in assessing the examiners or candidates. An without a proctor present. As a result, students can easily online examination system that has the ability to address commit academic malpractice during exams, educational academic malpractice should be the main concern to be able institutions with E-learning depend on an examination to trim down those acts at some degree. Saving time is one process on which students take the exam in a physical of the perks in having an Online examination system, but it controlled environment at the institution under a also had limitations on dependency to the quality of supervised condition, however, this contradicts the Internet service leaving both the proctor and the examiners concept of the live E-learning environment. not being able to use the system. The research looked into interviewing through a focus group the proctors of online Two primary benefits of online examination [4]. First, exams to identify root causes of academic malpractice at the is the large cost savings of the substitution of machines same time interview exam content creators on possible for labor in grading, and second is the potential for approaches on exam questions generators that allow a enhanced student learning due to more frequent validity of measure of outcomes. Generally, a final assessment. Another significant benefit is immediate validation done by the focus group respondents and end feedback to students on homework assignments and users for effectivity and usability. examinations. Upon submission of the assignments and exams, the software provides students with answer keys Keywords— academic malpractice; randomization; and their own responses. assessment; e-learning; online examination Another approach is controlling the local terminal of I. INTRODUCTION the examinee. There is a tool for learning system that lets you regulate an online assessment in the local terminal A. Background of Study called Respondus [8]. It can also assist in database inquiry to a few online courses. Existing Respondus partners in Online examination systems used by educational developing exams in E-learning are Moodle and institutions continue to address cases on academic Blackboard to name a few. One key feature of the tool is malpractice with a common problem today due to easier locking the environment to the online exam window method in attaining information for students in a learning which deters the examinee from further investigation or environment based on the internet. communication. Taking answers from another and plagiarizing Online examination results are assessed by the assignments are an example of an academic malpractice, computer resulting to the time and cost saved against its this is the problem educational institutions want to manual examination counterpart. Based on the advantages address [1]. Since the proctor was not able to keep tabs of the internet, modifications have been implemented to with every examiner. Randomization of exam has the the examination system concept by designing a website ability to address academic malpractice, this would with online examination that sets questions and answers provide the examiner’s with different exam that would by the proctor. Said approach is evaluated by the make them unable to commit such acts.. 126