Usability Evaluation of E-textbooks by EFL Teachers

Electronic textbooks (e-textbooks) provide great convenience to teachers and students in both being activity-oriented and having more than one media in it. Especially during the COVID-19, e-textbooks have become much more important than any other time for the effective teaching of online courses. Therefore, many publishers rearranged e-textbook versions of their printed publications in Turkey. Regarding human-computer interaction, making the usability tests of these e-textbooks will give ideas to e-textbook content developers. Thus, three of these e-textbooks were carefully analysed for this research. They are the products of Kurmay, Speed Up and YDS publishing houses. The study was carried out with seven voluntary English teachers who teach 8th grades, work at the Ministry of Turkish National Education and have at least 3 years of teaching experience. Due to the pandemic, the sessions were held online by using the ‘Zoom Meetings’, which is an online meeting software. The usability was analysed according to effectiveness, efficiency and satisfaction principles of the International Organization for Standardization (ISO). At the end of the test, it was observed that the usability of Reading Alley and Marathon Plus e-textbooks were high and the usability of Speed Up was very high.


Introduction
Due to the COVID-19 pandemic, education in Turkey has been carried out via distance education platforms since March 2020. As in face-to-face courses, textbooks are also important course materials for online courses. However, since screen sharing is crucial in distance education, doing an online course with printed textbooks or PDF versions may not be efficient for both teachers and students in terms of human-computer interaction. In this case, e-textbooks can make online courses effective, efficient and satisfy in distance education. According to Choppin and his colleagues, e-textbooks which are simply digitised versions of textbooks, have online annotation features and may also have interactive multimedia contents. According to Ozturk and Can, the views of teachers and students about e-textbooks are positive, and e-textbooks positively affect students' success and motivation (137)(138)(139)(140)(141)(142)(143)(144)(145)(146)(147)(148)(149)(150)(151)(152)(153). PDFs or printed textbooks may not be efficient to teach online lessons with listening activities, such as English lessons. As Lokar stated, e-textbooks make teachers' work easier and faster (101-106).
Thus, to make teachers' work easier and faster, many private publishing houses have focused on creating enriched e-textbooks versions of their printed publications during the distance education period. They also wanted to make a difference in the market and to impress their customers by highlighting their products. As a result, a lot of e-textbooks were developed during this period by e-textbook content developers and publishing houses. As Susser and Robb stated, it is satisfactory that they produce a large number and variety of resources on language education. Still, this situation leaves researchers to evaluate the quality of these resources (279)(280)(281)(282)(283)(284)(285)(286)(287)(288)(289)(290)(291)(292)(293)(294)(295).
International Organisation for Standardisation defines usability as "the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use". According to the definition of ISO, effectiveness is that users achieve specified goals with accuracy and completeness; efficiency is the resources such as time, effort, costs and materials used about the results achieved and satisfaction is that a product or service meets the user's needs and expectations. In this study, usability tests of three EFL e-textbooks were made. They were carried out by ISO's usability principles.
E-textbook producers may sometimes overlook the use cases of their products in the field and the design rules necessary for human-computer interaction. According to Bagis, the interaction between the user and the interface is examined through observations and measurements made in real environments (25-31). Phyo states that good design features create a positive experience for users and can lead to more desirable outcomes. The purpose of this research is to give suggestions to e-textbook developers about the usability of their e-textbooks from the view of the EFL teachers who are the main practitioners in the use of e-textbooks. Thus, they can develop e-textbooks that have more desirable outcomes.

Review of Literature
According to Ardito, the term 'electronic book' was first used by Andries Van Dam in the 1960s. The high visibility of e-books in the academic field started in 1971 with the Gutenberg Project, which made plain textbooks available free of charge in electronic format (Al Saadi et al.) Later, PDFs, in which digital content can be designed, were released and then a transformation started from books in other formats into interactive e-books (Bozkurt and Bozkaya, 2013). According to Ongoz and Baki, some researchers have argued that e-books differ from printed books as they offer some extra features that allow users to interact with the text through the use of audio, images, and links. Studies on the use of e-books in the teaching phase date back to the 1990s. According to Krug, understanding how users think can help create environments that meet users' goals and objectives. Saritepeci and Yildiz (2013) conducted a usability test of an e-textbook prepared by MINE (Ministry of National Education) of Turkey for the 6th grade Social Studies lesson. As a result of the test, the researchers determined that the e-textbook had a page preview on the right, the content in the middle, and the animation and activities related to the content were on the right. Since this fragmented structure disrupted the integrity of the e-textbook, they suggested that an easier and more memorable structure can be created with a single page view instead of this structure.

Materials and Methods
In this study, a qualitative research approach was adopted and a scanning model was used. Since the research is based on the views of English teachers, it is also descriptive.

Participants
According to Nielsen, five evaluators are sufficient to locate most problems in a usability study. In this study, participants consisted of 7 voluntary EFL teachers; 5 women and 2 men. They teach 8th grades and have at least 3 years of teaching experience in MINE in Konya, Turkey. A free online course including ten web 2.0 tools was given as a gift to the participants as they contributed to the research.

Selection of E-textbooks for Usability Testing
The e-textbooks used in this usability test were determined by eliminating those that gave opening errors. Those were closed unexpectedly, those have old curriculum content, those were paid ones and those did not have listening activities. The number of e-textbooks was eight at the beginning. It was consulted to three expert EFL teachers and the number was limited to the publishing houses that shared their contents on EBA which is an online social education platform run by MINE of Turkey. EBA means Education and Information Network. The selected e-textbooks and their links are below.
Reading Alley

E-Textbook Related Tasks
The e-textbook tasks were shaped by taking the opinions of three English teachers and two instructional design experts. The tasks were tested with 2 different users who did not participate in usability testing. After this pilot test, some tasks were reduced, some tasks were combined and simplified. Thus, the number of tasks was determined as eight.

Data Collection and Analysis
Google forms were used for demographic and usability test surveys. Due to COVID-19, usability test meetings were held via Zoom Meetings, an online meeting software. The researchers installed e-textbook software on their computers and gave remote mouse control to the participants. Thus, the participants did not have to install them on their personal computers. Participants were asked to think aloud while performing the given tasks. Screen recording was made during the test to determine the time participants used for each task and the confusion they experienced during the test.
The usability test questionnaire consisted of 9 items; 7 Likert types (1-Strongly Disagree, 5-Strongly Agree) and 2 open-ended items. The first three items measured the effectiveness, items 4-7 measured the efficiency and items 8-9 measured the degree of satisfaction.
To determine the usability of e-textbooks, the result obtained by subtracting 1, the lowest score, from 5, which is the highest score, in five-point Likert items, was divided into five equal parts (4/5=0,80). By adding 0,80 to the lowest score, the groups were formed as follows:

Results and Discussion
In this section, the total time (in seconds) used by the participants while performing all the tasks in the e-textbooks was examined and a descriptive analysis was made by comparing them with the demographic survey data. In addition, the time used by the participants was evaluated based on tasks, and it was discussed which e-textbook tool is more usable for a particular task. In Table 3, it was observed that there was no significant relationship between the graduation, years of professional experience, gender of the participants and the total time used for tasks. However, it was observed that there was a significant relationship between the ages of the participants, the frequency of e-textbook use, computer skills, vision problems and the total time used for tasks. The reason why there was a time difference between Participant-3 and Participant-6 in all e-textbooks may be the generation differences of the participants as well as their computer experience and e-textbook usage frequencies.

Findings of Times used for Tasks
One of the reasons why Participant-3 and Participant-4 use more time is that they find the sizes of icons and texts small as they have visual defects. According to demographic data, it is seen that Participant-1 is familiar with Reading Alley and Participant-6 is familiar with Reading Alley and Speed Up. The reason why these participants used less total time in these e-books may be their familiarity with these e-textbooks. In addition, it was observed that the e-textbook usage frequencies of the participants who use the least total time for all e-textbook tasks are "always". In this case, it may be said that the e-textbook usage frequencies of participants are an important factor in e-textbook usability tests.

Evaluation of Usability Levels of E-textbooks
In the questionnaire applied at the end of the usability test, the averages of the effectiveness, efficiency and satisfaction items were determined separately and reflected in Table 4. The average score of effectiveness in all e-books is more than 4.20. Therefore, it can be said that the effectiveness levels of all e-books are very high. As for efficiency, it is seen that the efficiency scores of the Reading Alley and Speed Up e-textbooks are high, and that of the Marathon Plus e-textbook is moderate. According to average scores, it is seen that the satisfaction levels of all e-books are high. When it comes to the weighted means, it can be said that the usability of the Speed Up e-textbook is very high, and the usability of the Reading Alley and Marathon Plus e-textbooks is high.
Participants performed all tasks in the Reading Alley e-textbook with an average of 96,95 seconds. The participant views about this e-textbook, which has a 3,83 average score in a usability test, are as follows: "I think it is a nice feature that the clicked icon becomes prominent when you click on it".  "Useful tools like 'Contents' could be in a more visible position. I think the drop-down menu bar is dysfunctional."  "I found the shape and text size of the menu icons insufficient for eyeglasses users."  Participants performed their tasks in the Speed Up e-textbook with an average of 73,32 seconds. The participant views about this e-textbook, which has 4,29 average scores in a usability test, are as follows: The overall colour choices for this e-textbook are vibrant and engaging. I liked that the toolbar is fixed and in a functional position for screen usage.  I didn't like that the listening media player come into the middle of the page whenever I click it. Also, the listening activity was a bit fast for the 8th grade.  I liked that the tools were larger. It's also nice to have the option to change the background colour.  Participants performed their tasks in the Marathon Plus e-textbook with an average of 127,84 seconds. The participant views about this e-textbook, which has a 3,71 average score in a usability test, are as follows: I found it useful that the functions of the tools appear as text when hovering over the tool icons.  I liked the location and functionality of the listening media player in terms of page usage.  The toolbar was in blue and dark blue tones; The figure-ground relationship could have been more remarkable.

Conclusions and Recommendations
In this section, the views of the participants about the usability of e-textbooks were evaluated and recommendations have been made.
• It is recommended that e-textbook content developers prefer vibrant colours in e-textbooks and ensure that the background colour of it is customisable. • After clicking on marker tools such as eraser and pencil, the participants expect the mouse cursor to turn into the icons of those tools. • Participants criticised the use of dark blue and blue tones together in the Marathon Plus e-textbook regarding the figure-ground relationship. Therefore, e-textbook content developers are advised to use contrasting colours in figure-ground relationships by instructional material design principles. • Participants stated that the menu toolbar should be designed in a more minimalist and simple way. They found the Speed Up e-textbook more usable with fewer tool icons. • The participants stated that they care that an exercise in e-textbooks should have both textbook and interactive forms together. In the textbook version of the activities, students can do individual exercises and see their true-false answers after they finish the activity. In the interactive form, student/teacher users receive audio feedback on their correct or incorrect answers and practice until they find the right answer. • Icons chosen for tools should be understood correctly and in the same manner by almost everyone. • It is necessary to avoid using similar icons for tools used in different tasks in the same e-textbook. In the Reading Alley, a magnifying glass icon is used for both the 'search' and 'select' icons. Likewise, in Marathon Plus, a left arrow icon is both used for the 'return' tool to see the whole page and the 'undo' tool to restore a deleted drawing. These can be negative examples of that situation. • No matter how understandable the icons are, it will be very useful if there is a user manual included in e-textbooks, as in Speed Up. Or else, as in Marathon Plus, one or two-word explanation about the function of the tools can appear when the mouse hovers over them. • In this usability test, the participants criticised that there is no change in the colour, shape, etc. of the tool icons when they click on them in Speed Up. Although this feature was present in Reading Alley and Marathon Plus, it was observed that an unclear, pale colour change did not satisfy the participants. That's why e-textbook content developers are advised to make a tool that users clicked stand out with a different colour, shape, etc.