Astroturystyka
- Eyetracking usability research

Usability test performed as a theoretical part of my master degree diploma project. As my thesis subject was "Eyetracking methodologies in a web design usability testing", I managed the eyetracking research project testing a hi-fi wireframe of the website I designed for Astroturystyka - the brand of a public benefit organization - Polaris OPP.

Introduction

Main Astroturystyka's goal is to activate society and popularise the astronomy by organising workshops and various events. The most important functionality of its website is an online booking system for the participants. To provide the best service, the booking system should be easily visible for its users.

For that purpose, I decided to test two different navigation solutions (a hamburger menu and a sidebar navigation) and two different content layouts on a homepage (content&call-to-actions displayed in cards and a classical layout with head image on a slider with a few most important CTAs). I prepared four mixed versions of the website to evaluate and analyze the usability of these solutions.
1. Version with the sidebar navigation and the classical content layout on the homepage.
2. Version with the sidebar navigation and the cards layout.
3. Version with the hamburger menu and the classical content layout on the homepage.
4. Version with the hamburger menu and the cards layout.
version 1version 1
version 2version 2
version 3version 3
version 4version 4

Research hypotheses

During the design process, I stated assumptions that lead me to form two research hypotheses:

1. User interaction might be more effecient with the sidebar navigation. The use of the so-called "hamburger menu" may makes users confused and less likely to reach the navigation.

2. User interactions might be also more efficient when designing pages with a hierarchical arrangement of elements to allow the user to move their eyes according to a certain "path".

Task scenario

All of the subjects were asked to make a reservation on the "Constructor's workshops" on 20th April, 6:30 pm using the website's booking system.

Test subjects

The research was carried out at the Pedagogical University of Cracow with a group of 40 participants. They were divided into four seperate subgroups of 10 users assigned to each website version. All of the participants were students ranging in age from 20 to 30 years old. Additional questionnaire shown that they have good computer skills and rarely have problems with finding informations on the Internet.

Results and analysis in a shortcut

AOIs for each version's homepage, overall:
Version 1:

As for the first version, 9 of 10 users moved from the homepage to the next subpage using sidebar menu.

1 of 10 people used the "Go to the booking" button at the bottom of the homepage. The sidebar menu was also area with the highest number of fixations.
Version 2:

The results for the second version of the project showed a similar trend. In this version, side navigation was also distinguished by the highest number of clicks and eye fixation of respondents.
Version 3:

The third version contains a different distribution of clicks among users. For all of 12 clicks of 10 users group, the hamburger menu have been clicked only 4 times. "Go to the booking" button was clicked 5 times and the button "All offers" located on the slider was clicked 3 times. These differences also reflect fixation times. The "Go to the booking" button at the bottom of the page was the most eye-catching in this version. The next eye-catching element was the "All offers" button and the header "News". The menu button was therefore only the fourth area, counting by fixations time.
Version 4:

Less variety of clicked places could be seen on the fourth version. There were only 3 clicks outside the menu icon, on the "All offers" button located on the right side of the graphic layout. All of the remaining seven clicks were on the hamburger menu icon. This icon was the area with the highest number of fixations. In this version of the project there were also a large variety of fixation areas, which distinguishes this version from the second version, with the same cards layout on the homepage. Also the number of saccades in the fourth version was much higher than in other projects. For the majority of the respondents, saccades crossed over to the whole page, noticeably jumping in an irregular manner across all elements on the website.

Interpretations

Testing four versions of the website, which are combinations of all the prepared design solutions, gave surprising and interesting perspective for an analysis, because on each of the versions it was possible to notice different tendencies among users' behaviour. As it turned out, each solution made an impact on the perceiving the other one.

1. For both first and second version, the vast majority of eyesight fixation were focused on the menu and it was the element that started interactions and transitions to the next subpages. In these two versions it is therefore difficult to talk about the advantage of the cards layout over the content with slider or vice versa.

The next two versions of the page are pages with the hidden ("hamburger") navigation. These versions had a much greater number of clicks and fixations on areas outside the navigation. The analysis of this phenomenon allows concluding that some users did not notice the menu icon at all, or this element did not draw their attention so much, so they were somehow forced to find an information on the rest of the page layout. The third version of the website, had 6 of 10 users who did not look at the menu icon at all. These users went to the next subpage using buttons: "All offers" or "Go to the booking". It can be assumed that some mistakes have been made in designing or locating the navigation icon, so it could be said that this element was very unlikely to notice or to recognize. However, as for comparison, the fourth version had only two users who did not notice the "hamburger" symbol at all. All of the remaining subjects reached the navigation during their interactions.

By analyzing the above phenomena observed in the research, the hypothesis referring to the superiority of side navigation over hidden navigation can be confirmed.

2. To verify the second hypothesis, it was crucial to analyze the above mentioned differences in the number of clicks on the menu icon and fixations between the results for the third and the fourth version of the website. Comparing the scan paths in the results of the third and the fourth version, there were noticable differences in the length and frequency of the saccades. Scan paths in the third version were much longer, there were much less saccades then on the fourth version. On the third version, users turned their gaze from the text to the next striking element. Such behaviour led directly to clicks on the one of two gold-coloured buttons "All offers" or "Go to the booking". This type of long and uncomplicated saccadic eye movements also occured among those 5 respondents who had used the hamburger menu. This was a completely different situation than in the case of users who had used the menu in the fourth version. In this version, the respondents who had clicked on the navigation symbol were using the website creating eye paths in a much more chaotic way. They looked at the buttons referring to subpages with offers or bookings, but did not make clicks in these areas. The number of saccades was much higher and crossed over practically the whole website's layout. This situation may suggest that, as a result of excessive amount of elements, some kind of information noise has occurred and made users' perception overloaded. This has made it difficult for them to see the possibility of going to an important (for the task scenario) subpage. This phenomenon may also be the reason why respondents in the fourth version visited the menu more often. It can be assumed that the need to find the navigation became a kind of an emergency wheel in a situation where no other element initiated the will to click. A remark referring to too much information on the homepage was also present in the survey in the answer of the 8 of 10 respondents in the fourth group and was defined as a factor causing difficulty in performing the task.

The above observations allowed to confirm the second hypothesis concerning the impact of the simplicity of the homepage layout on the website usability. As it turned out, in the case where users did not have an access to clear and a visible side navigation, the factor that made it easier to navigate through the site was to provide a graphic layout that would make relevant elements simple to notice among the whole content.

Sum up & what I have learned

Eyetracking testing of my own project was an exciting and enlightening experience. It broadened my knowledge and way of perceiving the role of research in the design in general.

I would conclude some of my thoughts as follow:
- Eyetracking as a method of usability testing gives a great oportunity to arrange an interview with the user at the same time. Interactions that are live commented by the user provide a lot of informations which could be transcripted and analyzed by the researcher. Due to some specific restraints related to the process of writing my diploma, I could not broaden my methodology and include these comments in the thesis after realising these possibilities. However, this is the thing I would strongly recommend to every student who would like to perform a similiar eyetracking research.
- According to the research results, in case of the specific task-oriented kind of interactions, sidebar navigation always may be the good first option to consider to provide the best user experience.
- Informaction architecture plays a huge role in a proccess of designing such a complex service. It is important to define the most important functionalities and the most crucial content to state what should be presented on a homepage. It might be a good practice to think about the possible visual paths during wireframing the layout so the users could not be distracted by a great number of different buttons and various elements and could easily move their eyes according to a certain arrangement. However, I had a lot of doubts about which informations exactly should be visible on the homepage. Should it be really "News feed" as usual? Maybe "Contant us" CTA button could be more important for users? These questions could be answered only after a well proccessed research in the beginning. Maybe even the website is not so obvious solution to provide a good quality service design and to answer the users' needs ;-)
GO UP ↑