Practical tools for measuring websites and identifying flaws

For many people, the premise that an interface should be tested and undergo comprehensive research is no longer surprising. Larger, more conscious organisations (such as OTP eBiz) are starting to supplement the development process with strings of interviews (including interviews to assess mental models)

Research and testing is recommended throughout the entire development process; situations where we want to perform aspecific measurement on an existing interfacecan also arise afterdevelopment has been completed because either we perceive a hiccup or our users tell us about it. Alternatively,we may find a flaw with the interface that could have beenremedied mucheasierand at alower costif we had noticed it long ago,at an earlier design and development stage.

We have compiled a list of some of thefree and low-cost solutions available onlineto test interfaces and collect related user feedback. There is an endless choice of solutions, so we will only look at a few and adopt a practical approach to the matter.

As Jacob Nielsen, the pioneer of UX design said, the greatest challenge when designing certain applications, websites, etc. is thestructuringandnamingof information. There are various applications for structuring the contents found on an interface. Pioneers of the field developed multiple methods to solve these problems. We are introducing some of these below:

Card sorting is an excellent, tried and tested method for groupinginformation found on an interfaceand categorizing the groups.During this process, the moderator asks participants to structure and group the planned contents of the website in a way that is logical for them. The different groups can be named and renamed. Card sorting offers insight onuser opinions about the contents of our websiteand also on thedistribution of the planned information that is considered logical by users.Sorting can be open or closed. In the former case, users sort the contents themselves and form and name the groups. In the case of closed sorting, members of the focus group sort content elements in predefined categories.

This method offers the advantage of helping tocreate and organise website structureand alsosorting the contents and naming the categories.Because the contents are created by the users themselves, the structure of the future website will presumably be logical for them.

Sorting can be performed in person and in groups or one-on-one interviews. In these cases, we can alsocollect more in-depth qualitative data and investigate the reasons.

If we arefocusing on quantity and quantitative data instead, in other words want to work on a larger sample, it is recommended to opt for an online solution that is easy to understand and access for users. In these scenarios, we use theOptilmalSort software.

Optimal sort
This allows us to test whether the number and name of menu items is actually optimal for users, not only us.

Benefits:

• If used online, ithas the greatest reach(hundreds or thousands of users). It can also be used off-line on a smaller sample and in this scenario, give us qualitative data.

Drawbacks:

• When opting for online data collection,we do not know the reasons.It is best to combine the two methods and to back-test something that we have discussed on a smaller sample on a larger sample.

Pro use #1:

• We asked participants to group contentsunder the menu items that we defined.

Pro use #2:

• We asked participants togroup contents and to name the groups.We also discussed with them theirreasonsto gain insight about their thought process.

Beside this way of content categorization, we can earn great deal of knowledge from the analysis of web page traffic. In this field there are a wide variety of tools to be applied as well. With the help of these tools, we can learn about the source un-exhaustively, which is, what was the kind of the source page from where our users arrived to our website. This is especially important from marketing and SEO perspectives because we can learn where it is worth to place more ads.

Moreover, we can learn the ratio among platforms on which our website has been visited. That can be particularly useful, if we need to choose which platform is more urgent to develop. In addition we can also analyse the frequency of clicking on a given point and the covered road our users completed throughout their visit on our webpage. Another great tool is the automatically generated heatmap that can show us the most prominent and the least visited areas on the screen. Simple screen recordings, however, provide us valuable information about the pain points of our users.

Optimalsort - source: https://support.optimalworkshop.com/hc/en-us/articles/202415424-Interpret-the-OptimalSort-Participant-Centric-Analysis-PCA-

Google Analytics
Google Analytics is the most widely known and most often used analytical solutionthat offers much more than merely measuring user traffic, its most popular usage. Analytics can be used for more complex measurements and analyses besides measuring basic traffic data (pageviews, bounce rate, session duration, etc.). For example it can be used to measure how many people clicked on a specific button, making it quite flexible and practical, rightfully giving it the status ofthe Swiss knife of web measurement.

Pro use #1:

• Set a custom event and you will see it together with thepage views.

Pro use #2:

• Success on the web is also measured by money. Analytics is just as apt at calculatingrevenue per useras the number of visitors.

Benefits:

• Varied and diverse capabilities
• Universally used, hence data is comparable
• Free for small to medium traffic sites
• Any (javascript) event can be flexibly measured
• Backed by a relatively safe parent company

Drawbacks:

• Very costly for high traffic websites (over 1 million daily views)
• Data is only available with a one day lag
• Data is not stored in encrypted form

Pro use #1:

• Set a custom event and you will see it together with thepage views.

Pro use #2:

• Success on the web is also measured by money. Analytics is just as apt at calculatingrevenue per useras the number of visitors.

Google analytics - source: https://neilpatel.com/blog/how-to-get-actionable-data-from-google-analytics-in-10-minutes/

Hotjar
Hotjaris a simple tool for measuring user behaviour.Itshows data graphically in a more visual mannercompared to Google Analytics, transparently showing us how users use the website.

Although the website is known for its excellent heatmaps, the greatest advantage is to essentially see user sessions captured on video. Every UX test should begin by watching videos of randomly chosen users navigate the website for an hour or two.Flaws, shortfalls and misleading elements can thus be easily identified.Traditional heatmaps follow the user’s mouse movements and clicks to allows us toeasily visualisehow users navigate the website.

The site also provides tools for surveying and involving users. These are simple tools and not nearly as well-equipped as a laboratory-based solution, but may nevertheless be useful for addressing issues from time to time.Needless to say, eye tracking tests cannot be performed, but they are actually not necessary in many cases because clicks reflect actual use. So this tool cansupplement often costly laboratory measurements.

Benefits:

• Heatmaps are easy to use and understand.
• Video recording of the user's work session, which is essentially equivalent to observing the monitor from the perspective of the user's head.
• Freefor lower traffic websites (2000 page views/day).
• Excellent mobile support.

Drawbacks:

• A simple tool, howeverit does not allow more in-depth and complex research.
• Only available in English, although theinterfaces accessible to users can be completely tailored to the website, including the text.
• Costlyfor larger pages,so it is recommended to work based on a smaller sample.

Pro use #1:

• Watching user’s video, browsing the website and closely observingwhere they reach obstacles.

Pro use #2:

• Removing all unused elements from the websitebased on the heatmap.

Possible alternatives:

• Clicktale
• Decibel Insight
• Mouseflow
• Zarget

Screen capturing software
It is a frequently recurring need tomake user tests screen recordings as it is useful to watch them lateron, since we want to analyse them in details, which is discovering newdrop-offsor just making sure that our user’s focus is directed to where we intended it.For these purpose we mostly prefer to use the tools below:

Skype
If one wants to research something remotely (rather than in person), the screen sharing function of chat applications (Hangouts, Skype) can recreatefull mini laboratory testsfor just pennies.

We prefer to use Skype for this function because almost everyone has it on their computer. This makes things easier for both the client and the test subjects. It also has the advantage of allowing testers to work on their own computer, in their own environment so they don’t have to struggle with getting acquainted with the mouse or mouse pad during testing. In their own environment, users will navigate more freely, as they would normally do; meanwhile we can observe actual use.

It also offers an advantage in terms ofresearchorganisationlogistics, allowing us to work more easily with users in rural areas or even villages who are thus spared from having to travel - only a good Internet connection is needed.

Benefits:

• Users can stay in their natural environment which is naturally decreasing their stress level and prevents any discomforts they may feel in a lab setting. Moreover, they don’t have to adopt to a new situation, she can use her own devices that she knows the best.

Drawbacks:

• In case of smaller bandwidth the recording’s quality is poor.

Pro use #1:

• Once the main issues with the website have been identified, it is recommended to test the solution plan.We can publish the website immediately if we want to,but it is recommended to sit down.

Pro use #2:

• We can record video on the user's own device. This yieldsmore accurate resultsbut often comes up against logistic hurdles.

Lookback
Lookbackis a popular solution to get toknow how members of our target group (even within our own environment) use the given interface.With this tool, we can follow the “use” process of a given website or application. Lookback allows us to gain insight onhow the interface- application is used in the users’ natural environment.In other words, designers canget to knowexactlyhow members of the target group use the product.

The app is available for both iOS and Android and we often use it to test various apps under development as well as existing websites.After downloading the app (here for example:), we need to register and can send a link to the tested interface to testers. If they click on the link, theapp will record everything that happens on the screen plus any external noises.The tester can also testaloneat a chosen time, andlive testing and broadcasting (live mode) is also possible.

On the recording, the software also indicateswhat the user has just tapped/clicked.The recordings can be uploaded to the Lookback website and can be viewed subsequently, and we can alsoperform administration of the events as needed.It can be used to test prototypes, apps and websites on both mobile (iOS, Android) and desktop.

We can glean practical information by collectively looking at the recordingsand can then contact our testers forsome qualitative samplingwhen discussing theirexperiences and positive and negative impressions.This can be done either individually or in a group.

Benefits:

• Simple use on every possible platform (iOS, Android, desktop), broad testing opportunities for prototypes, completed websites/apps or websites/apps under development. Total compatibility across the board.

Drawbacks:

• During use, the recording stops when the mobile phone’s screen shuts off if the tester was not navigating the interface while discussing something. To prevent this, we recommend turning the screensaver off. It also runs on certain Android devices

Pro use #1:

• Performing data analyses based on what we have seen, highlighting key moments.

Pro use #2:

• After analysing the recordings, contact the test subject and discuss their experiences; i.e. supplement the survey with qualitative data collection.

Lookback

Silverback
Silverback also records activities performed on the screen. The application records audio and also records the tester in the bottom right-hand corner, minor details of the tester’s reactions and nonverbal communication that can be captured on camera. The software indicates mouse movements and clicks on the recording. Silverback can be used to conduct complete research on large samples.

Benefits:

• Important parts in the recordings can be highlighted and these can be exported  separately.

Drawbacks:

• QuickTime Player is capable of screen and click recording as well, thus, if you do not need the highlighting function specifically, you can use it without downloading anything else.

Pro use #1:

• Free use with a demo version (this generates recordings featuring a watermark).

Pro use #2:

• Paid version with no watermarks. On the video extracts,we can highlight the most important events for our research.

Silverback - source: http://www.stakelon.com/2014/01/building-a-discount-user-testing-lab-part-1//

Collecting user opinions

If you want to identify the potential flaws of your website,show it to users.Generally speaking, do not be afraid to inquire or ask. Anyone in our surroundingswho is not involved in the developmentcan be a wonderful and useful helper. A co-worker, partner, child or any other user is capable of offering new perspective. In this case, we are not speaking strictly of testingbut of the more general and fundamental aspects of the interface.

However, if you choose not to involve the people surrounding you in the process, you can opt to use a service. There are several services offering help by asking simple questions from users who are unknown to us but can offer assistance in testing. https://www.surveymonkey.com/, https://usabilityhub.com/, https://www.uservoice.com)

Benefits:

• A great amount of data can be collected very quickly and with the help of online surveys creating programs, data analysis and visualization are easily done.

Drawbacks:

• It is very easy to make methodological errors. It is worth to familiarize with the subject in advance and study methodological books (please see suggested literature below).

Pro use #1:

• Ask for user's attentionfor a short time to answer a few simple questions.This can be in the form ofquick guerrilla testing or a short interview.We do not want to get answers to big questions, but rather many small ones. Sample questions: “Where do you think registration is on this website?” “Where would you start process XY?”

Pro use #2:

• We can not only investigate simple issues, but alsoquite complex ones regarding the interface.In these scenarios, data collection can be more in-depth and complex. We can ask the questionsfollowing a strict, fixed script or a more relaxed one.The latter may be warranted if we want to drill deep on certain topics. This of course requires a greater number of testers.However general experience shows thatinterface testing with 5-8 usersoften identifies more than 80% of major UX flaws.

We can perform better if we read the literature on testing and practice regularly. For starters, refer to Steve Krug’s book, Don't Make Me Think or or Jakob Nielsen’s piece.

More reading:

• Jacob Nielsen: Why You Only Need to Test with 5 Users
• Steve Portigal: Interviewing users
• Russ Unger – Carolyn Chandler: A Project guide to UX design
• Earl Babbie: The practice of social research, Cengage Learning Asia Pte Ltd Singapore, 2016.