Last year, I facilitated my first usability test. It was part of a college assignment where I had to choose a product to test, conduct a usability study, and then report on the findings.
It was a tough assignment, but I loved the experience, and it was during the literature review that I found the System Usability Scale (SUS), a survey tool that is famous among usability professionals in both the academic and business world.
What is the System Usability Scale (SUS)?
The SUS is a survey that consists of ten Likert-scale questions that are used to measure the subjective usability of products and services. John Brooke initially designed the SUS in 1986, and since that time, it has grown to become the most widely used survey for measuring perceptions of usability (Sauro 2011).
Figure 1 presents the SUS in its original format:
The score of the SUS is measured on a scale of 0 to 100, and although calculating the score is tricky at first, the math gets easier with practice.
To calculate the score of the SUS:
- Subtract the number 1 from the answer given in questions 1, 3, 5, 7, and 9.
- Subtract the answer given for questions 2, 4, 6, 8 and 10 from the number 5.
- Sum the score from each answer.
- Multiply the sum by 2.5 to get the SUS score.
As a general rule, a product that scores 68 is above average, and any score below 68 is below average (Sauro 2011). For a more precise reading of the results, you can convert the raw SUS score into a Percentile Rank or Letter Grade by using the table in Figure 2.
A Brief History of the SUS
Brooke (2013) originally developed the SUS while he was working for the UK division of the Digital Equipment Corporation (DEC) during the early 1980s. At that time, Brooke was employed as a human factors researcher and was testing the usability of DEC’s ALL-IN-1 office automation system. Due to time constraints surrounding the availability of test participants, he and his team needed a way to collect subjective usability data quickly and effectively. As a result of this demand, Brooke and his team at DEC deliberated over a pool of fifty questionnaire statements, consulted with multiple I.T. professionals and produced the ten questions of the System Usability Scale.
As the SUS proved to be successful in early testing, Brooke, in agreement with DEC, made the test freely available to his associates so that they could compare usability data from multiple systems. Owing to its effectiveness, and because it was free to use, the SUS slowly grew in popularity. In his essay SUS: A Retrospective, Brooke (2013 p.29) writes that the survey has been cited in over 1,200 academic papers and has been used in numerous unpublished usability evaluations.
Key Features of the System Usability Scale (SUS)
The following features have made the SUS popular among usability researchers for the past 30 years.
It’s Free to Use
As mentioned earlier, in 1986 Brooke made the test freely available to any researcher that wanted to use it, and it is still free to use today.
It’s Reliable
Brooke (2013) quotes numerous studies that have proven that the SUS is a reliable and valid measure of usability. Furthermore, Sauro (2011) states that the SUS has a Cronbach Alpha score of .92, making it more reliable than many commercial tests.
It’s Versatile
As the questions in the SUS focus on general usability, the survey can be used to evaluate a wide variety of products and services. In most usability scenarios the SUS can be used ‘off the shelf’ as the researcher won’t need to alter the questions.
It’s Quick and Easy to Use
The SUS has only ten items so that test participants can complete the survey quickly. Additionally, the questions are written clearly and are easy for the reader to understand.
Personal Experience of the SUS
I first learned about the SUS while reading a paper published by Beul-Leusmann et al. (2014). For my usability study, I examined a public transport Information app aimed at Irish customers. I was attracted to the Beul-Leusmann et al. study because the authors reported on how the perceived quality of an apps’ user interface can affect the perceived quality of the apps’ performance overall. In their study, the researchers made a prototype public transport information app, and they used the SUS to measure its usability. The results of their research showed that users responded positively to the app they had designed because it offered a high quality of usability, and because of this data, I adopted the SUS into my usability test to see if it would produce similar findings.
My study included a task-driven think-aloud usability test along with interview questions and the SUS survey. Overall, my usability study found that the app I was testing had various issues which negatively influenced its perceived quality. The SUS survey was very insightful to the total test data because the app received a low score on the SUS scale. This data supported my findings and gave me the confidence to categorically state that the app I was testing had some severe usability issues.
Benefits and Challenges of the SUS
The most significant benefit of the SUS is that it is a highly reliable survey that is quick, easy, and free to use. It is challenging to tally the score, especially when you’re as nervous of math as I am, but once you practice the process it becomes straightforward to calculate. The only identifiable fault with the SUS is that it does not diagnose specific usability problems. Although it does provide you with data about how the participants perceived the usability of the product, it won’t tell you what is not usable, or what features need to be fixed. However, this is a minimal payoff for what is a genuinely insightful usability survey.
References
Beul-Leusmann, S. Samsel, C. Wiederhold, M. Krempels, K. Jakobs, E. Ziefle, M. (2014) ‘Usability Evaluation of Mobile Passenger Information Systems’. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 8517(1), 217-228, available: doi:10.1007/978-3-319-07668-3_22.
Brooke, J. (1996), SUS – A quick and Dirty Usability Scale, available: https://hell.meiert.org/core/pdf/sus.pdf
Brooke, J (2013) SUS: A retrospective, Journal of Usability Studies, 8(2), 29-40, available http://uxpajournal.org/sus-a-retrospective/ [accessed 11 February 2019].
Sauro, J. (2011). SUStisfied? Little-Known System Usability Scale Facts. User Experience Magazine, 10(3), available: http://uxpamagazine.org/sustified/ [accessed 11 February 2019].
Featured Image
Photo by David Travis on Unsplash
Michael’s Reflective Learning Blog 