Measuring design impact with the System Usability Scale

A defensible, repeatable measurement loop that quantifies the impact of every design decision using SUS benchmark scoring.

Client
Second Office
Role
Senior Product Designer
Timeline
2022

Alex is an entrepreneur in the USA, who has a vision of developing an iOS app for his ongoing project. He wants to hire a team from India as they are affordable and talented. Yet, he does not want to hire them as freelancers or on a contract basis. He wants to hire them as his employees and establish an office in India itself.

My Role — Market Researcher, User Interviews, UX | User Interface Design, MVP Audit Volunteer, SM manager

Team — Rajat Saini (Team Lead), Vishal Chandwani (Associate UX Designer), Sagar Mahato (Deputy UX Designer), Kaushal Batra (Graduate Design Trainee)

01 · Hypothesis

Problem Hypothesis' to Understand the User

😃
Key Metric
We conducted the first stage interview with the concerned HR workforce. We made a group of 8 HR individuals. And then we interviewed them upon a few open-ended statements about the end-user. And then we card-sorted their thoughts, ideas and suggestions as these 3 main pointers.

Problem 1 - TRUST

Here, trust is a key component; Alex won't give his money to untrustworthy market participants to hire on his behalf. Alex is naturally cautious, therefore he will require ongoing input from the recruiting authority. Alex will require a well-known market participant with enough publicly available information about them to be able to trust staffing agencies.

Problem 2 - LIABILITY OF HIRING

Alex does not want to go through the time consuming legal requirements to open an office in India because doing so takes a lot of effort and money. Management in India is a difficulty for him since even after recruiting, there is a chance that workers won't complete their responsibilities.

Problem 3 - RESOURCES

Alex doesn't have enough funds to spare for hiring an immense workforce in India, all alone. He does not want to assign the same work to his own HR staff. Alex finds it difficult to put resources in place for hiring in India.

02 · Audience

Target Audience

😃
Key Metric
We must first determine who will be our user and audience. Finding a good user is simple, but distinguishing who is not our user is difficult. These characteristics determine who our core audience is.
Target audience breakdown
Target audience breakdown
03 · Persona

User Persona from the Earlier Research

User Persona 1
User Persona 1
User Persona 2
User Persona 2

Persona Insights for User Frustrations

"Currency Variation – Our User is concerned about the continual change in currency."
"Keeping up with the recruiting process – Our user is upset with the lack of contact from HR, which is due to time zone differences, high communication costs, and tapping for every update."
Information Architecture
Information Architecture
04 · Solutions

Proposed Solution

1. For the Currency Variation issue

We recognized that currency adjustment would be a significant influence, thus we proposed adopting a standard conversion for the same. Once selected, the platform's currency will be set to the user's default currency.

Currency variation solution
Currency variation solution

2. For Keeping up with the Recruiting Process issue

For the second frustration, we decided to create a table that will indicate each candidate's progress. So, the true problem is to design an intuitive dashboard with tremendous ease to reduce any such discomfort.

Recruiting process dashboard
Recruiting process dashboard
05 · Testing

Conducting Usability Testing

😃
Key Metric
Usability testing assesses whether our audience can effectively accomplish activities on their own. We gave simple tasks to testers and watched their behaviour throughout usability testing. The user's first step was to post a job on SecondOffice.

Insights from the User Testing

"We gathered overall insights from our key stakeholders, Our Client and Human Resources. These are the individuals who will make use of our platform."

Key issue 1: During the testing, we saw that it took our user longer to compute the currency conversion. We identified this as a feature that should be included in our platform.

Key issue 2: We also discovered that the phrases "CTC" and "ECTC" made very little sense to users from demographics other than INDIA.

Key issue 3: Currency standardisation. We discovered that currency jargon changed from nation to country. For example, in INDIA, normal payroll is expressed as Salary * Lakh per annum, but in the US, it is salary * $10k per annum.

Key issue 4: It was also mentioned by a client that their organisation has a distinct hiring process for different profiles, thus they would want some control over the hiring process.

06 · Evaluation

S.U.S. Evaluation prior to re-design

"We asked a group of eight people to use our platform and complete the tasks listed below."
🛠
Key Metric
Inference: Since the overall SUS score is less than 60, i.e. 51.87 - the product needs to be seriously re-considered in terms of easability, accesibility and clarity.
SUS analysis prior to re-designs
SUS analysis prior to re-designs
07 · Design Iterations

‘Before and After’ Designs for specific problems

Redesign 1

CTC and ECTC are industrial buzzwords used in INDIA that are widely known in the demography. In other nations, however, this was not the case. As a result, we included an information indicator icon to assist the user in determining what the jargon meant.

Information indicator icon
Information indicator icon
Hover state for definition
Hover state for definition

Redesign 2

Another concern raised during user testing was the standardisation of currencies in compensation. We discovered that the salary chips provided above were not assisting users since they were not standard in their currency.

A 0-3 LPA chip is OK for INDIA, but when translated to USD, it becomes 0-3.7K, which is horrible because the baseline in the US should be 0-10K. We turned them into an input field, which solved our issue of standardising the salary fields.

Before: Salary chips
Before: Salary chips
After: Standardized input fields
After: Standardized input fields

Redesign 3

Because there was no filtration of dates in this design, it was difficult to examine the assessment of an employee for any given date. Instead, we used to evaluate the person directly. Following the adoption of the new design, we are now arranging them based on dates, which also sorts the ratings for each employee.

Before: No date filtration
Before: No date filtration
After: Arranging based on dates
After: Arranging based on dates

Redesign 4

We also observed that consumers require a preliminary platform explanation for a better understanding of the flow. Users were also rushing to hire rather than filling out the firm details, which might jeopardise the organization's reliability. We made one time onboarding overlays which helped users to understand the basic information architecture of the platform.

Before: No onboarding
Before: No onboarding
After: Onboarding overlays
After: Onboarding overlays

Redesign 5

We agreed that users should be prompted to finish their profiles before proceeding with job posting, thus we established a progress bar to assist users in traversing to complete their profiles.

Progress bar step 1
Progress bar step 1
Progress bar step 2
Progress bar step 2
Progress bar step 3
Progress bar step 3
08 · Evaluation Post-Redesign

S.U.S Evaluation after the redesigns

"Once again, we asked a different group of eight people, to use our platform and complete the tasks listed below. It is important to continually keep the stakeholders and the end-users in loop with the modifications."
🛠
Key Metric
Inference: Gladly, the overall end SUS score is more than 60, i.e. 72.57 - the product is now ready to get to use. The success rate of tasks has significantly increased. Furthermore, when we compared the former score (51.87) to the improved value (72.57), we saw a 20.70 percent rise in the System Usability Score.
SUS evaluation of the product after re-designs
SUS evaluation of the product after re-designs
09 · Conclusion

Closure

In the early stage of this product, from ideas to concepts to hand-offs, we as designers put our best to involve and adapt the best practices for the best results. And, looking upon the results, the efforts felt worth it. Based on user feedback, we have chosen to add several features in the future. We will also create a store page to assist our customer in obtaining actual assets required by their personnel.

The constant process of testing, finding flaws, and correcting them will continue throughout the product cycle with this lean UX design process.

00 · Intro
Scroll normally to explore
Skip to main content