January 29th 2024–March 8th 2025
Team
Diego Reyes-Alicea, Hannah B., Selene P., Tabitha M.
ROLE
Usability testing, qualitative analysis, test facilitator, testing evaluator, test observer, UI design
Project Overview
This project covers my experience conducting a usability test, in a team setting, for the DeKalb County Georgia government website.
The Problem
The DeKalb County website was identified as a poorly performing website via a Heuristic Evaluation.
The Solution
In oder to address the discovered issues with the DeKalb Count website. I provided the most important changes that will dramatically improve the website's performance.
70%
Average failure of all 3 tasks
2.2/5
Represents the website's low confidence score reflecting poor usability
Testing Methodology
We conducted scenario-based observations on how DeKalb County residents interacted with the website to find services. We used a summative testing method, which evaluates the quality and performance of the site. We then analyzed the data using system usability scale scoring along with a after scenario questionnaire to support reasoning for design improvements.
Scenario-based Observation
Summative Assessment
Data Analysis
Design Improvement
Test Details
Rotated Roles For Each Test
Facilitator
Observer
Evaluator
Each participant was required to the following tasks to the best of their ability following a think aloud protocol:
Task 1: Request recycling roll cart online
Task 2: Find your city commissioner’s email address
Task 3: Search for Web Designer remote jobs through WorkSource DeKalb
Evaluation Methods
Quantitative Metrics
Success Rate
Can the participants perform the task and successfully complete it through the happy path?
Task Completed with Assistance
How many times did the participant ask for help?
Task Time
How long did it take for users to complete the task?
Error Rate
How many errors were made by the user?
Subjective Metrics
After-scenario Questionnaire (ASQ)
System Usability Scale (SUS)
Usability Test Results
Task 1: Request recycling roll cart online
Task Metrics
Avg. Task Time: 2:10min
Success Rate: 0%
Error Rate: 73%
ASQ Results
Ease of Completion: 4.0/7
Time Satisfaction: 5.8/7
Support Satisfaction: 4.2/7
Task 2: Find your city commissioner's email address
Task Metrics
Avg. Task Time: 3:30min
Success Rate: 20%
Error Rate: 72%
ASQ Results
Ease of Completion: 3.2/7
Time Satisfaction: 4.8/7
Support Satisfaction: 3.8/7
Task 3: Search for web designer remote jobs through WorkSource DeKalb
Task Metrics
Avg. Task Time: 3:24min
Success Rate: 60%
Error Rate: 70%
ASQ Results
Ease of Completion: 4.6/7
Time Satisfaction: 6.0/7
Support Satisfaction: 6.4/7
System Usability Scale Grades
Participant
SUS Score
Letter Grade
1
86
A
2
27.5
F
3
57.5
F
4
92.5
A
5
60
D
Avg. SUS Score: 66
This score average indicates marginal usability, the industry standard is 68.
Improvement Suggestions
We scaled importance of changes on a scale from 1 (low importance) to 5 (high importance). Here are the most important changes from ranges 4–5:
Level 5
Redesign navigation bar to improve legibility and organization.
Ensure the website is responsive.
Level 4
Include a summary at the top of the page in order to describe its contents.
Revisit the website’s design system to create a more consistent experience.
Improvements Mockup
#1
#2
Improvements to navigation to help users with readability.
#3
Updated content organization to support users' search.
Conclusion
The website exhibited significant usability challenges, as evidenced by high error rates and low task completion success rates across all tasks. Participants encountered obstacles such as unclear labeling, ineffective navigation paths, and reliance on the search bar, leading to frustration and difficulty in achieving their goals.
