Usability Test Report: DeKalb County Website
Timeline: January 29th 2024–March 8th 2025
Team: Diego Reyes-Alicea, Hannah Blum, Selene Padilla, Tabitha Milon
Role: Usability testing, qualitative analysis, testing facilitator, testing evaluator, testing observer, UI design
Timeline: January 29th 2024–March 8th 2025
Team: Diego Reyes-Alicea, Hannah Blum, Selene Padilla, Tabitha Milon
Role: Usability testing, qualitative analysis, testing facilitator, testing evaluator, testing observer, UI design
Usability Test Report: DeKalb County Website
Diego J. Reyes-Alicea
Project Overview
In this study, we observed how DeKalb County residents interacted with the website to find services. We used a summative testing method, which evaluates the quality and performance of the site. We gave users tasks to perform while capturing both quantitative and subjective data. With these methods, we discovered what tools worked within the site and the challenges residents encountered.
Method
In this study, we observed how DeKalb County residents interacted with the website to find services. We used a summative testing method, which evaluates the quality and performance of the site. We gave users tasks to perform while capturing both quantitative and subjective data. With these methods, we discovered what tools worked within the site and the challenges residents encountered.
Scenario based observation
Summative assessment
Data analysis
Improvement suggestions
Usability Testing
We screened participants to ensure that they were residents or landlords of DeKalb County Georgia, and at least 18 years old. We observed five residents who ranged from 22 to 44. Participants who were not residents of DeKalb County Georgia, not over the age of 18, and didn’t have access to the internet or a computer were excluded.
We used a remote moderated approach for the test sessions. In each session, a team member took one of the three roles; test facilitator, evaluator, or observer. Each session lasted between 30–40 minutes.
Evaluation Methods
Quantitative metrics were used to evaluate the participants’ performance and subjective metrics were used to gather insights about how users felt about the website. The data for these metrics were collected by the evaluator and observer using log sheets and questionnaires.
Quantitative Metrics
Success Rate
Can the participants perform the task and successfully complete it through the happy path?
Task Completed with Assistance
How many times did the participant ask for help?
Task Time
How long did it take for users to complete the task?
Error Rate
How many errors were made by the user?
Subjective Metrics
After-scenario Questionnaire (ASQ)
The purpose of this questionnaire was to gather information about the participants’ experience and identify areas for improvement.
System Usability Scale (SUS)
Helped us understand the participants’ entire experience with the site. The All Positive version was utilized. We used Bangor et al. (2009) method to interpret the SUS scores.
Task 1 Findings
Scenario: Request recycling roll cart online
No participant was able to complete the task. Participants failed due to incorrect navigation or selecting the wrong link (example: email request vs. online form). The task relied on users noticing subtle distinctions that weren’t intuitive.
Task Time: 2:10min
Despite moderate time spent, none succeeded–indicating users quickly reached dead ends or confusion.
Success Rate
0%
Failures
5
2/5 Gave Up, 3/5 Got it Wrong
Error Rate
73%
After-Scenario Questionnaire Results
Users felt moderately satisfied with the time it took but the ease and support was unsatisfactory, aligning with high error rates and abandonment.
Q1
Ease of Completion
4.0
Q2
Time Satisfaction
5.8
Q3
Support Satisfaction
4.2
Recommendations
Clarify labels for online vs. in-person.
Reduce alternate versions of links/buttons.
Add visual elements to highlight the correct path.
Task 2 Findings
Scenario: Find your city commissioner’s email address
Low task success suggests structural and labeling issues around government contact information. Users misunderstood who they were suppose to contact based on UI.
Task Time: 3:30min
High time average highlights inconsistent user experience, from confusion to persistence, with little payoff.
Success Rate
20%
Failures
4
3/5 Gave Up, 1/5 Got it Wrong
Error Rate
72%
After-Scenario Questionnaire Results
Users felt this was the most difficult task. They struggled to complete it and found the support information inadequate.
Q1
Ease of Completion
3.2
Q2
Time Satisfaction
4.8
Q3
Support Satisfaction
3.8
Recommendations
Clarify labels: “City Commission” vs. “County Commissioner”.
Improve search accuracy and results with the use of a search filter.
Ensure full navigation bar is accessible on all screens.
Task 3 Findings
Scenario: Search for Web Designer remote jobs through WorkSource DeKalb
Task 3 was the most successful, though some users veered off path or ended up on the unrelated job bards.
Task Time: 3:24min
Some completed the task, others were lost. Variability in navigational effectiveness and comprehension.
Success Rate
60%
Failures
3
2/5 Didn't complete the task
Error Rate
70%
After-Scenario Questionnaire Results
Users were most satisfied with this task overall, especially with support information. It indicates this path was better structured with containing clearer labels.
Q1
Ease of Completion
4.6
Q2
Time Satisfaction
6.0
Q3
Support Satisfaction
6.4
Recommendations
Improve visibility of the correct job board.
Reduce external job board distractions and external links.
Make the: Government > Workforce > Jobseekers > Openings, path more prominent.
System Usability Scale Grades
92.5% (A), 87.5% (A)
60% (D), 57.6% (F), 27.5% (F)
Improvement Suggestions
We scaled importance of changes on a scale from 1 (low importance) to 5 (high importance). Here are the most important changes from ranges 3–5:
Level 5
Improve the organizational structure of the website, most common issue in was the user’s inability to locate an item in the navigation bar.
Ensure the website is responsive to accommodate all screen sizes, from laptops and tablets to mobile devices.
Level 4
Make content clear and visible on every page. A summary at the top of the page can help users decide quickly if they are in the right location.
Revisit the website’s design system to create a more consistent experience.
Level 3
Conduct a content inventory of the website to ensure that all links lead to the correct/working pages.
Limit the amount of externally linked pages to decrease confusion for the user.
Improvements Mock Up
Scroll left in the gray area to view screens.
#1
Clear button tiles that link to the proper
working page mirroring the design system.
#2
Improvements to navigation to help with
users with readability.
#3
Updated content organization to support users search.
(1/2) Implemented Changes
#1
#2
#3
(2/2) Original Website
Conclusion
The website exhibited significant usability challenges, as evidenced by high error rates and low task completion success rates across all tasks. Participants encountered obstacles such as unclear labeling, ineffective navigation paths, and reliance on the search bar, leading to frustration and difficulty in achieving their goals.
Interested in collaborating? Let’s connect!
copyright Diego J. Reyes-Alicea | designed by Diego J. Reyes-Alicea | updated July 2025
Interested in collaborating? Let’s connect!
copyright Diego J. Reyes-Alicea | designed by Diego J. Reyes-Alicea
updated July 2025
Project Overview
In this study, we observed how DeKalb County residents interacted with the website to find services. We used a summative testing method, which evaluates the quality and performance of the site. We gave users tasks to perform while capturing both quantitative and subjective data. With these methods, we discovered what tools worked within the site and the challenges residents encountered.
Method
In this study, we observed how DeKalb County residents interacted with the website to find services. We used a summative testing method, which evaluates the quality and performance of the site. We gave users tasks to perform while capturing both quantitative and subjective data. With these methods, we discovered what tools worked within the site and the challenges residents encountered.
Scenario based observation
Summative assessment
Data analysis
Improvement suggestions
Usability Testing
We screened participants to ensure that they were residents or landlords of DeKalb County Georgia, and at least 18 years old. We observed five residents who ranged from 22 to 44. Participants who were not residents of DeKalb County Georgia, not over the age of 18, and didn’t have access to the internet or a computer were excluded.
We used a remote moderated approach for the test sessions. In each session, a team member took one of the three roles; test facilitator, evaluator, or observer. Each session lasted between 30–40 minutes.
Evaluation Methods
Quantitative metrics were used to evaluate the participants’ performance and subjective metrics were used to gather insights about how users felt about the website. The data for these metrics were collected by the evaluator and observer using log sheets and questionnaires.
Quantitative Metrics
Success Rate
Can the participants perform the task and successfully complete it through the happy path?
Task Completed with Assistance
How many times did the participant ask for help?
Task Time
How long did it take for users to complete the task?
Error Rate
How many errors were made by the user?
Subjective Metrics
Subjective Metrics
After-scenario Questionnaire (ASQ)
The purpose of this questionnaire was to gather information about the participants’ experience and identify areas for improvement.
System Usability Scale (SUS)
Helped us understand the participants’ entire experience with the site. The All Positive version was utilized. We used Bangor et al. (2009) method to interpret the SUS scores.
Task 1 Findings
Scenario: Request recycling roll cart online
No participant was able to complete the task. Participants failed due to incorrect navigation or selecting the wrong link (example: email request vs. online form). The task relied on users noticing subtle distinctions that weren’t intuitive.
Success Rate
0%
Failures
5
2/5 Gave Up,
3/5 Got it Wrong
Error Rate
73%
Task Time: 2:10min
Despite moderate time spent, none succeeded–indicating users quickly reached dead ends or confusion.
After-Scenario Questionnaire Results
Users felt moderately satisfied with the time it took but the ease and support was unsatisfactory, aligning with high error rates and abandonment.
Q1
Ease of Completion
4.0
Q2
Time Satisfaction
5.8
Q3
Support Satisfaction
4.2
Recommendations
Clarify labels for online vs. in-person.
Reduce alternate versions of links/buttons.
Add visual elements to highlight the correct path.
Task 2 Findings
Scenario: Find your city commissioner’s email address
Low task success suggests structural and labeling issues around government contact information. Users misunderstood who they were suppose to contact based on UI.
Success Rate
20%
Failures
4
3/5 Gave Up,
1/5 Got it Wrong
Error Rate
72%
Task Time: 3:30min
High time average highlights inconsistent user experience, from confusion to persistence, with little payoff.
After-Scenario Questionnaire Results
Users felt this was the most difficult task. They struggled to complete it and found the support information inadequate.
Q1
Ease of Completion
3.2
Q2
Time Satisfaction
4.8
Q3
Support Satisfaction
3.8
Recommendations
Clarify labels: “City Commission” vs. “County Commissioner”.
Improve search accuracy and results with the use of a search filter.
Ensure full navigation bar is accessible on all screens.
Recommendations
Clarify labels: “City Commission” vs. “County Commissioner”.
Improve search accuracy and results with the use of a search filter.
Ensure full navigation bar is accessible on all screens.
Task 3 Findings
Scenario: Search for Web Designer remote jobs through WorkSource DeKalb
Task 3 was the most successful, though some users veered off path or ended up on the unrelated job bards.
Success Rate
60%
Failures
3
2/5 Didn't complete
the task
Error Rate
70%
Task Time: 3:24min
Some completed the task, others were lost. Variability in navigational effectiveness and comprehension.
After-Scenario Questionnaire Results
Users were most satisfied with this task overall, especially with support information. It indicates this path was better structured with containing clearer labels.
Q1
Ease of Completion
4.6
Q2
Time Satisfaction
6.0
Q3
Support Satisfaction
6.4
Recommendations
Improve visibility of the correct job board.
Reduce external job board distractions and external links.
Make the: Government > Workforce > Jobseekers > Openings, path more prominent.
System Usability Scale Grades
92.5% (A), 87.5% (A)
60% (D), 57.6% (F), 27.5% (F)
Improvement Suggestions
We scaled importance of changes on a scale from 1 (low importance) to 5 (high importance). Here are the most important changes from ranges 3–5:
Level 5
Improve the organizational structure of the website, most common issue in was the user’s inability to locate an item in the navigation bar.
Ensure the website is responsive to accommodate all screen sizes, from laptops and tablets to mobile devices.
Level 4
Make content clear and visible on every page. A summary at the top of the page can help users decide quickly if they are in the right location.
Revisit the website’s design system to create a more consistent experience.
Level 3
Conduct a content inventory of the website to ensure that all links lead to the correct/working pages.
Limit the amount of externally linked pages to decrease confusion for the user.
Improvements Mock Up
Scroll left in the gray area to view screens.
Improvements
Mock Up
Scroll left in the gray area to view screens.
#2
Clear button tiles that link to the proper working page mirroring the design system.
#1
Improvements to navigation to help with users with readability.
#3
Updated content organization to support users search.
(1/2) Implemented Changes
#2
#1
#3
(2/2) Original Website
Conclusion
The website exhibited significant usability challenges, as evidenced by high error rates and low task completion success rates across all tasks. Participants encountered obstacles such as unclear labeling, ineffective navigation paths, and reliance on the search bar, leading to frustration and difficulty in achieving their goals.
Interested in collaborating? Let’s connect!
copyright Diego J. Reyes-Alicea | designed by Diego J. Reyes-Alicea
updated July 2025
Interested in collaborating? Let’s connect!
copyright Diego J. Reyes-Alicea
designed by Diego J. Reyes-Alicea
updated July 2025
AI News Bias Analysis
Timeline: April 7th 2025–June 9th 2025
Team: Bhavika Arigala, Diego J. Reyes-Alicea, Suleyman Sarigoz, Vedant Darak
Role: Lead UI designer, qualitative/quantitative research and analysis, thematic analysis
AI News Bias Analysis
Usability Test Report: DeKalb County Website
Timeline: April 7th 2025–June 9th 2025
Team: Bhavika Arigala, Diego J. Reyes-Alicea, Suleyman Sarigoz, Vedant Darak
Role: Lead UI designer, qualitative/quantitative research and analysis, thematic analysis
Timeline: January 29th 2024–March 8th 2025
Team: Diego Reyes-Alicea, Hannah Blum, Selene Padilla, Tabitha Milon
Role: Usability testing, qualitative analysis, testing facilitator, testing evaluator, testing observer, UI design
