User Testing with Userlytics and the American Water Works Association
As their annual conference was weeks away, newmedia quickly developed a prototype to test user navigation and information preferences. Userlytics, our user testing partner, made it easy for the newmedia team to quickly synthesize data and update the prototype. Without data, web designers work from assumptions. With data, the end user experience becomes relevant to design implementations.
The challenge- build a “leap forward” prototype robust enough for users to test at AWWA’s annual conference, just one month from the start of our partnership. The new prototype needed to be both modern in design and effective in structure. Obtaining feedback at the conference was imperative, as AWWA’s primary audience of water professionals/executives, operators, and engineers would all be gathered in one place. AWWA wanted to conduct testing with industry users, as each industry professional group utilizes the site differently to fulfill their own specific needs.
Our process began with discovery. We met with the client to determine what elements of the site were most important to test and to better understand their users and the problems they experience on the current site.
What We Learned
Many users were struggling to find relevant content, because the current menu contained many menu items with equal weight. Users are not given direction on where to look.
We proposed a few updates to their navigation, including a simplified structure with a hamburger menu and a clean look for primary and secondary navigation items. Along with the structure, we created a moodboard to set the design intentions for the prototype and a questionnaire to present to users testing the prototype at the conference. Our deliverable at the end of discovery became a prototype, robust enough for users to test, which included our proposed solution for navigation.
Our primary question for testing became, “Would a modern and simplified navigation make it easier for users to find content on the AWWA site?”
Fortunately, we were able to partner with Userlytics to complete the testing. As they effortlessly managed the “how” of testing, we were able to focus on the “what”. Not only did Userlytics capture screencasts of users’ experiences on the site, they also delivered the questionnaire at the end of the test. AWWA and newmedia collaborated on developing the questionnaire with thoughtful and specific questions. We created three pages with tasks for users to find specific content on the site. We wanted to accurately measure success rates and users’ reactions to the new navigation and other changes on the site.
Userlytics made the process so easy! Their dashboard allowed us to track results and modify our questionnaire. As the conference arrived, newmedia relied on AWWA to observe and document users’ reactions as they tested the prototype. Meanwhile, we watched test results arrive in real time on our Userlytics dashboard. At the end of the first day, our internal team met to watch screencasts, evaluate data, and recommend changes for the second day of user testing. On the day between tests, we talked through our recommendations with the client to create a second iteration of the navigation.
On the first day of testing, some users struggled to find the menu, not immediately recognizing the hamburger menu. We found that many people were not using the main navigation at all, as they simply could not find it. As frequent web users, we can take for granted that people know what a hamburger menu looks like. We added the word “menu” for Day 2 testing. We also added an orange color to the expansion points within the menu, noting orange as the color for interaction throughout the site. Both changes were effective, significantly decreasing the number of users struggling to complete their tasks and increasing positive comments about the new design. After the second day of testing, we compiled our final recommendations based on our expertise and takeaways from the user testing results.
This project would not have been possible without the positive collaboration we had with our client. We cannot understand, test, and design for the end user without the client. Clients cannot lean into human centered design practices without us. AWWA and newmedia were both happy with the results from the conference. As a team, we proved the validity of a simplified navigation and modern cleaner looking AWWA website in real-time.