One of a software developer's worst nightmares is to push code and have a system crash. Or, to craft a permission set and have the wrong “set” of people be able to access information.
And, it can all boil down to misplacing one letter. Ellen Spertus, a Computer Science professor and blogger tells of a time when she was working for Google creating social ads (anyone remember MySpace) and wrote some C++ code that looked like this:
for (int i = 0; i < user->interests->length(); i++) {
for (int j = 0; j < user->interests(i)->keywords.length(); j++) {
keywords->add(user->interests(i)->keywords(i)) {
}
}
I couldn’t have told you what the mistake is but just so you know - the last argument should be “j” not “i” - but no one not her own unit tests or her reviewer caught it! The code was pushed and promptly crashed all the computers in the data center.
Quality Assurance and User Acceptance Testing
Hence, the importance of QA or Quality Assurance and User Acceptance Testing or UAT. QA and UAT are safety nets so that systems don’t crash and users have a smooth experience when using a product.
However, it is more than finding a bug such as the wrong color or a misspelling. It could be performance time in how long it takes for a page to load or how a flow goes with a screen.
Quality Assurance and User Acceptance Testing are ongoing processes that test a software product along the way to make sure the product falls within the parameters set to achieve success. It verifies everyone on the team has adhered to the procedures agreed upon and everything in place works together. Both QA and UAT are critical pieces in software development.
At Airship we call our QA/UAT individuals Inspectors. And, they are the ones that help refine and polish an application.
User Acceptance Testing - Internal
And, not just at the end of development. It is important for inspectors to test along the way. Why? Because a builder may do something that works technically but doesn’t flow from a user perspective. So, every button, every layout gets tested along the way and when the product is finished they go back and do an overall test on the whole project.
For example, user personas within an application. One of our clients has different persona’s and each has a different level of access within the application. So, a community manager has access to all the information but a project manager only has so many permissions. You also have a regular user that has an entirely different view and will only use the product on a mobile device.
As a UAT, you have to walk through the product as each of the personas making sure that each screen is permissible for that particular user. You also have to ask questions about the wording - does a project manager use the same vernacular as a community manager? And, does your end-user who is external understand any of the lingo your internal users use?
Recently, we were testing the ability for renegotiations within an application. Meaning a candidate was offered a job and countered the salary. The manager could then make a new offer. But, I noticed that within the app the renegotiation happened but there wasn’t a notification that let the applicant know that they were being offered a higher salary.
This led to a conversation with the builders to discuss what would work. And, then you have to look at the budget because you have to create a solution that will allow the budget and timelines to continue to be met. And, in software, there are multiple ways to solve a problem. So, together the team has to determine the best solution that gives the client the best product outcome.
It is in the details that we refine aspects of an application to make it better, easier, and more efficient for our clients always striving to stay within the client's timeline and budget.
User Acceptance Testing - External
Once we’ve walked through the application internally as Inspectors, we then have users from the client organization walk through the application testing for bugs or verbiage or flows that don’t work correctly.
The goal is that by the time the application is given to the client's users for UAT that there are only minor tweaks, not major features or flow changes. During this final testing time, no new features are added. The team is working together to fine-tune and finish the application in order for it to be released as an MVP (minimum viable product).
There are also a few risks inherent in this process. For instance, if an organization doesn’t understand the importance of user acceptance testing then there is a risk that the application doesn’t launch with everything in its proper place. And, if UAT isn’t done in a timely manner, there will be budget issues.
At Airship, we work with clients to help them understand the importance of their role in the iterative development process. UAT is a role in which a client's part is of critical importance.
By working collaboratively with clients throughout the development process we can live into our vision of creating transformational change for our clients via remarkable experiences.
