If you read one article describing User Acceptance Testing (UAT), you’ll walk away with a solid understanding of what it’s all about. If you read 5 or more articles, you might just wind up confused. That’s because “User Acceptance Testing” is one of those annoyingly overloaded terms that means different things to different organizations.
Now, I’m not “that guy” who insists upon specific definitions for cloudy terms and tells everyone they’re wrong if they disagree. However, there are at least three different ideas out there about what UAT is, and one of them is the more useful concept to embrace, especially for you fine folks reading this. So, for the purposes of this article, I’ll present UAT by its most useful definition. But I’ll also address the other two definitions and explain where they are coming from.
What UAT Is:
User Acceptance Testing is a software testing activity in which actual users test the product to confirm that it works in real-life situations to fulfill its original intent.
UAT is often the last phase of testing, following developer testing and QA testing. (Your organization may use different activity names in place of these, e.g. Unit Testing, Functional Testing, Integration Testing, System Testing, etc.)
Actual users
Here, actual users are:
- the users already using a previous iteration of the product; or
- users who will use the product once it is released; or
- potential users of the kind you wish to attract to your product; or
- sample users who are reasonable approximations of the above.
The key is to realize that software developers, software testers, project managers, product owners, etc. are NOT the actual users of the software, and not who you should be targeting for UAT.
Real-life situations
UAT is planned testing, so it may not capture literal real-life use. Users may be instructed to perform certain tasks, but those tasks should reflect real-life scenarios in real-life user conditions as much as possible.
Original intent
There was, presumably, a reason why the product or feature came to be in the first place. Usually, some combination of user needs and business goals were deemed good enough reason to greenlight a software development effort. Software specs—requirements, designs, etc.—soon followed.
Most software testing activities focus on whether a product or feature matches the software specs. UAT instead focuses on whether the whether the product or feature sufficiently meets the original user needs and business goals. It’s the difference between verification and validation.
Key concept: Verification vs. Validation
In software testing, validation and verification are not interchangeable terms. Definitions of these two terms don’t get much pithier than these frequently referenced quotes
from Barry Boehm:
Verification is: “Are we building the product right?”
Validation is: “Are we building the right product?”
These definitions are memorable and strike at the heart of the difference between validation and verification. They are also a little too concise and clever for their own good—because, you know, what does that actually mean in practical terms? So, let’s elaborate a little further…
Verification is the act of determining how well something matches agreed-upon specifications.
When you think of “software testing”, you’re probably thinking about verification activities. Verification confirms that software sufficiently meets its predefined software requirements, specs, and designs.
Success or failure in a verification process is determined by the software’s behavior. Non-conformities (bugs) are found and captured. If a bug is considered important enough to fix as a software change, the specific desired software behavior is clear—it’s already been defined.
Verification could be performed by actual users, but rarely is, as it is usually inefficient to do so. So, verification is performed by technical professionals using all sorts of methods, from automated testing to manual test scripts to ad-hoc testing.
Validation is the act of determining we how well something serves its intended purpose.
When you think of “usability testing”, you’re probably thinking of validation activities, but that is not the only kind of validation. When it comes to usability testing, the focus is on how well the user deals with the interface against reasonable expectations. When it comes to UAT, the focus is on how well the product fulfills its stated purpose when in the hands of the user. That stated purpose was likely documented as user requirements and business requirements at the start of the project.
Validation activities such as UAT may be planned and facilitated by experts, but need to be performed by actual users of the software in order to be effective. Success or failure in a validation process is determined by user behavior. Users’ issues and negative reactions (whether directly stated by the user, or observed by another party) are captured. If an issue is considered important enough to address in a software change, the desired software behavior is not immediately clear—thought and redesign are needed.
Takeaways:
- Project success runs on verification.
- Product success is enhanced by validation.
- UAT at its finest is predominantly a validation activity.
What UAT Also Is (But Probably Shouldn’t Be)
While many good sources on the subject of UAT are in line with the definition presented above, many others have a different idea about what UAT is about. This isn’t a huge problem, but it’s unfortunate for, let’s say, three reasons: 1) confusion; 2) there are other terms readily available that already mean what they use UAT to mean; and 3) Using UAT in these other ways pushes aside an important concept that UAT embodies.
This article wouldn’t be as helpful to you if I simply ignored the fact that individuals and organizations often use the term User Acceptance Testing to mean something different. So here goes.
UAT as Client Acceptance Testing
Many organizations treat UAT as equivalent to plain ol’ Acceptance Testing or Client Acceptance Testing. Acceptance testing of this sort is a process hurdle, where a software delivery is evaluated, and a sign-off is required in order to proceed. The agreed-upon acceptance process may consist of any kind of testing or no testing at all, and frequently has nothing to do with validation.
Aside from the words they have in common, the reason for conflating acceptance testing and UAT is a matter of perspective—and sometimes a looser definition of “user” than is desirable.
Let’s say you’re a software vendor for hire. Client acceptance of your software releases is how you close out projects and get paid. Business requirements and user requirements are the client’s responsibility, and may not even by fully shared with you. Your responsibility is to come to an agreement with the client on what it to be built, and to do a good job building it. You may have little or no access to the product’s actual end-users; indeed, you may even think of the client and the user as equivalent. For you, the goal of client acceptance is to confirm the client is satisfied enough to move forward: go, or no-go. There is often motivation on both sides to check that box as quickly as possible.
When this is your perspective, it makes sense that your definition of UAT doesn’t include validation, and might not even include actual users. The activity takes place at the same points in the software lifecycle, but it has a different purpose for you.
However, if your perspective is that of a savvy client, or if your company is building its own product, then your attention should be not only on how software milestones are approved, but on determining if the software product is actually a valid solution.
UAT as Usability Testing
If you scan the landscape, it’s not uncommon to see User Acceptance Testing defined effectively (or exactly) the same as Usability Testing—also referred to as User Testing.
This is understandable. UAT and usability testing have a lot in common: both are validation activities performed by real users, and the practical feedback you’d get from Usability Testing and UAT might overlap quite a bit.
The problem, however, is the focus and the goal of the testing. Usability testing is validation that a given UI design is intuitive and pleasant for the user; UAT is validation that the product as a whole is meeting the user’s needs. Again, there is some overlap here. But the point is, focusing on one risks sacrificing the other, especially if you institutionally view UAT and usability testing as equivalent concepts.
Why UAT Matters
A software product or feature could pass comprehensive verification tests with flying colors, please the client, show little-to-no issues in usability testing, and still fail validation during UAT. To me, that may be reason enough to keep the term “User Acceptance Testing” separate from “Acceptance Testing” and “User Testing”. But there’s another reason, too.
If you convince yourself that UAT is something else, it’s easy to simply not perform that kind of validation at all. On real-life projects where money and jobs are involved, there is a ton of motivation to keep things in the “success” column and keep moving forward.
Validation can be a scary prospect. It’s common for UAT to occur very late in the game, and it’s natural to not want validation testing to actually find problems.
There are lots of reasons why serious issues could arise, seemingly out of nowhere, on a project that had been a glowing success right up until User Acceptance Testing. Maybe reasonable assumptions made early on about what to build were off the mark. Maybe users were wrong about what they wanted in the first place. Maybe other software applications negatively interact with yours in real-life situations. Etc.
The issues uncovered in UAT can be fundamental and serious. It can be unclear how to correct the issues, and—for those organizations that think this way—it can be unclear who was “at fault”. It’s a big can of worms no one is eager to open.
The choice not to perform validation-based UAT may not even be a conscious one. If your organization doesn’t institutionalize validation testing in its processes, then it may not even occur to people running software projects to do it.
Final Thing
Failing to discover validation-level issues in your product during testing might make your software development project run more smoothly, but it’s at the expense of the released product. The problems are still there—they’ve simply been deferred to the future, where they’ll be more expensive to deal with.
Learn about how BetaTesting can help your company launch better products with our beta testing platform and huge community of global testers.
BetaTesting Blog 