One of the most common questions we get asked about our test design process is: “How should I setup my beta test?” How many tasks should I give testers? How many surveys should I setup?
There are a lot of different ways to setup your beta test, and it is usually driven by what you hope to achieve from your test.
QA / Bug Reports / Functional Testing
For technical testing, where you are trying to identify bugs, specific test design scenarios that cover your entire product are usually best. For example:
- Create an account in our app and fill out your profile.
- Go to Settings, click on “invite a friend”, and add their email address to the app.
- Go to ‘Newsfeed’, upload your first picture, and add location tags to it. Submit a bug report if you are unable to complete any of these tasks.
Asking for specific tasks helps testers stay focused and helps our clients find bugs across different operating systems, phones, and browsers that can be resolved quickly.
Usability or UX Testing
If you are more interested in general usability and UX/UI testing, we recommend keeping the test design process more high level and letting users explore your product more organically. For example:
- Create an account and begin using our app.
- Try to connect our app to your Google Home or Alexa.
- Use voice commands to control the settings in the app and share what voice commands you tried.
- Continue using the app every day for the next week and then answer our final survey.
Higher-level test tasks give testers more freedom to explore the app on their own. This provides a larger variety of use cases and opinions from testers. When you want to learn about your app’s design, ease of use, or net promoter score, a test designed to let users figure things out on their own will help you get more accurate scores.
BetaTesting Blog 