MORE Reasons to Use a Longitudinal Study to Test Your App

Here are more examples of insights that a longitudinal study can provide about your web or mobile application and its users, to help you improve your user experience and increase profitability.

Earlier this month, I revisited longitudinal studies and provided a number of specific-ish examples of valuable insights you might gain from running one.

To refresh your memory: the longitudinal study is a testing method whose defining characteristic is collecting data from the same users multiple times over a period longer than a typical user test. How much longer? Well, there’s no set duration for longitudinal studies. You might plan one to last three days, three weeks, or three months. Whatever it takes to fit your purpose. And, again, your purpose is to gain insights about your product and your users that you just can’t achieve with regular testing.

In that previous article on longitudinal studies, the examples spanned four categories:

  • How users’ attitudes toward your product change over time
  • Natural use and usage patterns over time
  • How users’ own data affects their behavior
  • How users handle long-term tasks

Below are four MORE categories of reasons of why you might want to use longitudinal testing to test your app. These examples of target insights might not fit your exact situation. They are to inspire you to think of all the ways you might use longitudinal testing for your own your product.

MORE reasons to use a longitudinal study

You might want to use a longitudinal study to test your app and help you to learn about things like:

Long-term learnability and forgettability issues

Regular user testing will do a good job testing the initial learnability of your user interface when it comes to the tasks you give to your participants. Longitudinal tests can inform you about: long-term learnability and forgettability; how interfaces optimized for short-term learnability affect experienced users; and more.

Example insight goals:

  • Are users still experiencing the same problems with the same tasks over time? Do new usability issues arise?
  • Do users forget how to perform common tasks over time, or after some time away from the using the product?
  • Does your highly learnable tool still satisfy users after they become “power users”?
  • Will your users forget how to use (or simply forget about) rarely-used features in your application?

The impact and effectiveness of your onboarding techniques

You can focus your testing to learn more about how your onboarding techniques, tutorials, and hint systems are engaged by users, and what long-term effect they have on product use.

Example insight goals:

  • Do users who skip tutorials have a worse experience using the app? If so, do those users ever overcome it?
  • Do external prompts—such as follow-up emails that explain app features and encourage their use—have the expected positive impact on the user?
  • Do specialized first-time-use flows help the user understand the purpose/usefulness of the app? Does this translate into increased account creation?

How users handle expert / unlockable functionality

Sometimes functionality doesn’t apply to new users. This may be due to a user’s level of ability, or because your product can unlock features over time. With a longitudinal study you can see the evolution.

Example insight goals:

  • When do your users feel comfortable tackling advanced tasks?
  • Are users overwhelmed or confused when advanced tasks are available to them from the outset?
  • What keeps users interested longer: unlocking features as they go, or starting with everything available?
  • Does your mobile game’s progression system continue to keep your players engaged through to the end? If not, where and why do they lose interest?
  • Does the availability of paid downloadable content keep users engaged with your app longer?

Other validation and discovery

With longitudinal testing you can validate your assumptions, and also just be open to discovering things about your users that you wouldn’t know to specifically target.

You have the opportunity to receive a ton of raw data and feedback from your participants. You can turn this data into insights, some of which you’ll be looking for, some of which you’ll simply stumble upon.

Example insights (as goals or happy accidents):

  • What actually motivates your users to use your application?
  • What is your user’s impression of your product before they use it? Does it match their understanding of the product after they use it?
  • What influences how a user makes a decision in your app?
  • What is most important to your users? Does that match your prior research and assumptions?
  • What is a sustainable level of use in the long term? Does that meet your business goals?

Learn about how BetaTesting can help your company launch better products with our beta testing platform and huge community of global testers.

%d bloggers like this: