The longitudinal study is a powerful tool for testing your product and learning about your users. Here are some examples of why you might use longitudinal studies.
User testing tends to be single-serving research. Whether it’s remote, unmoderated testing or an in-person moderated session, user testing tends to span a short period and gather a single set of data from each participant—a “snapshot” of his or her experience with your product.
A longitudinal study, on the other hand, is user testing in which you collect a data from the same participants multiple times over an extended period.
Longitudinal testing allows you to see how your users’ behaviors and attitudes change over time, and/or test out features and tasks that could not be performed naturally (or at all) in the single test session. It’s appropriate for both beta testing and post-release product testing.
Longitudinal studies are sometimes referred to as “Diary studies” as if the two terms are interchangeable. However, having your participants enter data into a diary is just one method of collecting data. You might use other methods instead of, or in addition to, a data diary.
In fact, you really have carte blanche to combine surveys, test data collection, and user instructions however you want. The flexibility of longitudinal studies is part of how they enable you to discover insights about your product and your users that you wouldn’t be able to otherwise.
Sound intriguing?
Below are some examples of why you might want to use longitudinal testing. The list may not include your team’s exact needs, but it should spark ideas about how you can use longitudinal studies to better understand your users and improve your product (or service or system or process).
Sound confusing?
Feel free to read more about longitudinal studies before or after you continue here.
Examples of why you’d want to use longitudinal studies
I don’t know you, but you might want to use longitudinal studies to help you to find out things like:
How users’ attitudes toward your product change over time
Longitudinal testing can show you how users’ attitudes about your product evolve over time. After you collect the data, you can map users’ journeys, and tie attitude changes to circumstances both inside and outside of the software application.
Example insight goals:
- Do users remain engaged with your app over time? If not, then where and why do your users become disenchanted?
- Is your mobile game still fun for users after they master the core game mechanics?
- How do task-completion failures and frustrations affect long-term engagement and enjoyment?
- How do users perceive your brand after significant interaction with your product and business?
Natural use and usage patterns over time
Over a longer period of time, users have a chance to use a product or service in ways that are more natural to them. As a result, you have a chance to see how users settle into usage patterns, how they discover and use features without prompting, etc.
Example insight goals:
- When and how often users use your product over time? Do they incorporate it into their daily or weekly habits?
- Find out whether internal and environmental prompts (content changes, internal and external notifications, etc.) have an impact on product usage.
- How often do users naturally use infrequently-used features? When and how often? Are there usability problems exposed by infrequent use? Do users find these features on their own without prompting?
- Why do users stop using a particular feature? (e.g., Do they forget about it? Do they find the function useless or frustrating?)
How users’ own data affects their behavior
In the limited time of a traditional test, it’s often necessary to use canned data to facilitate tests and evaluate success. With the time available in a longitudinal study, the user might be able to use their own data, which will be more meaningful to them than test data would.
Example insight goals:
- How do users behave differently when using their own data (versus using prepared test data)?
- How long does it take for users to add their own data to your social media app when they are self-directed?
- How do users’ preferences and activity change after they’ve added a “critical mass” of their own data?
How users handle long-term tasks
Some tasks take longer than can naturally occur within a single usability test session. For example, doing your taxes, or planning a vacation itinerary. In real life, users will start a task, leave, return, continue, leave again, etc.
Example insight goals:
- How well do users handle long-term tasks in your mobile app?
- Do your users want or need to break up tasks across multiple usage sessions (particularly in flows that were designed assuming completion a single session)?
- How well do users handle disengaging from and reengaging with a long-term multi-session task? Does usability decrease when the users spend more time away in between sessions?
Learn about how BetaTesting can help your company launch better products with our beta testing platform and huge community of global testers.
BetaTesting Blog 