The ultimate measure of successful research in a mature and well-understood category hinges on its ability to uncover new, game-changing insights. But how can you maximize your chances of uncovering truly new insights? One way is to use machine learning to analyze qualitative data to identify infrequently mentioned insights.
You might wonder whether infrequently mentioned insights can really be that important. They can be. In fact, it’s a common misconception that infrequently mentioned needs are less important than frequently mentioned needs. Academic research demonstrates otherwise[1].
Researchers at MIT have developed an algorithm that can process hundreds of thousands of data records to identify actionable new insights. The data can come from online user generated content (UGC) including user forums, product review sites, and other publicly available data sources. The machine can also be used to mine large amounts of proprietary data, such as call center recordings, open-ended survey responses, or transcripts of qualitative interviews.
The machine is powerful and unbiased. It can identify frequently mentioned insights as well as infrequently mentioned ones – those that are only discoverable due to the machine’s ability to process a large amount of data. Time and again, the machine finds “needle in a haystack” insights that researchers often don’t think to ask about and respondents often don’t think to mention during traditional qualitative research.
As an example, in conducting a research study on kitchen blenders, the MIT algorithm unearthed unique needs that customers rarely think about until they encounter a specific pain point.
Some examples of these unique needs include:
- Assured the parts never unscrew themselves while blending
- Easy to turn the blender on or off (i.e., the on/off switch doesn't become hard to push)
- Blender won’t ruin the surface it’s sitting on
When people encounter product issues, such as those implied in the above statements, they frequently go online to write about them. This rich experience-based content is the perfect type of UGC data for machine learning.
When sufficient data exists, machine learning is a powerful technique to gather impactful, infrequently heard new insights.
To learn more about how machine learning can be used to uncover actionable qualitative insights quickly and cost-effectively, watch our webinar on demand.
The webinar, hosted by Carmel Dibner, explores how machine learning can be integrated into your research toolkit.
[1] Griffin, Abbie and John R. Hauser (1993), "The Voice of the Customer," Marketing Science, 12, 1, (Winter), 1-27.
Tags: Machine Learning/AI