Experimenting with bot moderation supercharged a community on recycling
Including chatbots in qualitative research brings faster and better insights, as it allows a human moderator to focus on value-adding tasks.
How it started
Nestlé Australia, the world’s largest food producer, wanted to immerse into people’s recycling attitudes and habits, to discover opportunities to better support consumers in this area. Nestlé wanted to go beyond what people are saying, and observe what they are doing in-moment. This led to an experiment with in-context automation, a test of whether chatbots could get to better and faster insights.
How we inspired change
An A/B experiment was set up with two ad hoc insight communities, each lasting two weeks and counting 60 members (65% female, 35% male, all involved in kerb-side recycling). Tasks ranged from open-discussion topics to ethnographic diaries – consumers went on recycling missions, uploaded videos and completed surveys, to give us a 360° view on their habits and needs. The difference between the two communities: one was moderated by a human, the other by a chatbot.
How it lives on
The consumer immersion uncovered confusion about what and how to recycle. The amount of education required to adjust people’s behavior became apparent. In one exercise, after observing many conflicting behaviours, we opened an ‘ask any question’ topic. This revealed the vast gap between what people were doing and what they should have been doing; soft plastics, foil and pizza boxes proved all to be illuminating causes of confusion. The project highlighted the size of the challenge to educate people, as well as changing behaviour, if we are to do better. Additionally, we found an over-emphasis on recycling (often done incorrectly), rather than an understanding that, in front of recycling, we need to reduce and reuse. Nestlé discovered a range of fresh recycling insights which highlighted the say-do gap: people don’t recycle as well as they believe they do, and too much focus lies on waste recycling, and not enough on reusing and reducing.
We also found that chatbots will play an important role in future research, especially in repetitive tasks (e.g., diary exercises) and engagement moderation. Participants accept them as moderator (if positioned correctly), and the bots deliver a never-seen-before scale to qualitative research. However, the power is in the mix: humans (today) are still better at digging deeper, understanding emotions and getting qualitative insights. With a bot on their side, they can do so faster and cheaper, as part of the human tasks can be outsourced.
This experiment obtained the Best Paper Award at the Australian Market and Social Research Society (AMSRS) National Conference in 2019.
“We enjoyed working with Human8; they helped us push the boundaries of what marketing research can do, by pioneering new methodologies to get the job done more efficiently and effectively.”