How to make changes to brand trackers without unnecessary complexity
Stephanie Clapham
.png)
Measuring whether and how brand marketing changes people’s brand perception often remains more art than science. Tools to track brand perception can easily cost millions across multiple markets but often come with serious challenges in data quality. One of the major areas of data frustration in brand tracking is the inflexibility of brand surveys, with even small changes leading to disruptions to the data and bigger changes leading to unnecessary complexity.
Most brand trackers work on a project spec basis, in which the markets, survey elements like questions and brands, and audience targets are specified at the start of tracking during the setup phase. Even though brands and audiences evolve over time, trackers mostly need to remain stable to avoid disruptions to existing data that could render historical trend reporting impossible. This is due to the following challenges:
Expanding your tracker to keep up with brand expansion and growth into new territories should be seamless, but if the additional markets required are smaller or harder to reach this can quickly become complex. This is because sample vendors will often subcontract work they aren’t able to cover through their own reach, and their subcontractors might again subcontract part of the sampling work. In markets with limited online panel reach, alternative methods like CATI or face-to-face interviews might come into play, creating even more challenges for coherent data analysis.
This can hurt your existing tracker due to quality disparities, in which sample vendors have different quality control processes that can vary in effectiveness, as well as methodology inconsistencies, which can come with varying sample biases. All of this can lead to fairly complex investigations to identify where suspicious behaviour might be coming from. For brands, these inconsistencies can lead to misread consumer sentiment, flawed marketing optimisations, and ultimately, millions wasted in ad spend.
Brand tracking questions typically come in list-based formats, for example, “Which of the following brands do you know?” followed by a list of options. This is generally due to their space-saving capabilities, which means you can ask more questions about more brands in the survey.
The challenge with list-based questions is that the composition and length of the list can significantly affect the responses. This brand halo effect means that the type of brands that are listed can influence the likelihood of each being selected, resulting in list-based bias. For example, in a test with household cleaning brands, we found that pairing a well-known brand (Mr Muscle) with lesser-known brands resulted in a reported awareness level for the well-known brand that was 13% lower than when paired with other well-known brands. The same is true for smaller brands, too. When we paired a lesser-known household cleaning brand (Splosh) with a well-known brand, the reported awareness level for the lesser-known brand was up to 24% lower than when paired with other lesser-known brands.
As a result, the estimated levels are often wrong for list-based questions. More importantly, however, whenever a list is changed - for example, because a new brand is added or removed - the data for the other brands changes, thus causing a break in the data.
Refining or adjusting the audiences you need to track can also be challenging, depending on the complexity of the audience you are trying to reach. If you want to combine different characteristics, like reaching 25-45-year-old high-earning males, or want to reach niche audiences with low incidence rates, like parents of young children, doing so within the existing sampling framework would likely lead to insights with high margins of error and questionable reliability.
Tracking setups often account for these complexities in the setup process by setting certain quotas on the characteristics required to build a target audience so that an adequate sample is collected for these traits. But if audience characteristics shift over time, or your brand evolves to meet new audiences, this could even lead to a restart of your entire tracker if the new requirements are not possible within the existing sampling framework.
Ad-based sampling uses the digital advertising ecosystem to display questions in the form of ads to people around the world. This has many advantages over classic panel-based sampling, but the most significant one is that the sampling frame extends to almost every person who has a smartphone. Since almost 70% of the world’s population - more than 6 billion people - currently own a smartphone and almost all of them use services and apps that can display advertisements, this means that ad-based sampling provides the largest possible sampling frame of any existing sampling methodology.
Another key advantage of ad-based sampling is that it works in almost every country and region across the globe in a methodologically consistent way, thus enabling cross-country comparisons without the need to correct for potential biases from different sampling approaches (e.g. if an online panel was used to collect data for India, but a face to face approach in Pakistan).
.png)
To be able to make changes to the survey without inadvertently also changing the consistency of the data, we’re asking every question in a “siloed” format, presenting only one brand, image statement or other item at a time to respondents.
This has several important benefits: it enables us to add, remove or update brands and other survey elements without any impact on the consistency of the data; we can carefully control quotas for each question to ensure large samples throughout the survey even for funnel data; and we’re able to optimise the survey experience for respondents, removing the need to scroll through long lists of options and enhancing visuals like brand logos to be clear and prominent on the screen.
To avoid audience limitations without creating hundreds of quota cells, we’ve developed a technology that enables us to automatically set quotas on individual answer options to ensure that we always have sufficient responses for every relevant audience characteristic.
We then use multilevel regression and poststratification (MRP), a technique to estimate subgroup relationships and adjust them to match population demographics for better representativeness, to calculate the proportion of each possible audience combination within the overall population. This enables us to determine, for example, that 0.9% of the sample consists of high-income young men planning to purchase a car. We do this for thousands of possible audience combinations.
The process typically relies on significantly larger sample sizes, with 50,000 respondents per brand annually, often 5 to 10 times larger than traditional methodologies. It is also very computationally intense, but it enables us to allow for far deeper and far more flexible segmentation than through any other methodology.
Inflexibility to brand trackers can leave industry benchmarking a challenge if there are limitations to adding or changing markets, brands, or audiences. At best, tracking setups can become overly complex but still achievable, with multiple vendors providing a patchwork of sampling to meet market and audience requirements. At worst, however, data quality can take a real hit, and the insights generated are so unreliable that they quickly become redundant. Given that reaching small markets or niche audiences is an expensive task, this can accumulate to a costly waste of budget.
By combining ad-based sampling, siloed and modularised surveys, and Bayesian statistics, we can:
1 - Easily add markets to trackers
Our unique and vast sampling access through ad-based networks means there is very rarely any market, or city or DMA, that we cannot reach, meaning we are always able to meet the tracking expansion needs of brands.
2 - Enable flexibility to survey setups
Siloing questions makes us completely flexible in our tracker setups - we can easily add or remove brands, amend or extend image statements or other association questions without any disruption to the existing data.
3 - Cover deeper and more complex audiences
Our MRP modelling ensures that we produce highly reliable audience data on flexible and complex combinations of characteristics, all with low margins of error.
