Fluctuating results. Unreliable respondents. Tiny samples with big error margins. These are just a few of the challenges that stand between brands and real insight. And without fixing them, brand tracking becomes guesswork.

Brian Callander, Senior Data Scientist at Latana, is focused on solving these issues at their core. From building machine learning models that reduce margin of error by up to 90%, to designing smarter ways to detect low-quality responses, his work is reshaping how brand data is collected, cleaned, and interpreted.

In this interview, Brian explores the most common - and most costly - issues plaguing brand measurement today. Dive into his answers below. If there is something not covered, send us your question at news@latana.com.

Brand measurement's most pressing questions

Q. What are the biggest data challenges when it comes to measuring brand performance?

A. To be able to accurately measure brand performance, insights need to be derived from real-world changes and not just noise in the sample, and several factors make this particularly challenging to do:

• Inadequate cleaning processes that leave nonsense or inauthentic responses in the dataset can render a sample unreliable
• Small sample sizes, which get even smaller after cleaning or segmenting audiences, have large margins of error, leaving a lot of room for uncertainty
• Limited or non-existent data modelling, meaning insights are lacking enhanced accuracy or stability

All of the above can result in fluctuations between waves of data or strange results that don’t match expectations. Ultimately, this leaves brands to rely on making big-budget decisions based on guesswork rather than fact.

Q. What role can machine learning play in measuring brand performance?

We use machine learning at the core of our product to solve the issues traditional sampling presents: fostering data quality, accuracy, and high-precision audience segmentation. We built a Bayesian statistical model for this purpose, leveraging Multilevel Regression and Poststratification (MRP), for more precise weighting and processing.

Typical quota sampling methods consider each wave of data and each audience segment as unrelated. Latana’s MRP models, on the other hand, recognise that results tend to be similar in consecutive waves and that segments are not independent of one another. This allows our models to build a more coherent narrative of how brand data evolves over time, often resulting in more realistic estimates which don’t jump around wildly from wave to wave. The resulting MOEs are therefore up to 90% smaller than in typical quota sampling datasets. This helps make the results significantly more actionable for our clients, even for smaller, hard-to-reach or combined audience segments. 

Q. How can we use machine learning to better detect poor response or bot behaviour?

A. This is another scenario in which we use machine learning to enhance our data quality. We measure respondent reliability using a wide range of in-survey behaviour-based metrics. These metrics are related to typical survey biases, such as acquiescence bias or list-based bias. These behaviour-based metrics are then condensed into a single score using advanced ML techniques and validated through a range of targeted surveys using a battery of quality control questions. Those respondents who score below a certain quality threshold are then removed from the survey. 

This advanced assessment of each quality trait within the final scoring, rather than a typical binary method, means we avoid over-cleaning genuine respondents who might have made a mistake in their answering behaviour but have committed to providing real responses to brand questions. We also avoid under-cleaning by missing behaviours that are hard to define in a single quality captcha question. 

Q. What is Latana doing differently from the main trackers in the industry?

A. At Latana, we are unique in that we combine traditional research agency expertise with advanced technology and innovation. This means we can push the boundaries and explore with methods that are not typical of brand tracking, such as machine learning. 

We place a core focus on data quality and integrity - from the unique approaches we use to collect casual ad-based respondents, to the Bayesian enhancements to our datasets, to the advanced quality score cleaning methods we apply. All of this ensures that our margins of error are lower than any in the industry, that our audience segmentation is precise and extensive, and that our data insights are derived from genuine real-life events. 

Got questions about data collection or brand tracking? Submit them to news@latana.com, and we’ll provide you with expert answers.