Brand tracking has a technology problem. While AI, cloud infrastructure, and advanced statistical methods have reshaped many industries, most brand tracking processes still look like they did a decade ago: manual, slow, and inflexible.

At Latana, we saw an opportunity to build something different. From real-time data collection to fully automated pipelines and retrieval-augmented generation (RAG) systems, we’ve embraced the latest technology to transform how brand insights are generated and interpreted.

In this article, our Chief Technology Officer, Joe Corcoran, unpacks the biggest advancements behind the scenes, from fraud-resistant survey methods to scalable machine learning workflows, and explains how they’re reshaping what’s possible in brand tracking.

Joe is taking on the most pressing questions we get about brand tracking technology. Explore the full list below or submit your question to news@latana.com - we’ll make sure to answer it.

Brand tracking's most pressing questions

Q. What were the most significant technological breakthroughs that transformed the generation and analysis of brand tracking data over the last five years?

A. The exciting breakthroughs of the past few years, like the commodification of large language models, prevalence of serverless data infrastructure, advancements in the application of Bayesian statistical methods and use of synthetic data, have not yet made a big splash on generating and analysing data in brand tracking. Brand tracking as a field is dominated by large incumbents who developed their data processes a long time ago and for whom it can be difficult to adopt new technologies.

We started Latana just as these new technologies were becoming viable, so we were very nicely positioned to take advantage of them. For example, building robust, production-grade machine learning pipelines is fundamental to our business. Being a small, agile team gives us the ability to release new machine learning models to production all the time.

A lot of our customers have told us that they'd love to reduce the amount of time they spend staring at spreadsheets and PDFs. As such, we've just started to roll out a retrieval-augmented generation (RAG) infrastructure as an additional data interpretation layer in our product. This will drastically reduce the time it takes our customers to find the most impactful insights within our data, eliminating a lot of frustrating data digging work in the process. It's a really exciting time to be solving these problems.

Q. Data quality is a constant challenge in brand tracking. What do you think have been the most important advancements here?

A. One of the major causes of poor data quality is fraud, which occurs inherently when survey respondents are financially incentivised. We realised that there was no way around this some years ago. No matter how well you detect it, or how sophisticated your reconciliation process is, you just can't intercept every fraudulent answer. Respondents are good at hiding it, because they want to get paid! The brand tracking industry as a whole has not really risen to this challenge yet.

For this reason, we serve micro-surveys inside interactive ad placements. We meet the respondents where they are – reading, gaming, browsing social media – which means there's no need for them to opt into joining a panel. All of our respondents submit answers without incentives. Of course, this comes with its own set of challenges, such as displaying the surveys correctly on thousands of device types and designing the surveys to work with dynamic attribution. But it opens up a whole world of possibilities as well. Our reach is orders of magnitude broader now, even in harder-to-reach demographic segments. We do all of this via integrations with multiple demand-side platforms, each of which gives us different advantages in terms of access and collection cost.

In addition to the survey answers we collect, we also gather all manner of anonymised behavioural data from the respondent session. Our in-house machine learning models can then use this to augment the data before determining the quality of the answers. Over the past few years, we’ve cautiously introduced significant improvements to our quality models without causing data fluctuations. Being able to build all of this cost-effectively and at scale, thanks to affordable cloud data engineering infrastructure, has probably been the most important advancement for us.

‍Q. Automation and AI have streamlined areas of brand tracking, but they also come with risks. How can companies maintain data quality as they rely more on these technologies?

A. Automation has made it possible for companies to process big data without needing large teams of employees. For example, our system allows us to turn data collection around in a matter of hours rather than days or weeks. The automation not only improves the initial data collection, but it has also removed 95% of the manual work in processing and delivery. This reduction in turnaround time has completely changed the game for our customers.

AI opens up huge opportunities for monitoring and ranking data in terms of usefulness and credibility. It requires a willingness to use bleeding-edge technology and to develop your own tools, rather than waiting for the industry to hand you a complete solution.

I think one of the biggest mistakes one could make, though, would be to automate the entire data delivery process without any oversight. We’re very lucky to have subject experts at Latana who can make sure we’re building the right tools.

‍Q. Latana has significantly evolved over the years. Can you walk us through the most important technological advancements that have defined our brand tracking platform?

A. We’ve gone through many iterations of our platform, both the user-facing dashboard and the data pipelines behind it. In the early days, meeting the challenges of a high-volume data collection pipeline would have been much harder had we not taken advantage of serverless cloud components. Using streams for continuous data ingestion has given us a lot of peace of mind. I can sleep well at night knowing that we're set up to ingest millions of survey answers per hour without issue.

Customers use our data to make critical business decisions, so it’s quite reasonable that they expect data stability over a long time period. We can keep multiple versions of our pipeline online and route the traffic appropriately; being able to deploy bleeding-edge changes for new customers, and keep existing customers pinned to their original pipelines, was a huge win for us, too.

We’ve relied heavily on AWS over the years. We’ve used pretty much every possible way of running machine learning jobs at scale on AWS, from SageMaker to EKS to Batch, and recently settled on a way that gives us cost-effective data processing that doesn’t sacrifice scalability. Our machine learning processing times have gone from days down to minutes thanks to our data engineers getting the best possible performance out of AWS Batch. For each survey, we calculate millions of segmented demographic combinations in parallel, and all at a fraction of the cost of maintaining an OLTP database.

All of this would have been significantly harder to achieve 5-10 years ago. We’re in a great position to handle all manner of customers, from smaller businesses to enterprises now. After a number of years at the cutting edge of brand tracking, our investment in research and development is really paying off.

Got questions about brand tracking technology? Submit them to news@latana.com, and we’ll provide you with expert answers.