CYE#4 Marketing Mix Modeling Best Practices: Why Your Model Is a Container, Not a Report

Jon “JLo” Lorenzini introduces the Curve Your Enthusiasm series and welcomes guests John Wallace and Dr. Dirk Beyer. The discussion begins with Beyer’s professional background and his decision to return to marketing analytics after time spent in identity and fraud analytics. They reflect on the evolution of marketing mix modeling, from early regression models to sophisticated Bayesian methods, the brief rise of multi-touch attribution, and the current resurgence of MMM in a privacy-conscious landscape. 

Beyer explains why marketing measurement is uniquely difficult compared to fields like supply chain management. In supply chain, outcomes are directly observable; in marketing, success relies on estimating counterfactuals, what would have happened without the ad spend. They discuss the necessity of combining multiple data sources, experimentation, business outcomes, and platform data, into a coherent framework rather than relying on a single, monolithic model. 

Beyer introduces the concept of an MMM as a container or machine for predictive scenario evaluation. He argues that the mathematical structure of traditional models is well suited to holding diverse information, including lift curves and priors, allowing marketers to perform optimization through marginal analysis and first derivatives. This approach moves beyond simple regression to create a usable tool for future planning. 

The speakers discuss the technical challenges of modern MMM, specifically the high number of parameters relative to available data points. They highlight the popularity of hierarchical Bayesian models for their ability to manage degrees of freedom by allowing analysts to specify priors and group channels. They emphasize that the primary issue in marketing analytics is often a lack of data variability, which requires better experimentation to isolate effects. 

Addressing a question from the audience, they explore how to incorporate creative performance into models. They suggest that while individual creative pieces are too granular for an MMM, they can be accounted for using creative scores derived from meta-analyses, or by testing creative types and categories through lift tests, which can then inform the model’s inputs. 

Beyer critiques the industry’s tendency to over-promise and focus on which algorithm is “better,” emphasizing that models can only be as good as the underlying data. They discuss the role of open-source tools like Meridian or Orbit, noting that while these are powerful engineering feats, they require expert practitioners to set them up, define parameters, and interpret results correctly. 

This segment focuses on the nuances of data interpretation. Beyer discusses why platform-attributed revenue, while useful, contains biases like double counting and matching issues. He distinguishes between the statistical act of estimating parameters and the modeling act of ensuring the representation of market reality is accurate, stressing the importance of psychological and business context over pure math. 

In response to a final audience question, the speakers discuss instituting MMM for optimization support. They agree that while last-click attribution is limited, it provides timely information that can be modulated with incrementality findings from broader models to help guide day-to-day decisions. The session concludes with a summary of the importance of collaborative, transparent modeling.