Category Archives: Analytics

Machine Learning and Human Interpretability

The key idea behind Conductrics is that marketing optimization is really a reinforcement learning problem, a class of machine learning problem, rather than an AB testing problem. Framing optimization as a reinforcement learning problem allowed us to provide, from the very beginning, not just AB and multivariate testing tools, but also multi-armed bandits, predictive targeting, and a type of multi-touch decision attribution […]

Also posted in Testing and Data Science, Uncategorized | | Leave a comment


Easy Introduction to AB Testing and P-Values

UPDATED: 5/20/2021 A version of this post was originally published over at Conversion XL For all of the talk about how awesome (and big, don’t forget big) Big data is, one of the favorite tools in the conversion optimization toolkit, AB Testing, is decidedly small data. Optimization, winners and losers, Lean this that or the other […]

Also posted in Testing and Data Science, Uncategorized | | 2 Comments


Predictive Targeting: Managing Complexity

Personalization, one to one, predictive targeting, whatever you call it. Serving the optimal digital experience for each customer is often touted as the pinnacle of digital marketing efficacy. But if predictive targeting is so great, why isn’t everyone doing it right now?   The reason is that while targeting can be incredibility valuable, many in […]

Also posted in Testing and Data Science, Uncategorized | | 3 Comments


AB Testing: When Tests Collide

Normally, when we talk about AB Tests (standard or Bandit style), we tend to focus on things like the different test options, the reporting, the significance levels, etc.  However, once we start implementing tests, especially at scale, it becomes clear that we need a way to manage how we assign users to each test.  There […]

Also posted in Reporting, Testing and Data Science | | Leave a comment


Intelligent Agents: AB Testing, User Targeting, and Machine Learning

(Updated: Aug 2020) Whether you are in a large company running marketing campaigns, or in digital analytics, or a data scientist at a tech startup, you probably are on board with the importance of analytical decision-making.  Go to any related conference, blog, meet up, industry slack and you will hear many of the following terms: […]

Also posted in Testing and Data Science | | Tagged , | Leave a comment


A List of Data Science and Machine Learning Resources

Every now and then I get asked for some help or for some pointers on a machine learning/data science topic.  I tend respond with links to resources by folks that I consider to be experts in the topic area.   Over time my list has gotten a little larger so I decided to put it all […]

Also posted in Testing and Data Science, Uncategorized | | 7 Comments


Decision Attribution for Multi-Touch Optimization

Right now online advertisers and marketers have a lot of interest in attribution analysis.  Marketers are struggling to determine the impact of each individual campaign, across various channels, that make up their online marketing efforts.  While they can discern the aggregate effects, it is much more difficult to disentangle the individual effectiveness from any given […]

Also posted in Testing and Data Science, Uncategorized | | Tagged , , , , | Leave a comment


Conductrics’ Confidence and Lift Report – The Basics

While enabling Conductrics’ adaptive testing is a powerful way to auto-optimize your app, there are times when you might want to run  non-adaptive tests. Maybe you want to test something that you will apply in other media, or perhaps you want to do online research on your customers, to give you additional insights into their […]

Also posted in Conductrics API, Reporting, Testing and Data Science, Uncategorized | | Tagged | Leave a comment


Bandit Optimization and Emetrics

This past week, I had the pleasure of getting to speak at Emetrics Stockholm. For those not aware, Emetrics is one of the largest digital analytics conferences, with local events in several locations throughout the year. It was my first time in Stockholm. It is an absolutely gorgeous city. Plus, if you like Scotch, you […]

Also posted in Testing and Data Science, Travel | | Tagged , , | Leave a comment