Profit uplift modeling for direct marketing campaigns: approaches and applications for online shops

Before launching a direct marketing campaign, often, a sample of customers is testwise contacted. Their desired responses (e.g., website visits, conversions, revenues) as well as their past information and buying behavior is used to build a response model. Then, this model is applied to all customers and to select likely responders for the campaign. However, this traditional approach has two shortcomings:

Both shortcomings restrict the usefulness of the traditional approaches for maximizing profit. In this paper, we propose new profit uplift modeling approaches as alternatives: first, uplifts focus—in contrast to responses—on the incremental response due to a treatment using control groups of customers. Second, profit is more difficult to model since this outcome is only observable in a few cases but more closely related to the main objective than website visit, purchase, or revenue. The proposed new approaches in this paper extend findings from the field of binary and revenue uplift modeling (e.g., Radcliffe and Surry 1999, 2011; Kane et al. 2014; Rudaś and Jaroszewicz 2018; Gubela et al. 2020) and from the field of two-stage estimation via sample selection (see, e.g., Heckman 1979) and zero-inflated regression (see, e.g., Lambert 1992; Ridout et al. 2001) as well as one-stage parameter estimation via ordinary regression and random forest. We show that the new approaches are well suited to select “best” customers as targets for direct marketing campaigns and improve profit.

The paper is organized as follows: In Sect. 2, we discuss the traditional approaches and their shortcomings and, in Sect. 3, the new approaches. In Sect. 4, the superiority of the new over the traditional approaches is demonstrated using a new dataset from a major German online retailer (with a sample of n = 155,388 customers). In Sect. 5, the well-known Hillstrom direct marketing campaign dataset (with a sample of n = 64,000 customers) is used for the same purpose. The paper closes with conclusions and outlook.

Testing and predictive modeling are assumed to be the analytical cornerstones of today’s direct marketing (Blattberg et al. 2008). The modeling process usually consists of the following five steps: (1) Define the managerial problem in terms of a campaign and its intended effects, (2) translate this description to a predictive model with treatment, responses, and potential predictors, (3) sample customers for collecting responses, (4) calibrate and validate the predictive model, (5) apply the model to all customers and select “best” customers according to the predictive model. Typical managerial problems are selecting targets for an acquisition campaign at hand, deciding on customers to receive a catalog or inlay, or identifying promising customers for a customer tier program. Outcomes are the response to the treatment, predictors are customer characteristics (age, gender, income if available) and variables that describe past information and buying behavior in the customer database (see, e.g., Blattberg et al. 2008 for an overview).

Let \(Y_{i}\) be the binary (\(Y_{i} \in \{ 0,1\}\)) or continuous (\(Y_{i} \in {\mathbb{R}}\)) outcome for customer i in the customer sample, \({\mathbf{x}}_{i}\)(\({\mathbf{x}}_{i} \in {\mathbb{R}}^{m}\)) customer i's values for the m predictors, and \(\tau_{i}\) the indicator whether customer i received the treatment (= 1) or not (= 0). Then, the main goal for a traditional response modeling approach would be to predict the following purchase likelihood (in case of binary outcomes) or scalar values (in case of continuous outcomes) as customer scores for selecting targets,

The data of the treated customers (\(\tau_{i} = 1\)) are used to calibrate the response model (using, e.g., logistic regression or simple regression depending on the scale of the outcome). Then, the whole customer database is used for prediction. The customers with the highest (response) scores are targets for the campaign. However, this response modeling approach has one major shortcoming: it favors customers who respond most likely, but it does not take into account that some of them would also respond if not treated. When the treatment is a discount, a voucher, a catalog, an inlay, or one has to deal with postage, this could result in a waste of money for the company.

Therefore, recently, uplift models have been proposed: responses are again collected from a sample of treated customers (the treatment group), but also from a sample of not treated customers (the control group). An uplift model now predicts the difference in the response of a customer if treated (\(\tau_{i} = 1\)) and if not treated (\(\tau_{i} = 0\)), the so-called uplift score,

Terms like differential response (e.g., Radcliffe and Surry 1999), true lift (Lo 2002), or uplift (Radcliffe and Surry 2011) are used for the same idea. Formula (2) allows to estimate the effect of the treatment and enables the company to select customers where the treatment has the highest impact.

However, when trying to estimate the model parameters, a problem arises from the fact that per customer, only one of these two responses is observable: a customer is part of the treatment group (\(\tau_{i} = 1\)) or part of the control group (\(\tau_{i} = 0\)), not of both groups. Consequently, an uplift model cannot be estimated directly when using formula (2). Instead, one straightforward idea is to develop two separate models (the so-called two model approach): a first model is derived similar to formula (1), based on the treatment group. This model predicts the outcome if treated in terms of \({\mathbf{x}}_{i}\) for all customers. A second model is derived using the control group. This model predicts the outcome if not treated in terms of \({\mathbf{x}}_{i}\) for all customers (see Radcliffe and Surry 1999), the difference between the predictions of the two models is the uplift.

An alternative solution is the so-called interaction model proposed by Lo (2002): an interaction (response) model uses the treatment (\(\tau_{i}\)) and interactions between the predictors and the treatment as additional predictors. The interaction model can be calibrated on the treatment and the control group simultaneously. Then, for all customers, predictions for all customers are derived via formula (2) by setting the treatment for all customers to 1 in the first term and 0 in the second term.

Over the years, a remarkably high number of uplift modeling approaches, including algorithms to estimate their parameters, have been proposed. Table 1 gives an overview. As one can easily see, most of them aim at predicting uplifts for binary outcomes, e.g., indicators for a visit, conversion, or purchase. Here, logistic regression or decision trees can be applied to estimate model parameters. However, more recently, also revenue uplift modeling approaches have become popular (Gubela et al. 2020; Rudaś and Jaroszewicz 2018). The main idea behind this new development is that the revenue uplift more closely relates to economic objectives than a website visit or purchase uplift. However, in the next section, we will see that even revenue uplift modeling approaches sort customers suboptimally.

Another interesting aspect in Table 1 is that most recent uplift modeling approaches rely on transformed outcomes for parameter estimation. This transformation was introduced for binary outcomes in a seminal paper (Lai 2006) and later extended to continuous outcomes (Gubela et al. 2020; Rudaś and Jaroszewicz 2018). The main idea behind it is to transfer as much information as possible from the observed responses in the two groups into the dependent variable and so being able to directly estimate the uplift model parameters (a so-called direct model). So, e.g., Rudaś and Jaroszewicz (2018)—following the proposal of Lai (2006) for binary outcomes—proposed to estimate their revenue uplift model, directly using transformed revenue outcomes, for parameter estimation. \(q^{T}\) and \(q^{C}\) are the fractions of the treatment group and the control group in the customer sample. Rudaś and Jaroszewicz (2018) discuss in their paper that this weighting facilitates unbiased estimation of the model parameters when relying on linear models. The main idea behind the positive weighting of the observed revenues in the treatment sample and the negative weighting of the observed revenues in the control sample is that so the best possible information is forwarded to parameter estimation. It is assumed that the purchasers in the treatment group generate probably a (low to high) positive revenue uplift. Likewise, it is assumed that the purchasers in the control group generate probably a (low to high) negative revenue uplift.

Another major problem with uplift modeling approaches is to validate their predictions at the customer level as for these predictions—as mentioned above—no observations exist. The widespread solution for this problem is to develop so-called Qini curves and calculate the so-called Qini coefficient Q (Radcliffe 2007; Radcliffe and Surry 2011): the customers are sorted according to a descending (uplift) score and partitioned into deciles (or other partitions of the customers) with similar scores. Then, within the deciles, customer responses from the treatment group are averaged as well as customer responses from the control group. The difference of these two means is assumed to be the “observed” uplift in this decile. Figure 1 shows the typical results for such a validation applied to (uplift) scores from a sample dataset.

In both diagrams, the customers are sorted according to descending uplift predictions from left to right and grouped in deciles. In the right diagram, the calculated average (“observed”) uplift per decile is given, as discussed above. In the left diagram, from decile to decile, the average cumulative uplift is plotted, which means that for the first decile, the values in the left and right diagram are identical, but from then, aggregated values up to the current decile are given in the left diagram. The last value (here: 0.045) of this so-called Qini curve reflects the uplift across all ten deciles (all customers of the treatment and the control group) which is identical for all scorings based on the data. For comparisons, also the Qini curve for a random sorting (the random uplift model) is plotted in the left diagram. Its incremental uplift curve connects the zero point with the average uplift across all customers, the last value (0.045). The quality of an uplift model is judged by its ability to sort customer deciles according to decreasing “observed” uplifts (in the right diagram) but—similar to ROC curves—also by calculating the area between the Qini curve and the line for the random model (in the left diagram). Here the length of the x-axis is assumed to be 1. In Fig. 1, this value—the so-called Qini coefficient Q—is 0.0296 and could serve for comparisons with other uplift models (other sortings of customers according to their predicted scores). The random model has Q = 0, the maximum is data-dependent. It should be noted that these two diagrams can be generated for uplift models with binary response outcomes but also for uplift models with continuous outcomes (as in revenue uplift modeling or our new profit uplift modeling approach discussed in the following section).

In order to demonstrate that the new profit uplift modeling approaches are applicable and superior to traditional response and uplift modeling approaches, also a standard dataset from the uplift modeling literature is analyzed, the Hillstrom dataset (Radcliffe 2008). This dataset was made available by Kevin Hillstrom through his MineThatData blog and contains a sample of 64,000 customers which had been divided up into three nearly equally sized subsamples, two of them contacted via two direct marketing campaigns and one not contacted, serving as a control group (see the similar usage of this dataset, e.g., in Rudaś and Jaroszewicz 2018). Table 7 summarizes the descriptive uplift statistics of this dataset, where the two treated subsamples are merged. As can easily be seen, the conversion rate is much lower as in the BAUR dataset (on average 1.07% in the treatment group) but nevertheless shows a conversion rate uplift compared to the control group (on average, an uplift of 0.50%). The revenue uplift per customer is 0.60$, but this uplift seems to be arising solely from the conversion rate uplift since the average revenue spend by a purchaser in the treatment group (117.00$) is only slightly higher than in the control group (114.00$). Again, as in the BAUR dataset, we assume that the campaign offers a 20% discount and that the margin for the retailer is 30%. With these assumptions (not part of the original communication of the dataset, just an assumption to be able to analyze the dataset with our profit uplift modeling approaches), the overall profit uplift per customer is negative (− 0.07$). So, again we have to develop a scoring system that helps to restrict the direct marketing campaign to customers with a positive profit uplift prediction.

The original dataset also contains potential predictors for this scoring system, as given in Table 8. The original eight potential predictors (in Table 8 described as variable categories) were scaled nominally (e.g., history_segment with 7 values or channel with three values) or metrically (e.g., recency or history). For our further analysis with the three models, we dummy-coded the nominally scaled potential predictors and so received in total 25 metrically scaled variables (see Table 8).

As in Sect. 4, the customers were randomly partitioned into a train set (~ 70% or 44,800 customers) and a holdout test set (~ 30% or 19,200 customers), and the train set was preprocessed by setting means to zero, setting standard deviations to 1, and applying a Box–Cox-transformation to transform skew distributed variables into “normal shape”. The same preprocessing was applied to the test set, using the transformation parameters derived from the train set. Then, five models, similar as in Sect. 4, were applied: 50 subsamples (6/7) of the train set were drawn randomly, the models were calibrated, and the Qini curves and Qini coefficients were averaged. Figure 7 and Table 9 reflect the results of this modeling and prediction task. It should be mentioned that we only use a subsample of the models applied in Sect. 4 but the selection contains the “best” models from this comparison (e.g. especially the three “winners” OLS and RF direct models and Zeroinfl interaction model).

One can easily see that the three “best” profit uplift modeling approaches (one-stage OLS and RF as well as two-stage Zeroinfl), again, show similar results with random forest providing the best performance. But it should be mentioned that—maybe due to the few purchasers in the dataset with a high concentration of revenues and profits from few purchasers—the modeling leads to a worse performance compared to the application of the BAUR dataset. This problem of the Hillstrom dataset when it comes to modeling continuous outcomes has also been discussed by other authors in context of revenue uplift modeling; here we refer to the analysis in the paper by Rudaś and Jaroszewicz 2018).

In this paper, we demonstrated the usefulness of new profit uplift modeling approaches for direct marketing campaigns: the main idea is to contact a sample of customers testwise. Then, profit uplifts are modeled and the model is used to make predictions across all customers. Finally, these predictions are used to decide whether a customer should be contacted.

In contrast to former approaches, the proposed new approaches model profit uplift at the individual level and do not need an unrelated second step to transform modeled binary outcomes or revenues to profits. Various algorithms can be applied to estimate the model parameters. On the one side, two-stage algorithms especially tackle the problem of low rates of purchasers (in many cases: zero revenues and profits). So, the Heckman sample selection model separately models the observed binary outcome (purchase or not) and the related observed continuous outcome (profits for the treatment group, profits with negative sign for the control group in the direct model case, profits for both groups in the two model approach). Zero-inflated negative binomial or hurdle models—as an alternative—assume count data and therefore need an interaction model for estimation and prediction. On the other side, one-stage algorithms like OLS and RF performed surprisingly well, especially when applied to transformed profits.

The proposed profit uplift modeling approaches are based on very different assumptions but nevertheless provide quite similar predictions with a clear ordering of the customers according to their predicted profit uplift. The results support the meaningfulness of the approaches via cross-validation as the main contribution of this paper. Also, an extensive comparison with response models and revenue uplift models using two large datasets supports this superiority of the new approaches. Of course, further research is needed. So, e.g., the presented profit uplift modeling approaches have to demonstrate their usefulness also with other datasets.