Real-time bidding campaigns optimization using user profile settings

Advertising Networks act as an intermediary between advertisers and publishers. The main tasks of ANs include buying and selling impressions, displaying adverts on the publishers’ websites, and preventing fraud on the publicity ecosystem [1]. ANs have been a popular choice for some time but, a few years ago, a new sophisticated auction-based model called Real-time bidding (RTB) emerged. RTB offers important advantages over the AN model; it directly connects advertisers with publishers, eliminating intermediaries and their respective commissions, and increasing the cost-benefit ratio of both sides [2].

Real-time bidding is a real-time auction platform where buying and selling online impressions take place instantly via programmable criteria [3]. These bidding and winning processes are executed approximately within the time it takes a user to load a page (100 ms). Additionally, RTB can target audiences based on interests and profiles by examining the HTTP cookies stored on the users’ browsers [4].

There are different strategies to optimize online campaigns but some of them are more suitable than others depending on the platform. Optimizing online campaigns has been a very interesting line of research from the beginnings of online advertising. For example, Li Zhiwei et al. [5] proposed a solution to monetize the time it takes a full-resolution image to be displayed to show users an advert stored in a small file related to the image. Spotting this gap and proposing a solution to monetize it, could bring huge benefits to the advertising networks.

In the same line, other researchers realized that live streaming videos were an advertising niche that was not fully optimized [6]. Traditional video algorithms such as pre-roll and context mid-roll advertising worked well with the content on the network that could be fully processed which requires the content to be previously stored. However, for content that had to be analysed in real-time the performance was poor, this fact brought the opportunity for developing a new methodology for monetizing live stream videos called LiveSense. LiveSense implements a deep neural network trained with historical values to display a non-intrusive contextual advert at the right moment increasing the monetizing capacity of live stream videos.

Real-time bidding (RTB) optimization differs from other platforms such as search engine optimization in that advertisers bid based on individual impressions rather than on particular keywords [3]. To optimise an RTB platform, several factors are analysed such as the duration of the campaign, the preferences of each advertiser segment, the competitors’ bidding strategies, the reserve price of the publishers, or the number of networks [4].

Predicting the optimal bid price for each impression is one of the most recurring optimization techniques in the literature related to RTB campaigns. It is a very challenging problem because, if the price is too low, it is unlikely that the advertiser gets the bid and if the price is too high, the advertiser may pay in excess [7]. Additionally, advertisers prefer their adverts to be displayed uniformly over time to reach a more diverse audience [8].

One of the main benefits of RTB is that it allows advertisers to target specific audiences who are assumed to become receptive towards the marketing of a product [9]. This strategy of directing campaigns to a small group of people having common interests is useful and is known as microtargeting [10].

Liu et al. [11] showed that by combining using user features with metadata from web pages, it is possible to estimate which users are more likely to generate a conversion. Additionally, it is also possible to create models to predict the probability of new users accessing new web pages [11]. However, making appropriate matchmaking is not a simple task, and even small improvements, when applied to billions of transactions per day, turns into huge economic benefits [12].

Liu et al. [11] optimizes RTB campaigns through text mining and the selection of attributes from the users and web pages. In their approach, they optimize the online campaign based on the results. The approach detects the profile of the users more willing to convert based on machine learning models and the results of the campaign during its execution. Additionally, it implements text mining and natural language processing techniques.

The main difference between the research of Liu et al. [11] and ours is that they use the metadata from the advertiser’s campaigns and apply text mining techniques. They also use the results of the live campaigns to tune some of the parameters of the model. They are also specialized in small campaigns rather than in analyzing lots of data from multiple campaigns. Our approach does not apply text mining and is based purely on the parameters (numerical and categorical) from the users and the websites where the adverts are displayed. Additionally, we apply different approaches to refine and improve parameter optimization.

In this paper, we present a methodology focused on finding parameter configurations that maximise the number of visits and also the average conversion probability for those visits. It is similar to one of our previous publications but this time it is focused on RTB campaigns [10]. Additionally, the parameter configuration technique is combined with other approaches such as multiple configurations, discarding visits based on cost and profitability thresholds, configuration extrapolation, and increasing the search space, which is a natural continuation of the previous paper. The combination of multiple approaches with the configuration optimization increases, even more, the obtained performance.

Our goal consists of detecting configurations (sets of parameters from users and web attributes, along with their values) for suggesting them to advertisers so that they can launch their campaigns more effectively. An example of parameter configuration could be: {Browser = “Chrome”, Device OS = “Android”, Age = “25–34”, Gender = “Male”, Time = “10:00–11:00”, Country = “UK”}.

The major contributions of the proposed work are:The rest of the paper is organised as follows: In Sect. 3 a description of the Conversion rate estimator is presented, which calculates the probability of a conversion from a visit. Similarly, in Sect. 4 another important element of the proposed methodology is introduced: the Quality Score Calculator. This module evaluates how good a configuration is based on the number of visits and their profitability. In Sect. 5, experiments are conducted to evaluate the different strategies for improving optimization based on parameters. Finally, Sect. 7 presents the conclusions and some directions for future work.

RTB allows publishers to connect with multiple ANs and advertising agencies, making it a flexible ecosystem. Additionally, RTB eliminates the need for commissions or fees to AN by allowing advertisers to directly buy impressions from the publisher. Impressions can be bought individually, in a fine-grained fashion, which leads to more effective campaigns [13].

RTB has a hierarchical bidding system where impressions are offered through several subsystems and where the highest price is selected. When a bid is won by an advertiser, the advert is displayed on the publisher’s website [3].

As shown in Fig. 1, two modules define the RTB ecosystem: the demand-side platform (DSP), representing the advertisers’ interests, and the supply-side platform (SSP), representing the publishers’ interests. The DSP manages advertisers and ANs campaigns efficiently, bidding directly on the auctions. To select the best advert, the DSP implements Machine Learning models to estimate the probability of generating a conversion or a click from a given advert [14, 15]. On the other hand, the SSP aims at managing and optimising publishers’ web spaces. The SSP distributes the information of a publisher across multiple platforms whenever a user visits a website and selects the most cost-effective advert [16].

Our methodology falls on the side of the DSP module; the demand side of the platform. Its purpose is to identify configurations to make online campaigns in RTB more effective by selecting the right values for the users and websites parameters [10]. The overall architecture of the proposed methodology is shown in Fig. 2. The most important modules: the \(Conversion\,rate\) (CVR) estimator and the \(Quality\,Score\) calculator (QSC) are described in detail in the next sections.

In this section, we describe the Conversion Rate Estimator (CVR) estimator module. The CVR module calculates the conversion probability (the probability that a conversion is generated when an advert is displayed to a user) for each visit based on historical data. The CVR module is implemented using a supervised model based on logistic regression.

To build the CVR we used a database of display adverts, where each row represents a displayed advert on the website of a publisher. The dataset contains attributes related to the user, the publisher’s web page, the campaign identifier, the indicator of whether the conversion was generated, and the price paid for an impression.

Equation 1 shows the formulation of the profitability of an impression using the CVR and the \(Impression\,_{Price}\). The \(Impression\,_{Price}\) is the price of the impression whose value is extracted from the dataset. We implemented the supervised model to estimate the CVR value proposed in [14, 17]. According to Eq. 1, the profitability is higher for those impressions that have a high probability of generating a conversion (numerator) and a low price (denominator).We used a train/test split of 6.3 M and 10 M to build and validate the performance of the model. The dataset has a total of ten columns, out of which nine are categorical (cat1, ..., cat9) and form the input variables of the machine learning model \(e_i, i = 1,2,\ldots 9\), and the final column is Profitability, which is the target or the output variable of the model y. The output, y, represents the probability that a conversion (a purchase, a call, the filling of a form...) takes place. For instance, a predicted value of 0.225 means that the model estimates that the user has 22.5% chance of generating a conversion from the displayed advert. The model is used to expressed p, where \(p = P(y \vert e_i, \forall i = 1,2,\ldots 9)\).

The CVR model is built using a well-known methodology for making predictions based on a Logistic Regression model [18, 19]. We configured the adaptive learning rate value by following the criteria defined by Brendan McMahan et al. [20]. In their investigation, they explain that the value of alpha, the heuristic adaptive learning rate of the model, highly depends on the nature of the dataset and the type of data. In our particular case, we used several parameters and chose the one that gave the best performance. The results and performance of the logistic regression algorithm are presented in Sect. 5.2. To create the model, we used a simple but effective technique called hashing trick. The hashing trick is used to reduce the sparsity of the values of the dataset, not the number of features in the dataset [19].

A typical dataset for representing users’ visits has N different values for each category, where N can be bigger than the length of the arrays n and w. The hashing trick divides the values of N by D, where D = length(w) = length(n), so the sparsity of the values will be at maximum D. Initially, all of the elements of both vectors are set to zero. Afterwards, the elements of n and w are updated using Eqs. 2 and 3.Once the vectors w and n have been updated, we apply Eq. 4 to estimate using a sigmoid function the output of the model for each impression.Where each variable is described as follows:Once the model and its variables have been explained, we define algorithm 1, which summarises the implementation for both training and testing of the CVR.

The second module used in our methodology (see Fig. 2) is the Quality Score Calculator (QSC) and ranks how good a configuration is according to a defined metric called \(Quality\,Score\). The \(Quality\,Score\) rewards configurations that presumably have high profitability (sets of visits with a high probability of conversion at a low price) and also guarantee a sufficient number of impressions for the advertiser’s campaign. The \(Quality\,Score\) is calculated for all the possible configurations without repetition and its main purpose is to score all of the possible configurations (i.e. combinations of different attributes).

The QSC module makes use of all the attributes of the impressions and optimises them to achieve the highest performance. Firstly, QSC explores all the possible configurations with a single attribute, then, it continues with two attributes, then with three, and so on until it finally reaches all the combinations with all the attributes. All configurations are ranked according to \(Quality\,Score\) metric defined by Eq. 6. Equation 5 shows the formula to calculate the total number of combinations. The higher-ranked configurations scored by the QSC are offered to advertisers to increase the performance of their campaigns.To begin calculating the value of a configuration, we first estimate the average profitability. To this end, we calculate the profitability (using Eq. 1) for all the impressions that fit the configuration, and then, we calculate the average. Even though the configurations may have some unprofitable impressions, these impressions are offset by the rest of the impressions making the metric robust. Furthermore, we multiply the \(Profitability\,_{Average}\) by the minimum between the number of impressions that satisfy the configuration and the number of visits required by the advertiser, i.e., \(\min (rows(D'),\,T)\). The T represents the set threshold, and it is used to avoid converging to solutions with an excessive number of visits. We should bear in mind that we are looking for configurations with a sufficient number to cover the number required by the advertiser rather than configurations with the highest possible number of visits.The following lines give an example of how the process of calculating the \(Quality\,Scores\) for a combination of attributes works. Let’s imagine that we are testing the combination of columns (2, 5, 8). First, we select the second, fifth, and eighth attribute from the dataset. The result will be a subset with three selected attributes from the original dataset. From this subset, we select the unique records (i.e., rows) and calculate the \(Quality\,Scores\) for each record. As shown in Fig. 3, the first unique values of the subset are: 9,312,274, 582,437 and 9,312,274 respectively. Then, we proceed with the subset called D’, where the second, the fifth and the eighth attribute have values 9,312,274, 582,437 and 9,312,274 respectively, are extracted from the original dataset. For this subset D’, where D’ \(\subseteq\) D, the \(Quality\,Score\) is calculated using Eq. 6.

To speed up the algorithm, we implemented a dynamic programming approach creating a list of all the configurations that do not have enough visits. Then, we added the following condition: if a configuration of a campaign is a subset of a previously discarded configuration, then this new configuration gets rejected as well. As can be seen in Fig. 4 more intuitively, such action was taken on the basis that a subset cannot be greater than the set that contains it. For example, for attributes (1, 3, 4, 7) with values (458, 47, 58, 58) respectively, we check if the list of configurations with not enough visits contains any combination without repetition of these attributes with precisely these same values. To do so, we first check in the list, combinations of one element (attribute 1 with value 458, attribute 3 with value 47, and so on).

Later, we look in the list for combinations of two elements (attribute 1 with value 458 and attribute 3 with value 47, attribute 3 with value 47, attribute 4 with value 58, and so on), and the same technique is applied for the combinations of three elements. This simple improvement significantly reduces the running time to find out the optimal solution and reduces the time to explore all the configurations from 75 h when this improvement is not applied to 193.82 s when it is. Having got to this point, we define algorithm 2 to select the optimal configuration given the requirements of the advertisers.

In this section, we carry out some experiments to evaluate the yield of the proposed methodology. First, we preprocess the dataset. Secondly, we build the model to estimate the CVR value, then we evaluate the built model using well-known metrics by predicting the CVR of the RTB impressions. Next, we apply the algorithm to find the optimal configurations. Finally, we discuss each of the five experiments to demonstrate the different approaches in the context of attribute selection. All the proposed experiments have in common the following steps: (1) Building the CVR estimation model, (2) Predicting the CVR for all impressions, (3) Generating all possible campaign configurations and calculating the Quality Score, (4) Return the best solution for the given dataset. The steps “Calculate the Quality Score” and “Build the CVR estimation model” have been explained in detail in the previous Sects. 3 and 4).

To carry out the experiments, we used Python 3.6.1 and Anaconda runtime (x86_64). We performed all experiments using a MacBook computer having macOS High Sierra (2.3 GHz Intel Core i5, 8GB 2133 MHz, L2 Cache:256kb, L3: 4MB).

To measure the improvement of the proposed method, we have conducted five different experiments based on parameter optimisation combined with other ideas as discussed in the previous section. We have performed the experiments using 17 datasets, and the obtained results are the summation of the average profitability of all the datasets. The designed algorithm collects some parameters that indicate the performance of the campaign. This table shows the collected information in experiment II for the first of the 17 datasets. Other variables such as \(Quality\,Score\) or the remaining size of the dataset can be calculated from these values.

RTB platforms are becoming a very beneficial advertising model for publishers and advertisers. It is no wonder that the estimated volume of impressions managed by RTB networks will continue to increase in the coming years. It does not seem unreasonable to say that shortly RTB can replace advertising network platforms or at least take significant market share.

In this paper, we propose a novel methodology based on parameter configuration to find profitable campaigns for advertisers in an automatic fashion. In this sense, we think that the proposed approach is interesting because, to our knowledge, it covers a gap in RTB campaign optimisation research.

The developed experiments prove that the presented methodology improves the results of RTB campaigns in a substantial way. Moreover, the combination of parameter optimisation with other approaches such as small campaign selection, setting a threshold, configuration extrapolation, or increasing the solution search space, improves the obtained results even more. However, these results may vary depending on several variables of a campaign such as a target audience, the moment when it is launched, or the behaviour of the rest of the competitors.

Implementing the methodology would enable RTB platforms as well as advertising networks that manage third-party campaigns to provide advertisers with better configurations for their campaigns. Profitable campaigns will eventually boost the performance of advertising companies, making RTB a more attractive platform, which in turn will make RTB advertisers more willing to launch more campaigns.

It could seem that, as the selected number of configurations increases, the gain of the average profit by campaign becomes trivial. But, here, we argue that, first, in RTB many ad networks coexist with their advertisers, therefore, when a platform decides not to bid on a particular impression, it does not imply that it is lost, but instead, that it will be disputed among the rest of the advertisers. Additionally, it may be the case that some impressions may have a low probability conversion for an advertiser but a high probability for other advertisers as it depends on the nature of the advert. Secondly, there are two types of campaigns: those based on branding and others based on performance. Impressions not valuable from a performance-based perspective could be valuable for branding-based campaigns; where the goal is to increase the brand value instead of looking for profits in the short term.

For future work, it could be a good idea to combine several techniques. For example, setting an economic threshold with small campaign selection and increasing the search space. The new methodology could be tested in different scenarios with other payment methods such as pay-per-click or pay-per-acquisition. Using algorithms to find a suboptimal solution but in less time could also be a good starting point. To this end, multi-objective evolutionary algorithms such as NSGA-II (Non-dominated Sorting Genetic Algorithm) could be a good solution.