Supervised learning to support the process planning of contract logistics projects

Planning data of 13 contract logistics projects are available for this research work. The data set consists of six customers from the automotive and construction industries.

The data set includes the description of 13,707 process steps, each identified with an MTM code. The process steps are grouped in 440 processes. For example, one process describes the goods receipt of a pallet, while the storage of this pallet represents a new process. The shortest process includes only one MTM code, and the longest contains 155 MTM codes. The average is about 31 MTM codes per process. The MTM codes can be divided into 256 unique MTM codes. The aim of the supervised learning method is to suggest the most probable next process step with its associated MTM code to the planer. In this way, the planer should be supported during contract logistics process planning.

To determine what information is used to plan contract logistics processes, workshops were conducted with planners. In these workshops, the planners explained which information they use to plan the processes. This information must also be provided to the machine learning classifier so that the classifier can learn the dependencies between this information and the MTM code sequences. This information is referred to as features in the following. Table 2 lists these features and highlights those, which are included in the data set and, thus, are available for supervised learning. For example, information on load carrier dimensions, item specifications, or process flows (e.g., storage method) is missing either due to this information being unknown at the time of planning or because this information is implicitly available to the planners but is not documented. So it could not be captured automatically when creating the data set. The discrepancy of the comparison reveals the missing information and consequently complicates the application of supervised learning in this case.

The process steps and MTM codes to be planned depend on the process specification (see #1 in Table 2). In the data set, nine categories of main process (1.1) could be distinguished. These main process categories are: Goods receipt, storage, removal from storage, conveying, commissioning, sequencing, pre-assembly, repacking, and dispatching.

In addition, it is relevant which equipment (1.2) is used in the process. For example, different MTM codes are used for a forklift than for a pallet jack. In the data set were 26 different categories of equipment counted. The employee group (e.g., forklift driver, booker, packer, assembly operator) listed as 1.3 specifies what is to be done in a process (e.g., booker books, packer packs). The IT system (1.4) specifies how goods are to be booked. Supply means (1.5) defines the means of delivery by which the goods are received or shipped. For example, the unloading of a truck differs significantly from the delivery of a courier and express provider service provider. Storage and removal from storage (1.6–1.9) depend on the stacking situation, which storage type (e.g., block storage, high-rack storage) is used, whether the goods are stored directly (1-level-storage) or transferred to a stacker crane (2-level-storage), and which storage method (e.g., first in, first out) is used. The features mentioned above are process specific. To avoid missing data, the value “irrelevant” is inserted when a feature is irrelevant for decision-making in a process, e.g., in storage processes, the supply means are irrelevant.

The factory or warehouse layout (2) influences whether and how many drives around curves must be planned, which is to be planned in the MTM code sequence.

The article specifications (3) also influence the MTM code selection. The handling of piece goods and bulk goods (3.2) is different. Particular goods, such as food (3.3) or hazardous material (3.4), have special safety regulations that must be considered. The dimensions (3.5) and weight (3.6) of an item influence the handling during picking and pre-assembly. In addition, customs clearance (3.7) affect the process of goods receipt and goods issue. Since some of these characteristics are unavailable, the article name (e.g., axle, curing part 182, or set 112 engine cover) was used as category (3.1).

The load carriers (4) also influence the process. The load carrier type (4.1) describes which load carrier it is (e.g., pallet, mesh box, AKL container, and package). If no load carrier is specified, a single piece is handled. In addition, processes partly differ in whether a load carrier is loaded with full goods, mixed goods, or bulky goods or whether it is empty (4.2). The dimensions (4.3) and weight (4.4) of the load carrier and the possibility of stacking it (4.5) influence the process design as well.

Further influences arise from special customer requirements, location specifications, and industry specifications. For this reason, the customer name (5), location (6), and industry (7) industry were taken into account in the present data set.

In addition, it is essential at which position of the process a new process step is to be added. Hence, previous process steps (8) must be taken into account. This contribution explores how many previous process steps should be considered. Since there are no or only a few predecessors for the first process steps, the features “Up to n previous MTM codes” (pMTMC) were generated (8.1–8.9). Table 3 shows what “Up to n” means based on the example process mentioned in Table 1. This analysis examines up to nine predecessors.

Figure 1 displays the histogram of the 256 unique MTM codes used to identify the process steps, with logarithmic scaling of the y-axis. For better readability, only every sixth MTM code is written on the abscissa. The figure shows a strong imbalance of the MTM code classes. The most frequent MTM code is used 1371 times. The rarest MTM code is used only once. This leads to the problem that rare MTM codes can either be learned but not used, or not learned at all. To have enough samples for train the classifier, all MTM codes occurring currently less than seven times were removed from the data set. This reduces the amount of different MTM codes to 156. The data set decreases from 13,707 samples to 13,432 samples.

The aim is to predict the most probable next MTM code, so the classifier has to choose from the 256 possible MTM codes (multi-class). Since the classes vary in frequency (see Fig. 1) and only categorical features are currently available (see Table 2), predicting the next process step is an imbalanced multi-class classification problem with categorical data. The following section describes the method used to deal with this problem.

The used methods were coded in Python using the scikit-learn library [21], version 1.2.1, if not otherwise noted. Figure 2 illustrates the proceeding of supervised learning in this study.

To check the plausibility of the results, the feature combinations (FC) in the data set are analyzed in more detail. The analysis takes advantage of the categorical nature of all features used. Due to this categorical nature, a classifier can only make a meaningful prediction based on a known feature combination. A transfer to unknown feature combinations is not possible with the methods used. An example shall clarify this: The classifier knows how unloading pallets from a truck proceeds. It cannot transfer this knowledge to unloading packages from a truck.

The following cases may occur in the data set:

A large fraction of case (a) and case (c1) is useful for classification because these cases are well classifiable. Since the FC corresponding to case (c2) are not classifiable, the fraction of (c2) must be kept as low as possible. Similarly, the fraction of cases (b) should be kept as low as possible. However, in these cases, a correct classification can be achieved by chance or if the classifier consider only a feature subset.

This section examines the models and the predecessor features. The complete training data set with 10,745 samples and 156 different MTM codes is used. The nine different “up to” features are tested together with the previously selected features s₀. For this purpose, a cross-validation was performed on the training data set for each of the nine feature combinations. Tables 4, 5 and 6 list the exact values for the ROC AUC score as well as for the ACC including the standard deviations (SD) across the 5 folds in the cross-validation for the three different models.

The DT classifier and the RF achieve the best ROC AUC score with up to 3 pMTMC and the best ACC with up to 6 (DT) or up to 7 pMTMC (RF), see Tables 4 and 5. The XG classifier requires up to 4 pMTMC to achieve its best ROC AUC score and up to 7 pMTMC to reach its best ACC, see Table 6.

Each classifier achieves higher ROC AUC scores than ACC scores. This is quite common, as the ROC AUC score is usually between 0.5 and 1 and the ACC score between 0 and 1. In this case, the high amount of classes and the imbalance of the classes increase the difference.

Figure 3 compares the ROC AUC score of the three classifiers depending on the feature combination. It depicts that the XG classifier achieves the best and most stable ROC AUC scores of the three compared classifiers independent of the feature combination. This implies using the XG classifier and the feature up to 4 pMTMC for further steps.

As with the ROC AUC score, the XG classifier also achieves the highest scores for ACC. Comparing the ACC scores over all classifiers, it is noticeable that the ACC increases more strongly with the addition of predecessors as the ROC AUC score and reaches its maximum at up to 6 or 7 pMTMC.

To check the plausibility of these results, the feature combinations are evaluated. Figure 4 illustrates the grouping of the FC according to the approach described in Sect. 3.2.

Figure 4 shows that as the number of predecessors increases, the size of the must-have group initially increases, and the size of the non-distinguishable group decreases. This effect is positive and results in increasing scores. From up to 3 pMTMC, however, the must-have group starts to shrink. Although the non-distinguishable group continues to shrink, only the group of unique FC group grows. This fragmentation is disadvantageous due to the categorical nature of the features. The trend of growing unique FC levels out similar to the scores of ROC AUC and ACC. This emphasizes the plausibility of the results.

The conclusion of these results is, on the one hand, that the XG Classifier achieves the best results for this data set. On the other hand, considering up to 4 pMTMC is sufficient. Considering more predecessors may improve the ACC even slightly. However, this effect is probably due to the imbalance of the data set. Taking too many predecessors into account leads to a higher fragmentation of the FC and, thus, to lower scores.

In the next step, a grid search with cross-validation is performed to optimize the hyper-parameters of the XG Boost classifier. Table 7 lists the grid and the best parameters.

The best configuration (maximal depth = 11 and 200 estimators) leads to the best ROC AUC score of 0.9812 and an ACC of 0.7901 in the training data set. Consequently, a minor improvement could be achieved through optimization.

Finally, the XG classifier is evaluated with the selected feature combination and the optimized hyper-parameters on the previously unused test data set. Figure 5 plots the result of this evaluation and compares the test results with the training results. The ACC reaches a score of 0.8091 and the ROC AUC score is 0.9752 which means a good model fit. Furthermore, the test results are very similar to the scores achieved in training. Surprisingly, the ACC score on the test data is a bit higher than on the training data. Thus, a suitable generalization of the model can be assumed.

Assuming that the classifier suggests a list of probable MTM codes, the top k accuracy is calculated, where k is the length of the list. For example, assuming the XG classifier should suggest a list of the five most probable MTM Codes, the top 5 accuracy indicates if the correct MTM code is in this list. Figure 6 shows the top k accuracy from k = 1 to 10.

The top one accuracy is precisely the accuracy mentioned before, with 0.809. The more extensive the list of proposed MTM codes, the higher the probability that the correct MTM code is listed. For example, the top five accuracy is 0.933.

Machine learning was performed on a Lenovo Think Pad P16s with an Intel Core i7-1260P and 32 GB of RAM. Training the optimized model took 77.39 seconds. Predicting all the test data took 0.28 seconds, giving 0.0001 seconds per prediction.