A lightweight knowledge-based PSO for SVM hyper-parameters tuning in a dynamic environment

Hyper-parameter optimization is a crucial task for designing kernel-based machine learning models. Their values can be set by using various optimization algorithms. But a data-dependent objective function makes hyper-parameter’s configuration changes over time in a dynamic environment. A dynamic environment is an environment where training data keep getting added continuously over time. To find the optimum values for the hyper-parameters in such an environment, one needs to run the optimization algorithm repeatedly over time. But due to the dependency of the objective function on data, the average time complexity of the optimization process increases. This paper work proposes a novel knowledge-based approach that uses particle swarm optimization (PSO) as the base optimization algorithm to optimize the hyper-parameters of support vector machine. We have introduced two major modules for designing this framework—a knowledge transfer module and a drift detection module. The knowledge transfer module in our proposed framework generates knowledge by running PSO and transfers this knowledge to the consequent time instances. On the other hand, the drift detection module is responsible for detecting changes in the objective function when new data get added to the existing data. This drift detection module helps to utilize the transferred knowledge at a particular time instance to reduce the execution time of the overall optimization process. The proposed framework has been evaluated using various standard datasets such as Adult, DNA, Nist-Digits, Segment, Splice, Mushroom and Usps for five consecutive time instances and found the average execution time as 30.72 s, which is better than the execution time 36.89 s recorded using general PSO. We have also found that our proposed framework performs the optimization of hyper-parameters much faster than the other existing approaches such as grid search, chained-PSO, dynamic model selection and quantized dynamic multi-PSO.