1 Introduction
2 Background
2.1 Particle Swarm Optimization (PSO)
2.2 Genetic Algorithm (GA)
3 Problem Explanation, Material, and Methods
3.1 PSO and GA Improvement Methods
3.1.1 Hybrid Particle Swarm Optimization (HPSO)
3.1.2 Hybrid Particle Swarm Optimization with Dragonfly (PSO–DA)
3.1.3 Improved Genetic Algorithm (IGA)
3.1.4 Modified Genetic Algorithm (MGA)
3.2 Optimize ANFIS with PSO and GA Variants
3.2.1 Initialization
3.2.2 Individual Evaluation
4 Discussion of Results
Methods | Parameters | Values |
---|---|---|
GA, IGA | Mutation rate | 0.33 |
Crossover rate | 0.67 | |
PSO, HPSO, PSODA | Inertia weight | 2 |
Best global experience | 2.2 | |
Best personal experience | 2.4 | |
w-damp | 0.98 | |
Bat | Minimal and maximal values of the acoustic frequency (\([F_{\max } ,F_{\min } ]\)) | [1,− 1] |
Pulse intensity attenuation coefficient (\(\gamma\)) | 0.95 | |
Pulse frequency increase factor (\(\delta\)) | 0.05 | |
Maximum pulse frequency (\(R^{0}\)) | 0.75 | |
Maximum pulse loudness (A) | 0.25 | |
BBO | Habitat modification probability | 1 |
Immigration probability limits | [0,1] | |
Step size | 1 | |
Max immigration (I) and Max emigration (E) | 1 | |
Mutation probability | 0.005 |
4.1 General Experimental Setting
-
Unimodal problems (i.e., F1–F5).
-
Multimodal problems (benchmark).
Function | Dim | Range | \(f_{\min }\) |
---|---|---|---|
\(F_{1} (x) = \sum\limits_{i = 1}^{n} {x_{i}^{2} }\) | 10 | [− 100,100] | 0 |
\(F_{2} (x) = \sum\limits_{i = 1}^{n} {\left| {x_{i} } \right|} + \prod\limits_{i = 1}^{n} {\left| {x_{i} } \right|}\) | 10 | [− 10,10] | 0 |
\(F_{3} (x) = \sum\limits_{i = 1}^{n} {(\sum\limits_{j - 1}^{i} {x_{j} } )^{2} }\) | 10 | [− 100,100] | 0 |
\(F_{4} (x) = \mathop {\max }\limits_{i} \{ \left| {x_{i} } \right|,\,\,\,\,1 \le i \le n\}\) | 10 | [− 100,100] | 0 |
\(F_{5} (x) = \sum\limits_{i = 1}^{n - 1} {[100(x_{i + 1} - x_{i}^{2} ) + (x_{i} - 1)^{2} ]}\) | 10 | [− 30,30] | 0 |
\(F_{6} (x) = \frac{1}{4000}\sum\limits_{i = 1}^{n} {x_{i}^{2} } - \prod\limits_{i = 1}^{n} {\cos (\frac{{x_{i} }}{\sqrt i })} + 1\) | 10 | [− 600,600] | 0 |
\(F_{7} (x) = \left( {\frac{1}{500} + \sum\limits_{j = 1}^{25} {\frac{1}{{j + \sum\limits_{i = 1}^{2} {\left( {x_{i} - a_{ij} } \right)^{6} } }}} } \right)^{ - 1}\) | 2 | [− 65.53,65.53] | 1 |
\(F_{8} (x) = \sum\limits_{i = 1}^{11} {\left[ {a_{i} - \frac{{x_{1} \left( {b_{i}^{2} + b_{i} x_{2} } \right)}}{{b_{i}^{2} + b_{i} x_{3} + x_{4} }}} \right]^{2} }\) | 4 | [− 5,5] | 0.0003 |
\(F_{9} (x) = 4x_{1}^{2} - 2.1x_{1}^{4} + \frac{1}{3}x_{1}^{6} + x_{1} x_{2} - 4x_{2}^{2} + 4x_{2}^{4}\) | 2 | [− 5,5] | − 1.0316 |
\(\begin{gathered} F_{10} (x) = \left[ {1 + \left( {x_{1} + x_{2} + 1} \right)^{2} \left( {19 - 14x_{1} + 3x_{1}^{2} - 14x_{2} + 6x_{1} x_{2} + 3x_{2}^{2} } \right)} \right] \hfill \\ \quad \quad \quad \times \left[ {30 + \left( {2x_{1} - 3x_{2} } \right)^{2} \times \left( {18 - 32x_{1} + 12x_{1}^{2} + 48x_{2} - 36x_{1} x_{2} + 27x_{2}^{2} } \right)} \right] \hfill \\ \end{gathered}\) | 2 | [− 2,2] | 3 |
4.1.1 Analysis of Compared Methods on Benchmark Functions
Test functions | F1 | F2 | F3 | F4 | F5 | |||||
---|---|---|---|---|---|---|---|---|---|---|
Metric | AVG | STD | AVG | STD | AVG | STD | AVG | STD | AVG | STD |
GA | 4.19E−07 | 2.11E−07 | 2.96E−04 | 4.68E−05 | 2.91E−10 | 3.31E−10 | 4.73E−07 | 5.16E−07 | 6.18E−02 | 5.81E−02 |
PSO | 1.88E−08 | 4.11E−08 | 3.71E−06 | 1.88E−06 | 3.85E−12 | 4.15E−13 | 5.13E−10 | 2.19E−11 | 4.89E−02 | 2.69E−03 |
IGA | 3.17E−07 | 1.06E−08 | 2.11E−04 | 4.33E−04 | 1.63E−08 | 3.28E−09 | 1.59E−08 | 4.49E−08 | 7.50E−03 | 1.18E−04 |
MGA | 1.08E−10 | 3.27E−10 | 4.13E−10 | 2.18E−11 | 5.22E−18 | 1.87E−18 | 4.28E−13 | 2.05E−14 | 7.11E−03 | 4.10E−04 |
PSODA | 2.12E−14 | 1.90E−15 | 5.08E−15 | 3.44E−15 | 7.18E−21 | 5.36E−22 | 3.38E−12 | 4.19E−13 | 5.27E−03 | 5.76E−03 |
HPSO | 4.26E−05 | 4.39E−05 | 5.28E−04 | 6.18E−04 | 5.39E−05 | 2.17E−05 | 2.88E−05 | 7.23E−05 | 5.29E−02 | 2.14E−02 |
Bat | 3.15E−04 | 7.29E−04 | 1.43E−03 | 5.32E−04 | 3.14E−03 | 3.12E−03 | 3.95E−04 | 6.29E−04 | 8.66E−02 | 1.03E−02 |
BBO | 4.26E−10 | 1.93E−10 | 4.55E−08 | 1.17E−08 | 5.07E−15 | 4.89E−15 | 8.45E−10 | 5.61E−10 | 4.76E−03 | 3.37E−03 |
Test functions | F6 | F7 | F8 | F9 | F10 | |||||
---|---|---|---|---|---|---|---|---|---|---|
GA | 2.62E−13 | 3.07E−13 | 1.4348 | 3.22E−01 | 0.0003 | 3.21E−06 | − 1.0317 | 3.29E−09 | 3.0018 | 4.11E−02 |
PSO | 6.02E−14 | 2.40E−15 | 1.4902 | 2.69E−01 | 0.0003 | 1.09E−13 | − 1.0318 | 4.86E−21 | 3.0000 | 6.30E−07 |
IGA | 6.25E−14 | 4.84E−14 | 1.2075 | 1.39E−01 | 0.0003 | 2.19E−08 | − 1.0316 | 1.09E−04 | 3.0015 | 5.38E−03 |
MGA | 0 | 0 | 9.9E−01 | 3.23E−08 | 0.0003 | 6.38E−19 | − 1.0317 | 2.18E−07 | 3.0000 | 5.03E−20 |
PSODA | 0 | 0 | 9.9E−01 | 3.18E−10 | 0.0003 | 4.11E−11 | − 1.0316 | 6.37E−18 | 3.0000 | 2.68E−10 |
HPSO | 3.27E−10 | 1.92E−10 | 1.5724 | 5.50E−01 | 0.0006 | 2.25E−05 | − 1.0318 | 5.03E−10 | 3.1000 | 1.003 |
Bat | 4.18E−12 | 3.23E−12 | 1.8349 | 3.89E−01 | 0.0076 | 1.94E−06 | − 1.0318 | 4.89E−07 | 3.1024 | 5.25E−01 |
BBO | 0 | 0 | 9.81E−01 | 4.28E−06 | 0.0003 | 3.29E−04 | − 1.0316 | 9.23E−01 | 3.0958 | 1.0183 |
Test functions | F1 | F2 | F3 | F4 | F5 | |||||
---|---|---|---|---|---|---|---|---|---|---|
Metric | AVG | STD | AVG | STD | AVG | STD | AVG | STD | AVG | STD |
GA | 0.0054 | 2.19E−04 | 0.0094 | 4.37E−06 | 0.0007 | 1.82E−12 | 0.0046 | 4.52E−04 | 0.0002 | 6.78E−05 |
PSO | 0.0052 | 4.28E−05 | 0.0089 | 4.87E−09 | 0.00074 | 1.05E−09 | 0.0033 | 3.79E−04 | 0.0031 | 5.38E−04 |
IGA | 0.0052 | 3.80E−18 | 0.0152 | 2.49E−11 | 0.00071 | 5.61E−12 | 0.0035 | 2.83E−05 | 0.0245 | 5.76E−05 |
MGA | 0.0045 | 1.76E−07 | 0.0007 | 3.98E−06 | 0.0007 | 4.79E−10 | 0.0018 | 5.46E−05 | 0.0197 | 2.59E−03 |
PSODA | 0.0039 | 5.38E−04 | 0.0008 | 7.30E−18 | 0.0006 | 4.02E−06 | 0.0018 | 3.11E−04 | 0.0011 | 2.42E−04 |
HPSO | 0.0062 | 2.88E−06 | 0.0128 | 4.18E−03 | 0.00088 | 5.29E−08 | 0.0047 | 2.56E−08 | 0.0024 | 4.95E−04 |
Bat | 0.0070 | 1.69E−05 | 0.0174 | 5.31E−03 | 0.00095 | 3.18E−06 | 0.0057 | 1.57E−03 | 0.1729 | 2.88E−02 |
BBO | 0.0050 | 5.22E−13 | 0.0011 | 1.70E−06 | 0.0007 | 2.89E−09 | 0.0020 | 1.89E−06 | 0.0239 | 6.88E−04 |
Test functions | F6 | F7 | F8 | F9 | F10 | |||||
---|---|---|---|---|---|---|---|---|---|---|
GA | 0.0043 | 3.49E−05 | 0.0012 | 2.68E−06 | 0.3804 | 4.20E−01 | 0.0042 | 1.19E−04 | 0.0010 | 7.20E−07 |
PSO | 0.0082 | 4.39E−04 | 0.0018 | 3.19E−09 | 0.0005 | 5.07E−12 | 0.0065 | 5.22E−06 | 0.0010 | 7.44E−07 |
IGA | 0.0086 | 5.09E−05 | 0.0025 | 2.33E−04 | 0.1253 | 2.48E−01 | 0.0089 | 4.51E−07 | 0.0010 | 5.18E−08 |
MGA | 0.0001 | 2.95E−06 | 0.0008 | 3.36E−10 | 0.2239 | 4.37E−01 | 0.0005 | 5.11E−13 | 0.0006 | 4.87E−14 |
PSODA | 0.0001 | 2.99E−06 | 0.0008 | 5.73E−13 | 0.0003 | 8.65E−12 | 0.1544 | 1.72E−01 | 0.0006 | 4.06E−14 |
HPSO | 0.0012 | 4.11E−04 | 0.0029 | 4.20E−08 | 0.0007 | 5.38E−15 | 0.0079 | 7.37E−04 | 0.0010 | 3.95E−05 |
Bat | 0.0078 | 4.77E−05 | 0.0037 | 2.75E−06 | 0.4107 | 1.08E−01 | 0.1563 | 1.09E−01 | 0.0011 | 3.49E−05 |
BBO | 0.0094 | 5.68E−04 | 0.0006 | 4.29E−11 | 0.1864 | 5.62E−01 | 0.0003 | 6.37E−09 | 0.0006 | 2.76E−09 |
Parameter | Value |
---|---|
Number of input | 10 |
Layer 1: number of membership function | 140 |
Layer 2: number of nodes | 140 |
Layer 3: number of nodes | 140 |
Layer 4: number of rules | 18 |
Layer 5: number of output | 1 |
Membership function | Gaussian |
4.1.2 Actual Train–Predict Train
4.1.3 Errors
Method | Train | Test | ||
---|---|---|---|---|
Error mean | Error Std | Error mean | Error Std | |
GA | 0.4586 | 6.9268 | 0.2699 | 5.6155 |
PSO | 0.1002 | 6.1799 | 0.0288 | 4.1593 |
HPSO | − 0.2043 | 6.4002 | − 0.3394 | 3.9592 |
PSO–DA | − 0.0552 | 5.6300 | − 0.1582 | 2.6873 |
IGA | − 0.1410 | 6.7674 | − 0.2814 | 5.1517 |
MGA | 0.1779 | 6.9048 | − 0.0728 | 5.5206 |
Bat | 0.7239 | 6.3980 | 0.2750 | 4.4894 |
BBO | − 0.1175 | 6.0749 | 0.2107 | 5.1976 |
4.2 Comparison of Other Methods
Methods | MAPE | Efficiency | R2 | VAF | MEDAE |
---|---|---|---|---|---|
GA | 16.70 | 0.82 | 0.56 | 0.59 | 0.83 |
PSO | 15.59 | 0.86 | 0.63 | 0.72 | 0.33 |
HPSO | 11.15 | 0.89 | 0.75 | 0.77 | 0.51 |
PSO–DA | 1.67 | 0.97 | 0.92 | 0.93 | 0.14 |
IGA | 16.35 | 0.84 | 0.58 | 0.66 | 0.62 |
MGA | 15.15 | 0.83 | 0.59 | 0.62 | 0.77 |
Bat | 18.67 | 0.78 | 0.58 | 0.58 | 0.80 |
BBO | 12.07 | 0.87 | 0.79 | 0.77 | 0.48 |
4.3 PSO–DA ANFIS Information
Rule numbers | Rules |
---|---|
1 | \(\begin{aligned} C_{11(1)} = & - 0.1155 C_{1} - 0.0471C_{2} - 0.0394C_{3} + 0.3621C_{4} - 0.1798C_{5} \\ &\quad + 0.3021C_{6} - 0.0451C_{7} - 0.11C_{8} + 0.0098C_{9} - 3.4646C_{10} + 45.1552 \\ \end{aligned}\) |
2 | \(\begin{aligned} C_{11(2)} = & - 0.0595 C_{1} - 0.0327 C_{2} - 0.0395 C_{3} + 0.4252 C_{4} - 0.2229C_{5} \\ &\quad + 0.6232 C_{6} - 0.0445 C_{7} - 0.2647C_{8} + 0.01C_{9} - 3.4798 C_{10} + 45.0383 \\ \end{aligned}\) |
3 | \(\begin{aligned} C_{11(3)} = & - 0.0447 C_{1} - 0.0478 C_{2} - 0.0393 C_{3} + 0.3049 C_{4} - 0.1772C_{5} \\ &\quad + 0.2875 C_{6} - 0.0462C_{7} - 0.1382C_{8} - 0.2404C_{9} - 2.8651 C_{10} + 36.2256 \\ \end{aligned}\) |
4 | \(\begin{aligned} C_{11(4)} = & - 0.0605 C_{1} - 0.0476 C_{2} - 0.0408 C_{3} + 0.3528 C_{4} - 0.1644 C_{5} \\ &\quad + 0.3031 C_{6} - 0.0433 C_{7} - 0.1854 C_{8} + 0.0116 C_{9} - 3.6469 C_{10} + 45.5615 \\ \end{aligned}\) |
5 | \(\begin{aligned} C_{11(5)} = & - 0.0613 C_{1} - 1.2013 C_{2} - 0.0007 C_{3} + 0.2849 C_{4} - 0.1646 C_{5} \\ & &\quad + 0.3281 C_{6} - 0.0481 C_{7} - 0.1984 C_{8} + 0.0254 C_{9} - 3.8826 C_{10} + 46.3025 \\ \end{aligned}\) |
6 | \(\begin{aligned} C_{11(6)} = & - 0.0061 C_{1} - 0.0703 C_{2} - 0.0409 C_{3} + 0.3506 C_{4} - 0.1718_{5} \\ &\quad + 0.3024 C_{6} - 0.0513 C{}_{7} - 0.1488 C_{8} + 0.0112 C_{9} - 3.3384 C_{10} + 44.5699 \\ \end{aligned}\) |
7 | \(\begin{aligned} C_{11(7)} = & - 0.0614 C_{1} - 0.3094 C_{2} - 0.0384 C_{3} + 0.1063 C_{4} - 0.0015 C_{5} \\ &\quad + 0.3002 C_{6} - 0.0454 C_{7} - 0.1612 C_{8} + 0.0087 C_{9} - 3.7786 C_{10} + 43.5307 \\ \end{aligned}\) |
8 | \(\begin{aligned} C_{11(8)} = & - 0.0591 C_{1} - 0.1390 C_{2} - 0.0395 C_{3} + 0.3467 C_{4} - 0.1637C_{5} \\ &\quad + 0.3469 C_{6} - 0.0451 C_{7} - 0.1699 C_{8} + 0.0091 C_{9} - 3.5540 C_{10} + 43.3374 \\ \end{aligned}\) |
9 | \(\begin{aligned} C_{11(9)} = & - 0.0599 C_{1} - 0.0495 C_{2} - 0.0944C_{3} + 0.3234 C_{4} - 0.1250 C_{5} \\ &\quad + 0.2858 C_{6} - 0.0468 C_{7} - 0.0953 C_{8} + 0.0083 C_{9} - 3.5426 C_{10} + 44.1564 \\ \end{aligned}\) |
10 | \(\begin{aligned} C_{11(10)} = & - 0.0539 C_{1} - 0.0367C_{2} - 0.0383 C_{3} + 0.3950 C_{4} - 0.1626 C_{5} \\ &\quad + 0.3528 C_{6} - 0.0450 C_{7} - 0.1556 C_{8} + 0.0330 C_{9} - 3.6017C_{10} + 50.91 \\ \end{aligned}\) |
11 | \(\begin{aligned} C_{11(11)} = & - 0.0397C_{1} - 0.0467C_{2} - 0.0194 C_{3} + 0.3538 C_{4} - 0.1689 C_{5} \\ &\quad + 0.2494 C_{6} - 0.0398 C_{7} - 0.1519 C_{8} + 0.0107C_{9} - 3.5988 C_{10} + 39.5035 \\ \end{aligned}\) |
12 | \(\begin{aligned} C_{11(12)} = & - 0.0721 C_{1} - 0.0497C_{2} - 0.0318 C_{3} + 0.3429 C_{4} - 0.1622 C_{5} \\ &\quad + 0.2888 C_{6} - 0.0822 C_{7} - 0.1854 C_{8} + 0.0070 C_{9} - 3.7603 C_{10} + 44.7857 \\ \end{aligned}\) |
13 | \(\begin{aligned} C_{11(13)} = & - 0.0592 C_{1} - 0.0453 C_{2} - 0.0401 C_{3} + 0.3538 C_{4} - 0.1591 C_{5} \\ &\quad + 0.2577 C_{6} - 0.0433 C_{7} - 0.2697C_{8} + 0.0147 C_{9} - 3.5458 C_{10} + 40.6151 \\ \end{aligned}\) |
14 | \(\begin{aligned} C_{11(14)} = & - 0.0605 C_{1} - 0.0494 C_{2} - 0.0410 C_{3} + 0.3982 C_{4} - 0.1391 C_{5} \\ &\quad + 0.3033 C_{6} - 0.0451 C_{7} - 0.1764 C_{8} + 0.0098 C_{9} - 3.5827C_{10} + 42.2834 \\ \end{aligned}\) |
4.3.1 Investigation of PSO–DA ANFIS Performance
Number of membership function | MAPE | Efficiency | R2 | RMSE |
---|---|---|---|---|
5 | 3.56 | 0.85 | 0.84 | 4.44 |
8 | 3.60 | 0.85 | 0.84 | 4.51 |
11 | 2.09 | 0.91 | 0.90 | 3.52 |
14 | 1.67 | 0.97 | 0.92 | 2.68 |
18 | 1.88 | 0.94 | 0.89 | 2.93 |
Method | Population size | Iteration | ||||||
---|---|---|---|---|---|---|---|---|
25 | 50 | 75 | 100 | 100 | 200 | 300 | 400 | |
MAPE | 11.76 | 1.67 | 1.75 | 1.71 | 14.83 | 11.52 | 7.37 | 1.67 |
Efficiency | 0.85 | 0.97 | 0.95 | 0.95 | 0.82 | 0.86 | 0.91 | 0.97 |
R2 | 0.87 | 0.92 | 0.92 | 0.91 | 0.85 | 0.86 | 0.85 | 0.92 |
RMSE | 3.43 | 2.68 | 2.70 | 2.69 | 4.68 | 4.09 | 3.71 | 2.68 |