Figure 1: IT3FBLS structure
These layers together compute the output of the model to achieve the regression task. The detailed implementation description of each layer is as follows.
Algorithm 1 IT3FBLS modeling algorithm |
|
Input: Training data {X, Y}; Initial model parameters; Regularization parameter; Learning rate; Maximum iteration number |
|
Output: Model prediction output |
|
(1). Initialization |
Compute the outputs of the fuzzy subsystems P and enhancement nodes H // ~ |
Concatenate P and H to construct matrix Q, and calculate the initial model prediction output |
|
(2). Calculate information entropy and regularity coefficient |
For each column of Q , estimate the PDF using KDE and calculate the corresponding information entropy // ~ |
Apply the exponential mapping function to transform the entropy into regularization weights and construct the diagonal matrix |
|
(3). Update |
Update the output layer weights |
(4). Gradient descent for internal parameters: |
Compute the gradient of loss function with respect to |
(5). Gradient-based secondary optimization |
After updating |
Update the output layer weights again using the updated |
|
Repeat Steps 2 through 5 until the maximum number of iterations is reached. |
|
(6). Output results |
After IT3FBLS training is completed, the model prediction output |
Algorithm 1 The pseudocode of the proposed IT3FBLS model is as follows:
Datasets |
Details of datasets |
No. of Samples |
Input variables |
||
Train |
Validation |
Test |
|||
1 |
Concrete compressive strength (CCS) |
515 |
257 |
258 |
8 |
Abalone |
2088 |
1044 |
1045 |
8 |
|
Housing |
253 |
126 |
127 |
13 |
|
2 |
Mackey–Glass |
400 |
200 |
200 |
4 |
Rossler attractor |
500 |
250 |
250 |
3 |
|
3 |
MSWI process |
428 |
214 |
215 |
5 |
TE chemical process |
250 |
125 |
125 |
11 |
|
Table 1: Dataset details for regression modeling
Datasets |
Hyperparameters of IT3FBLS |
|||||
J |
K |
M |
L |
|
|
|
Range |
[1,100] |
[1,50] |
[1,1000] |
[1,2050] |
[0,1] |
[0,1] |
CCS |
66 |
7 |
460 |
140 |
0.7 |
0.07 |
Abalone |
8 |
4 |
835 |
176 |
0.3 |
0.2 |
Housing |
48 |
11 |
271 |
490 |
0.45 |
0.4 |
Mackey–Glass |
8 |
40 |
20 |
800 |
0.3 |
0.3 |
Rossler attractor |
6 |
4 |
10 |
800 |
0.3 |
0.3 |
MSWI process |
42 |
6 |
125 |
1610 |
0.22 |
0.16 |
TE chemical process |
4 |
40 |
10 |
600 |
0.3 |
0.2 |
Table 2:Hyperparameter settings of the proposed algorithms on different datasets
Datasets |
Methods |
RMSE |
MAE |
R2 |
||||||
Mean |
Var |
Best |
Mean |
Var |
Best |
Mean |
Var |
Best |
||
Housing |
BLS |
1.8988E+01 |
1.7532E+01 |
4.9911E+00 |
5.0214E+00 |
1.7552E+00 |
3.3301E+00 |
-6.5651E+00 |
1.2706E+01 |
7.1119E-01 |
FNN |
6.1635E+00 |
9.4208E-01 |
5.4973E+00 |
4.0787E+00 |
6.4398E-01 |
3.6233E+00 |
5.5443E-01 |
1.3463E-01 |
6.4964E-01 |
|
FBLS |
5.9531E+00 |
1.1426E+00 |
4.7884E+00 |
3.9587E+00 |
3.9816E-01 |
3.3284E+00 |
5.7475E-01 |
1.6724E-01 |
7.8698E-01 |
|
IT2FNN |
4.1984E+00 |
4.1219E-01 |
3.5039E+00 |
3.0080E+00 |
1.9918E-01 |
2.6240E+00 |
7.9378E-01 |
4.1478E-02 |
8.5766E-01 |
|
IT2FBLS |
6.3649E+00 |
7.3477E-01 |
4.9374E+00 |
4.5117E+00 |
5.0624E-01 |
3.5953E+00 |
5.2438E-01 |
1.0990E-01 |
7.1737E-01 |
|
IT3FBLS-1 |
6.2516E+00 |
7.3677E-01 |
5.5300E+00 |
4.6600E+00 |
4.8271E-01 |
4.1938E+00 |
5.4123E-01 |
1.1426E-01 |
6.4546E-01 |
|
IT3FBLS-2 |
6.1518E+00 |
7.0482E-01 |
4.9612E+00 |
4.1655E+00 |
5.0065E-01 |
3.8651E+00 |
5.8664E-01 |
1.0812E-01 |
6.8520E-01 |
|
IT3FBLS |
5.8991E+00 |
3.6990E-01 |
4.2865E+00 |
3.8528E+00 |
2.2710E-01 |
3.2322E+00 |
6.0470E-01 |
4.7700E-02 |
7.9050E-01 |
|
CCS |
BLS |
3.5018E+01 |
3.8531E+01 |
1.0920E+01 |
9.9500E+00 |
4.3984E+00 |
6.3229E+00 |
-7.8145E+00 |
2.3569E+01 |
6.0136E-01 |
FNN |
1.2610E+01 |
6.2004E-01 |
1.1921E+01 |
8.8290E+00 |
9.0309E-01 |
8.0701E+00 |
4.6742E-01 |
5.3011E-02 |
5.2493E-01 |
|
FBLS |
9.9873E+00 |
1.0259E+00 |
8.1331E+00 |
7.5818E+00 |
8.2169E-01 |
6.0590E+00 |
6.6320E-01 |
6.8675E-02 |
7.4887E-01 |
|
IT2FNN |
8.3817E+00 |
3.6140E-01 |
7.4174E+00 |
6.3209E+00 |
2.7262E-01 |
5.5798E+00 |
5.2650E-01 |
1.9941E-02 |
7.1607E-01 |
|
IT2FBLS |
1.2468E+01 |
1.3018E+00 |
9.3524E+00 |
9.6935E+00 |
1.0329E+00 |
7.3395E+00 |
4.7491E-01 |
1.0449E-01 |
7.0760E-01 |
|
IT3FBLS-1 |
1.3538E+01 |
2.8278E-01 |
1.3235E+01 |
1.1004E+01 |
1.9383E-01 |
1.0824E+01 |
3.8710E-01 |
2.5980E-02 |
4.1439E-01 |
|
IT3FBLS-2 |
1.3860E+01 |
7.2619E-01 |
1.3084E+01 |
1.1275E+01 |
5.5857E-01 |
1.0557E+01 |
3.5637E-01 |
6.7368E-02 |
4.2767E-01 |
|
IT3FBLS |
2.4234E+00 |
1.5360E-01 |
2.1406E+00 |
1.8176E+00 |
1.2040E-01 |
1.6300E+00 |
7.6473E-01 |
5.9300E-02 |
7.3200E-01 |
|
Abalone |
BLS |
8.2271E+00 |
1.1380E+01 |
2.6158E+00 |
2.6303E+00 |
6.5151E-01 |
1.9711E+00 |
-1.1315E+01 |
4.3493E+01 |
5.5819E-01 |
FNN |
2.9319E+00 |
3.5560E-02 |
2.9068E+00 |
2.2705E+00 |
5.0842E-02 |
2.2346E+00 |
4.4492E-01 |
1.3464E-02 |
4.5444E-01 |
|
FBLS |
2.7202E+00 |
2.7033E-01 |
2.3118E+00 |
1.9126E+00 |
1.3661E-01 |
1.7431E+00 |
5.1777E-01 |
9.9518E-02 |
6.5492E-01 |
|
IT2FNN |
2.4935E+00 |
1.7485E-01 |
2.3083E+00 |
1.8871E+00 |
1.2127E-01 |
1.7381E+00 |
5.9677E-01 |
5.8821E-02 |
6.5598E-01 |
|
IT2FBLS |
2.3091E+00 |
1.3151E-01 |
2.1305E+00 |
1.7435E+00 |
1.1133E-01 |
1.5805E+00 |
6.5468E-01 |
3.9673E-02 |
7.0694E-01 |
|
IT3FBLS-1 |
2.6090E+00 |
1.2095E-01 |
2.4669E+00 |
1.9028E+00 |
1.1005E-01 |
1.7313E+00 |
5.5973E-01 |
4.0807E-02 |
6.0707E-01 |
|
IT3FBLS-2 |
2.3936E+00 |
6.6976E-02 |
2.3062E+00 |
1.7696E+00 |
4.6055E-02 |
1.7170E+00 |
6.2984E-01 |
2.0760E-02 |
6.5660E-01 |
|
IT3FBLS |
2.3975E+00 |
1.0250E-01 |
2.2039E+00 |
1.6459E+00 |
1.0730E-01 |
1.4590E+00 |
5.3750E-01 |
3.9600E-02 |
6.0990E-01 |
|
Table 3:Performance comparison results of the testing set data on the benchmark dataset
Datasets |
Methods |
RMSE |
MAE |
R2 |
||||||
Mean |
Var |
Best |
Mean |
Var |
Best |
Mean |
Var |
Best |
||
Mackey |
BLS |
2.9764E-01 |
3.9052E-01 |
3.3935E-02 |
2.2310E-02 |
2.7710E-02 |
3.7310E-03 |
2.5341E+01 |
9.2931E-01 |
9.8257E-01 |
FNN |
2.3121E-02 |
6.6905E-03 |
1.2152E-02 |
1.8167E-02 |
4.8734E-03 |
9.7597E-03 |
9.9130E-01 |
4.7095E-03 |
9.9776E-01 |
|
FBLS |
2.4116E-03 |
1.5833E-04 |
2.2007E-03 |
1.9114E-03 |
1.1725E-04 |
1.7442E-03 |
9.9861E-01 |
1.2254E-05 |
9.9913E-01 |
|
IT2FNN |
4.3339E-02 |
8.2000E-03 |
3.0138E-02 |
3.2803E-02 |
5.5523E-03 |
2.2987E-02 |
9.7060E-01 |
1.1591E-02 |
9.8625E-01 |
|
IT2FBLS |
2.3594E-02 |
1.0772E-02 |
1.1484E-02 |
1.8609E-02 |
8.4352E-03 |
9.9888E-03 |
9.8991E-01 |
9.7716E-03 |
9.9800E-01 |
|
IT3FBLS-1 |
8.7532E-02 |
4.6528E-03 |
7.9216E-02 |
7.0162E-02 |
4.2087E-03 |
6.2543E-02 |
8.8373E-01 |
1.2219E-02 |
9.0502E-01 |
|
IT3FBLS-2 |
1.4820E-02 |
7.8595E-04 |
1.3528E-02 |
1.1452E-02 |
6.0698E-04 |
1.0386E-02 |
9.9667E-01 |
3.5497E-04 |
9.9723E-01 |
|
IT3FBLS |
1.3573E-02 |
3.8224E-04 |
1.2991E-02 |
1.0227E-02 |
2.7178E-04 |
9.8179E-03 |
9.9721E-01 |
1.5743E-04 |
9.9845E-01 |
|
Rossler system |
BLS |
5.1297E-01 |
3.2984E-03 |
4.1499E-01 |
6.4590E-02 |
2.0603E-04 |
3.6554E-04 |
9.4233E+01 |
1.7868E-04 |
9.7453E+00 |
FNN |
1.4850E-01 |
2.8354E-01 |
1.8088E-01 |
5.9146E-02 |
8.9973E-02 |
5.1575E-03 |
9.9580E-01 |
1.2641E-02 |
9.9129E-01 |
|
FBLS |
1.7019E-03 |
3.1604E-03 |
1.5816E-06 |
7.4489E-04 |
2.3595E-03 |
1.0278E-06 |
9.9911E-01 |
1.5678E-04 |
9.9929E-01 |
|
IT2FNN |
1.1028E+00 |
2.1280E-01 |
5.1417E-01 |
7.4501E-01 |
1.2732E-01 |
3.7827E-01 |
9.4394E-01 |
2.0560E-02 |
9.8823E-01 |
|
IT2FBLS |
5.7999E-01 |
6.0014E-01 |
1.6707E-01 |
3.4652E-01 |
3.9376E-01 |
8.3696E-02 |
9.6979E-01 |
7.5461E-02 |
9.9876E-01 |
|
IT3FBLS-1 |
1.7675E+00 |
2.2979E-01 |
1.5419E+00 |
1.4358E+00 |
2.5266E-01 |
1.1590E+00 |
8.5878E-01 |
3.7874E-02 |
8.9415E-01 |
|
IT3FBLS-2 |
2.0261E-01 |
2.2748E-02 |
1.7888E-01 |
7.7464E-02 |
2.2375E-03 |
6.3005E-02 |
9.9815E-01 |
4.3197E-04 |
9.9858E-01 |
|
IT3FBLS |
2.0795E-01 |
5.9879E-02 |
1.2618E-01 |
5. 3515E-02 |
1.0391E-02 |
4.4466E-02 |
9.9793E-01 |
1.2413E-03 |
9.9929E-01 |
|
Table 4:Performance comparison reslts of the testing set in the uncertain function dataset
|
Source of uncertainty |
Sampling interval (min) |
Input variable |
Output variable |
MSWI process |
Fuel composition fluctuations, air supply, combustion dynamics, strong nonlinear combustion, turbulent heat transfer, operating condition fluctuations |
1 |
Primary air volume, secondary air volume, average feeder speed, average drying grate speed, ammonia injection volume |
Furnace temperature |
TE process |
Reaction kinetics, heat exchange, material transport, multi-step chemical reactions, thermodynamics, fluid transport, transport failures and abnormal conditions |
3 |
D feed rate (stream 2), E feed rate (stream 3), A feed rate (stream 1), total feed rate (stream 4), compressor recirculation valve, discharge valve (stream 9), separator tank liquid flow (stream 10), stripper liquid product flow (stream 11), stripper water flow valve, reactor cooling water flow, condenser cooling water flow |
Reactor temperature |
Table 5:Uncertainty sources, sampling intervals, and process variable descriptions for the MSWI process and the TE process
Datasets |
Methods |
RMSE |
MAE |
R2 |
||||||
Mean |
Var |
Best |
Mean |
Var |
Best |
Mean |
Var |
Best |
||
MSWI |
BLS |
4.2869E+01 |
1.9537E+01 |
1.8329E+01 |
1.5747E+01 |
3.7545E+00 |
1.1383E+01 |
5.0442E+00 |
5.3205E+00 |
7.7202E-02 |
FNN |
1.8251E+01 |
1.2451E-01 |
1.8123E+01 |
1.4488E+01 |
1.4933E-01 |
1.4272E+01 |
8.4970E-02 |
1.2505E-02 |
9.7762E-02 |
|
FBLS |
1.3577E+01 |
5.2206E-01 |
1.2591E+01 |
1.0832E+01 |
4.4640E-01 |
1.0033E+01 |
4.9293E-01 |
3.8997E-02 |
5.6452E-01 |
|
IT2FNN |
1.4292E+01 |
3.6553E-01 |
1.3677E+01 |
1.1703E+01 |
3.7454E-01 |
1.0961E+01 |
4.3853E-01 |
2.8698E-02 |
4.8619E-01 |
|
IT2FBLS |
1.6738E+01 |
4.4930E-01 |
1.6093E+01 |
1.3856E+01 |
2.8086E-01 |
1.3419E+01 |
2.2992E-01 |
4.1974E-02 |
2.8854E-01 |
|
IT3FBLS-1 |
1.0180E+01 |
1.6538E-01 |
9.9034E+00 |
7.7655E+00 |
8.6324E-02 |
7.5849E+00 |
7.1528E-01 |
9.2605E-03 |
7.3059E-01 |
|
IT3FBLS-2 |
9.9189E+00 |
1.5445E-01 |
9.7150E+00 |
7.5013E+00 |
1.3604E-01 |
7.2075E+00 |
7.2968E-01 |
8.4652E-03 |
7.4074E-01 |
|
IT3FBLS |
9.1893E+00 |
1.7770E-01 |
8.6750E+00 |
7.2749E+00 |
1.8865E-01 |
7.0219E+00 |
7.4554E-01 |
2.0913E-02 |
7.6339E-01 |
|
TE process |
BLS |
5.9468E+01 |
1.2131E+02 |
4.7453E+00 |
9.2506E+00 |
1.1582E+01 |
2.5239E+00 |
-1.1923E+04 |
4.4483E+04 |
-1.4329E+01 |
FNN |
1.0766E+00 |
6.7992E-02 |
9.9546E-01 |
8.5259E-01 |
5.8838E-02 |
7.9559E-01 |
2.0840E-01 |
1.0168E-01 |
3.2542E-01 |
|
FBLS |
1.1011E+00 |
4.5375E-02 |
1.0186E+00 |
8.7610E-01 |
2.7352E-02 |
8.2791E-01 |
1.7336E-01 |
6.7758E-02 |
2.9376E-01 |
|
IT2FNN |
1.0602E+00 |
6.8572E-02 |
9.7473E-01 |
8.3604E-01 |
5.4041E-02 |
7.6247E-01 |
2.3175E-01 |
1.0123E-01 |
3.5323E-01 |
|
IT2FBLS |
1.1221E+00 |
6.5875E-02 |
9.8712E-01 |
9.1175E-01 |
5.2059E-02 |
8.1825E-01 |
1.4005E-01 |
9.9557E-02 |
3.3667E-01 |
|
IT3FBLS-1 |
1.1244E+00 |
4.1835E-02 |
1.0725E+00 |
9.2102E-01 |
3.6988E-02 |
8.7056E-01 |
1.3816E-01 |
6.4728E-02 |
2.1698E-01 |
|
IT3FBLS-2 |
1.0650E+00 |
4.2266E-02 |
1.0090E+00 |
8.7492E-01 |
3.8107E-02 |
8.1693E-01 |
1.8214E-01 |
6.2344E-02 |
3.0688E-01 |
|
IT3FBLS |
1.1102E+00 |
8.1063E-02 |
9.4379E-01 |
8.8026E-01 |
5.8782E-02 |
7.4087E-01 |
1.5670E-01 |
1.2226E-01 |
3.9363E-01 |
|
Table 6:Performance comparison results of the testing set in complex industrial process data sets
Figure 1: IT3FBLS structure
These layers together compute the output of the model to achieve the regression task. The detailed implementation description of each layer is as follows.
Figure 2: Prediction results of the testing set for the benchmark datase
Figure 3: Prediction results of the testing set of uncertain function dataset
Figure 4:MSWI process flow diagram
Figure 5:Structure diagram of Tennessee Eastman process
Figure 6:Prediction results of the testing set of complex industrial process data sets
Figure 7:Parameter sensitivity of IT3FBLS
Tables at a glance
Figures at a glance