Abstract

This article proposes a genetic algorithm (GA)-enhanced weight optimization method for temporal convolutional network (TCN-GAWO). TCN-GAWO combines the evolutionary process of the genetic algorithm with the gradient-based training and can achieve higher predication/fitting accuracy than traditional temporal convolutional network (TCN). Performances of TCN-GAWO are also more stable. In TCN-GAWO, multiple TCNs are generated with random initial weights first, then these TCNs are trained individually for given epochs, next the selection-crossover-mutation procedure is applied among TCNs to get the evolved offspring. Gradient-based training and selection-crossover-mutation are taken in turns until convergence. The TCN with the optimal performance is then selected. Performances of TCN-GAWO are thoroughly evaluated using realistic engineering data, including C-MAPSS dataset provided by NASA and jet engine lubrication oil dataset provided by airlines. Experimental results show that TCN-GAWO outperforms existing methods for both datasets, demonstrating the effectiveness and the wide range applicability of the proposed method in solving time series problems.

Graphical Abstract Figure
Graphical Abstract Figure
Close modal

References

1.
Lea
,
C.
,
Vidal
,
R.
,
Reiter
,
A.
, and
Hager
,
G. D.
,
2016
, “Temporal Convolutional Networks: A Unified Approach to Action Segmentation,”
Computer Vision—ECCV 2016 Workshops
,
Amsterdam, The Netherlands
,
Oct. 8–10 and 15–16
,
G.
Hua
and
H.
Jégou
, eds.,
Springer International Publishing
, pp.
47
54
.
2.
Zeng
,
J.
,
Zhao
,
Y.
,
Li
,
G.
,
Gao
,
Z.
,
Li
,
Y.
,
Barbat
,
S.
, and
Hu
,
Z.
,
2024
, “
Vehicle Crashworthiness Performance Prediction Through Fusion of Multiple Data Sources
,”
ASME J. Mech. Des.
,
146
(
5
), p.
051707
.
3.
Alqahtani
,
S.
,
Mishra
,
A.
, and
Diab
,
M.
,
2019
, “
Efficient Convolutional Neural Networks for Diacritic Restoration
,”
Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)
,
Hong Kong, China
,
Nov. 5–7
, Association for Computational Linguistics, pp.
1442
1448
.
4.
Zhang
,
Y.
,
Ren
,
K.
,
Zhang
,
C.
, and
Yan
,
T.
,
2022
, “
SG-TCN: Semantic Guidance Temporal Convolutional Network for Action Segmentation
,”
2022 International Joint Conference on Neural Networks (IJCNN)
,
Padua, Italy
,
July 18–23
, pp.
1
8
.
5.
Pascanu
,
R.
,
Mikolov
,
T.
, and
Bengio
,
Y.
,
2013
, “
On the Difficulty of Training Recurrent Neural Networks
,”
Proceedings of the 30th International Conference on Machine Learning
,
Atlanta, GA
,
June 17–19
,
S.
Dasgupta
and
D.
McAllester
, eds.,
PMLR
, Vol. 28, pp.
1310
1318
.
6.
Rumelhart
,
D. E.
,
Hinton
,
G. E.
, and
Williams
,
R. J.
,
1986
, “
Learning Representations by Back-Propagating Errors
,”
Nature
,
323
(
6088
), pp.
533
536
.
7.
De
,
S.
,
Maute
,
K.
, and
Doostan
,
A.
,
2021
, “
Reliability-Based Topology Optimization Using Stochastic Gradients
,”
Struct. Multidiscipl. Optim.
,
64
(
5
), pp.
3089
3108
.
8.
Bottou
,
L.
,
2010
, “
Large-Scale Machine Learning With Stochastic Gradient Descent
,” Proceedings of COMPSTAT’2010: 19th International Conference on Computational Statistics, Paris France, Aug. 22–27, 2010 Keynote, Invited and Contributed Papers,
Springer
, pp.
177
186
.
9.
Li
,
Z.
,
Zhou
,
F.
,
Chen
,
F.
, and
Li
,
H.
,
2017
, “Meta-sgd: Learning to Learn Quickly for Few-Shot Learning” arXiv preprint arXiv:1707.0983.
10.
Dokuz
,
Y.
, and
Tufekci
,
Z.
,
2021
, “
Mini-Batch Sample Selection Strategies for Deep Learning Based Speech Recognition
,”
Appl. Acoust.
,
171
, p.
107573
.
11.
Qian
,
N.
,
1999
, “
On the Momentum Term in Gradient Descent Learning Algorithms
,”
Neural Netw.
,
12
(
1
), pp.
145
151
.
12.
Nesterov
,
Y.
,
1983
, “
A Method for Unconstrained Convex Minimization Problem With the Rate of Convergence O (1/k2)
,”
Dokl. Akad. Nauk. SSSR
,
269
(
3
), p.
543
.
13.
Tieleman
,
T.
, and
Hinton
,
G.
,
2012
, “
Lecture 6.5-rmsprop: Divide the Gradient by a Running Average of Its Recent Magnitude
,”
COURSERA: Neural Netw. Mach. Learn.
,
4
(
2
), pp.
26
31
.
14.
Kingma
,
D. P.
, and
Ba
,
J.
,
2014
, “Adam: A Method for Stochastic Optimization,” CoRR, abs/1412.6980.
15.
Mitchell
,
M.
,
1996
,
An Introduction to Genetic Algorithms
,
MIT Press
,
Cambridge, MA
.
16.
Lu
,
T.-C.
,
2021
, “
Cnn Convolutional Layer Optimisation Based on Quantum Evolutionary Algorithm
,”
Connect. Sci.
,
33
(
3
), pp.
482
494
.
17.
Yu
,
Y.
, and
Zhang
,
M.
,
2021
, “
Control Chart Recognition Based on the Parallel Model of Cnn and Lstm With Ga Optimization
,”
Expert. Syst. Appl.
,
185
, p.
115689
.
18.
Arhore
,
E. G.
,
Yasaee
,
M.
, and
Dayyani
,
I.
,
2022
, “
Optimisation of Convolutional Neural Network Architecture Using Genetic Algorithm for the Prediction of Adhesively Bonded Joint Strength
,”
Struct. Multidiscipl. Optim.
,
65
(
9
), p.
256
.
19.
Luo
,
X.
,
Oyedele
,
L. O.
,
Ajayi
,
A. O.
,
Akinade
,
O. O.
,
Delgado
,
J. M. D.
,
Owolabi
,
H. A.
, and
Ahmed
,
A.
,
2020
, “
Genetic Algorithm-Determined Deep Feedforward Neural Network Architecture for Predicting Electricity Consumption in Real Buildings
,”
Energy AI
,
2
, p.
100015
.
20.
Gonçalves
,
C. B.
,
Souza
,
J. R.
, and
Fernandes
,
H.
,
2022
, “
Cnn Architecture Optimization Using Bio-inspired Algorithms for Breast Cancer Detection in Infrared Images
,”
Comput. Biol. Med.
,
142
, p.
105205
.
21.
Elsken
,
T.
,
Metzen
,
J. H.
, and
Hutter
,
F.
,
2019
, “
Neural Architecture Search: A Survey
,”
J. Mach. Learn. Res.
,
20
(
1
), pp.
1997
2017
.
22.
Tseng
,
I.
,
Cagan
,
J.
, and
Kotovsky
,
K.
,
2012
, “
Concurrent Optimization of Computationally Learned Stylistic Form and Functional Goals
,”
ASME J. Mech. Des.
,
134
(
11
), p.
111006
.
23.
Bai
,
S.
,
Kolter
,
J. Z.
, and
Koltun
,
V.
,
2018
, “An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling,” ArXiv, abs/1803.01271.
24.
Zhang
,
R.
,
Sun
,
F.
,
Song
,
Z.
,
Wang
,
X.
,
Du
,
Y.
, and
Dong
,
S.
,
2021
, “
Short-Term Traffic Flow Forecasting Model Based on Ga-tcn
,”
J. Adv. Transpor.
,
2021
, pp.
1
13
.
25.
Chen
,
Z.
,
Chen
,
B.
, and
Chen
,
X.
,
2022
, “
Remaining Useful Life Prediction of Turbofan Engine Based on Temporal Convolutional Networks Optimized by Genetic Algorithm
,”
J. Phys. Conf. Series
,
2181
(
1
), p.
012001
.
26.
Tan
,
B.
,
Li
,
Q.
, and
Zhang
,
T.
,
2023
, “
Application of TCN Algorithm in Aircraft System
,”
International Conference on Signal Processing, Computer Networks, and Communications (SPCNC 2022)
,
Zhengzhou, China
,
Dec. 16–17
,
H.
Wang
, ed., Vol. 12626, International Society for Optics and Photonics, SPIE, p.
126262B
.
27.
Zhu
,
J.
,
Su
,
L.
, and
Li
,
Y.
,
2022
, “
Wind Power Forecasting Based on New Hybrid Model With Tcn Residual Modification
,”
Energy AI
,
10
, p.
100199
.
28.
Saxena
,
A.
,
Goebel
,
K.
,
Simon
,
D.
, and
Eklund
,
N.
,
2008
, “
Damage Propagation Modeling for Aircraft Engine Run-to-Failure Simulation
,”
2008 International Conference on Prognostics and Health Management
,
Denver, CO
,
Oct. 6–9
, pp.
1
9
.
29.
Eastwick
,
C. N.
, and
Johnson
,
G.
,
2008
, “
Gear Windage: A Review
,”
ASME J. Mech. Des.
,
130
(
3
), p.
034001
.
30.
Holland
,
J. H.
,
1992
, “
Genetic Algorithms
,”
Sci. Am.
,
267
(
1
), pp.
66
73
.
31.
Vaswani
,
A.
,
Shazeer
,
N.
,
Parmar
,
N.
,
Uszkoreit
,
J.
,
Jones
,
L.
,
Gomez
,
A. N.
,
Kaiser
,
Ł.
, and
Polosukhin
,
I.
,
2017
, “
Attention Is All You Need
,”
Proceedings of the 31st International Conference on Neural Information Processing Systems, NIPS'17
,
Long Beach, CA
,
Dec. 4–9
, Curran Associates Inc., pp.
6000
6010
.
32.
Zeng
,
A.
,
Chen
,
M.
,
Zhang
,
L.
, and
Xu
,
Q.
,
2023
, “
Are Transformers Effective for Time Series Forecasting?
,”
Proceedings of the AAAI Conference on Artificial Intelligence
,
Washington, DC
,
Feb. 7–14
, Vol.
37
, pp.
11121
11128
.
33.
Hochreiter
,
S.
, and
Schmidhuber
,
J.
,
1997
, “
Long Short-Term Memory
,”
Neural Comput.
,
9
(
8
), pp.
1735
1780
.
34.
Chung
,
J.
,
Gulcehre
,
C.
,
Cho
,
K.
, and
Bengio
,
Y.
,
2015
, “
Gated Feedback Recurrent Neural Networks
,”
International Conference on Machine Learning
,
Lille, France
,
July 6–11
,
PMLR
, pp.
2067
2075
.
35.
Glorot
,
X.
, and
Bengio
,
Y.
,
2010
, “
Understanding the Difficulty of Training Deep Feedforward Neural Networks
,”
Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, JMLR Workshop and Conference Proceedings
,
Sardinia, Italy
,
May 13–15
, pp.
249
256
.
36.
Adam
,
L.
, and
Dorota
,
L.
,
2012
, “
Roulette-Wheel Selection Via Stochastic Acceptance
,”
Physica A: Stat. Mech. Appl.
,
391
(
6
), pp.
2193
2196
.
You do not currently have access to this content.