基于LSTM时间序列重建的生产装置异常检测

doi:10.11949/j.issn.0438-1157.20181050

Abstract

Abstract:

Industrial production equipment usually sets sensor alarm thresholds for alarms, but it is difficult to capture time series abnormalities below the alarm thresholds. The traditional statistics based detection method has great challenges in these time series anomaly detection. In this paper, an approach to the anomaly detection of process units is proposed by using the long short term memory (LSTM) time series reconstruction. At first, an LSTM network is introduced to vectorize the time series of sensor data, and another LSTM network is utilized to reconstruct the time series in reverse sequence. Then, the errors between the reconstructed values and the actual values are used to estimate the anomaly probability by the maximum likelihood estimation method. Eventually, anormaly detection is achieved by learning the abnormal alarm thresholds. Simulation resutls on the ECG standard testing data, energy data and sensor data of the dangerous goods tank have shown the effectiveness of the proposed method on data with different lengths.

Key words: algorithm, neural networks, parameter estimation, LSTM, time series, anomaly detection, maximum likelihood estimation

CLC Number:

TP 391

Shan DOU, Guangyu ZHANG, Zhihua XIONG. Anomaly detection of process unit based on LSTM time series reconstruction[J].CIESC Journal, 2019, 70(2): 481-486.

Figures/Tables 7

Fig.1

Fig.2

Table 1

Procedure of MLE parameter estimation[25]"

1.设异常概率 $p$ 随重建误差e呈高斯分布，则其似然函数如式(4)所示；

2.对似然函数取对数并求和，整理后为

$H (μ, Σ) = ∑ e i ∈ V 1 l n p (e i | μ, Σ)$ (8)

3.对 $H μ, Σ$ 求偏导，使得其满足：

$? H (μ, Σ) ? μ = 0 ? H (μ, Σ) ? Σ = 0$

4.求解上述方程得到参数估计值如下所示，其中N为数据集 $N V 1$ 的时间点总数。

$μ = 1 N ∑ i = 1 N e i Σ = 1 N ∑ i = 1 N (e i - μ) (e i - μ) T$ (9)

Table 1

Fig.3

Fig.4

Fig.5

Table 2

Experimental results"

Data	P		R		$F 0.1$
Data	RNN	LSTM	RNN	LSTM	RNN	LSTM
ECG	0.98	1.00	0.45	0.43	0.96	0.97
EN	0.89	0.94	0.10	0.12	0.83	0.88
SE	0.89	0.92	0.20	0.26	0.86	0.89

Table 2

References 30

1	周大镯, 刘月芬, 马文秀. 时间序列异常检测[J]. 计算机工程与应用, 2008, 44(35): 145-147.
	ZhouD Z, LiuY F, MaW X. Effective time series outlier detection algorithm based on segmentation[J]. Computer Engineering and Applications, 2008, 44(35): 145-147.
2	HawkinsD M. Identification of Outliers[M]. London: Chapman and Hall, 1980: 25.
3	BillorN, HadiA S, VellemanP F. BACON: blocked adaptive computationally efficient outlier nominators[J]. Computational Statistics & Data Analysis, 2000, 34(3): 279-298.
4	KnorrE M, NgR T. A unified notion of outliers: properties and computation[C]//KDD. 1997: 219-222.
5	李钢, 周东华. 基于SPM的多变量连续过程在线故障预测方法[J]. 化工学报, 2008, 59(7): 1829-1833.
	LiG, ZhouD H. SPM-based online fault prediction approach for multivariate continuous processes[J]. Journal of Chemical Industry and Engineering(China), 2008, 59(7): 1829-1833.
6	张浩, 刘振娟, 李宏光, 等. 基于关联变量时滞分析卷积神经网络的生产过程时间序列预测方法[J]. 化工学报, 2017, 68(9): 3501-3510.
	ZhangH, LiuZ J, LiH G, et al. Process time series prediction based on application of correlated process variables to CNN time delayed analyses[J]. CIESC Journal, 2017, 68(9): 3501-3510.
7	PageE S. Continuous inspection schemes[J]. Biometrika, 1954, 41(1/2): 100-115.
8	BassevilleM, NikiforovI V. Detection of Abrupt Changes: Theory and Application[M]. Englewood Cliffs: Prentice Hall, 1993: 30.
9	MaJ, PerkinsS. Time-series novelty detection using one-class support vector machines[C]//Neural Networks, 2003. Proceedings of the International Joint Conference on. IEEE, 2003: 1741-1745.
10	王少军. 时间序列预测的可重构计算研究[D]. 哈尔滨:哈尔滨工业大学, 2012: 35.
	WangS J. Research on reconfigurable computing for time series forecasting[D]. Harbin: Harbin Institute of Technology, 2012: 35.
11	HochreiterS, SchmidhuberJ. Long short-term memory[J]. Neural Computation, 1997, 9(8): 1735-1780.
12	SutskeverI, VinyalsO, LeQ V. Sequence to sequence learning with neural networks[C]//Advances in Neural Information Processing Systems, 2014: 3104-3112.
13	MiwaM, BansalM. End-to-end relation extraction using LSTMs on sequences and tree structures[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2016: 1105-1116.
14	ChoiY, BreckE, CardieC. Joint extraction of entities and relations for opinion recognition[C]//Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 2006: 431-439.
15	GravesA, SchmidhuberJ. Offline handwriting recognition with multidimensional recurrent neural networks[C]//Advances in Neural Information Processing Systems, 2009: 545-552.
16	MessinaR, LouradourJ. Segmentation-free handwritten Chinese text recognition with LSTM-RNN[C]//Document Analysis and Recognition (ICDAR), 2015 13th International Conference on. IEEE, 2015: 171-175.
17	HuangZ, XuW, YuK. Bidirectional LSTM-CRF models for sequence tagging[J]. arXivpreprint arXiv: 1508.01991, 2015.
18	MaX, HovyE. End-to-end sequence labeling via bi-directional LSTM-CNNs-CRF[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2016: 1064-1074.
19	ChoK, van MerrienboerB, BahdanauD, et al. On the properties of neural machine translation: encoder–decoder approaches[C]//Proceedings of SSST-8, Eighth Workshop on Syntax, Semantics and Structure in Statistical Translation, 2014: 103-111.
20	ChoK, VanM B, GulcehreC, et al. Learning phrase representations using RNN encoder–decoder for statistical machine translation[C]//Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2014: 1724-1734.
21	CybenkoG. Approximation by superpositions of a sigmoidal function[J]. Mathematics of Control, Signals and Systems, 1989, 2(4): 303-314.
22	MaasA L, HannunA Y, NgA Y. Rectifier nonlinearities improve neural network acoustic models[J]. Proc. Icml., 2013, 30(1): 3.
23	GravesA. Long short-term memory[M]//Supervised Sequence Labelling with Recurrent Neural Networks. Heidelberg, Berlin: Springer, 2012: 37-45.
24	ChungJ, GulcehreC, ChoK H, et al. Empirical evaluation of gated recurrent neural networks on sequence modeling[J]. arXivpreprint arXiv: 1412.3555, 2014.
25	BockR D, AitkinM. Marginal maximum likelihood estimation of item parameters: application of an EM algorithm[J]. Psychometrika, 1981, 46(4): 443-459.
26	MalhotraP, VigL, ShroffG, et al. Long short term memory networks for anomaly detection in time series[C]//23rd European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning, ESANN 2015. Bruges, Belgium, 2015: 89.
27	GoutteC, GaussierE. A probabilistic interpretation of precision, recall and F-score, with implication for evaluation[C]//European Conference on Information Retrieval. Heidelberg, Berlin: Springer, 2005: 345-359.
28	LeQ V, NgiamJ, CoatesA, et al. On optimization methods for deep learning[C]//Proceedings of the 28th International Conference on International Conference on Machine Learning. Omnipress, 2011: 265-272.
29	KingmaD P, BaJ. Adam: a method for stochastic optimization[J]. arXivpreprint arXiv: 1412.6980, 2014.
30	ConnorJ T, MartinR D, AtlasL E. Recurrent neural networks and robust time series prediction[J]. IEEE Transactions on Neural Networks, 1994, 5(2): 240-254.

Related Articles 15

[1]	Fei LI, Cuili YANG, Wenjing LI, Junfei QIAO. Optimal control of wastewater treatment process using NSGAII algorithm based on multi-objective uniform distribution [J]. CIESC Journal, 2019, 70(5): 1868-1878.
[2]	Shipin YANG, Zhen HUANG, Lijuan LI, Jianquan SONG, Jing YE, Hui WANG. A deep dive diagnostic and correction algorithm for mismatched sub-models in complicated chemical processes [J]. CIESC Journal, 2019, 70(4): 1485-1493.
[3]	Qichao XU, Jinbo JIANG, Xudong PENG, Jiyun LI, Yuming WANG. Unified model and geometrical optimization of bi-directional groove of dry gas seal based on genetic algorithm [J]. CIESC Journal, 2019, 70(3): 995-1005.
[4]	Zhizhen WANG, Zhiyun ZOU. Nonlinear predictive control strategies of pH neutralization process based on neural networks [J]. CIESC Journal, 2019, 70(2): 678-686.
[5]	Qilei LIU, Kun FENG, Linlin LIU, Jian DU, Qingwei MENG, Lei ZHANG. Reaction solvent design method based on Dragon descriptors and modified decision tree-genetic algorithm [J]. CIESC Journal, 2019, 70(2): 533-540.
[6]	Zhencheng YE, Huanlan ZHOU, Debao RAO. Hybrid modeling and optimization of acetylene hydrogenation process [J]. CIESC Journal, 2019, 70(2): 496-507.
[7]	Wenpeng JI, Huizhong YANG. Multi-manifold soft sensor based on modified expanding search clustering algorithm [J]. CIESC Journal, 2019, 70(2): 723-729.
[8]	Jian ZHU, Bo YANG, Yongjian WANG, Xiaojie TANG, Hongguang LI. New operation optimization method with time series based on Levenshtein distance hierarchical clustering [J]. CIESC Journal, 2019, 70(2): 581-589.
[9]	Bingbin GU, Weili XIONG. Fault diagnosis based on PCA method with multi-block information extraction [J]. CIESC Journal, 2019, 70(2): 736-749.
[10]	Yongfei XUE, Yalin WANG, Bei SUN, Qianzhong LI, Jiazhou SUN. Improved state transfer algorithm-based kinetics parameter estimation for cascaded plug flow reactors [J]. CIESC Journal, 2019, 70(2): 607-616.
[11]	Zhiqiang GENG, Meng XU, Qunxiong ZHU, Yongming HAN, Xiangbai GU. Research and application of soft measurement model for complex chemical processes based on deep learning [J]. CIESC Journal, 2019, 70(2): 564-571.
[12]	Jian TANG, Junfei QIAO. Dioxin emission concentration soft measuring approach of municipal solid waste incineration based on selective ensemble kernel learning algorithm [J]. CIESC Journal, 2019, 70(2): 696-706.
[13]	Jian CAO, Peng MU, Xiangbai GU, Qunxiong ZHU. Automatic generation method of process knowledge based on P-graph [J]. CIESC Journal, 2019, 70(2): 467-474.
[14]	Wenjing LI, Meng LI, Junfei QIAO. Effluent BOD soft measurement based on mutual information and self-organizing RBF neural network [J]. CIESC Journal, 2019, 70(2): 687-695.
[15]	Bo DAI, Zeyu ZHOU, Yan ZHANG, Shuangshuang LIN, Xuejun LIU. Design of safe distance monitoring system for hazardous chemicals storage stack [J]. CIESC Journal, 2019, 70(2): 707-715.

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed

Comments

Recommended 10

[1]	LING Lixia, ZHANG Riguang, WANG Baojun, XIE Kechang. Pyrolysis Mechanisms of Quinoline and Isoquinoline with Density Functional Theory[J]. , 2009, 17(5): 805 -813 .
[2]	LEI Zhigang, LONG Aibin, JIA Meiru, LIU Xueyi. Experimental and Kinetic Study of Selective Catalytic Reduction of NO with NH₃ over CuO/Al₂O₃/Cordierite Catalyst[J]. , 2010, 18(5): 721 -729 .
[3]	SU Haifeng, LIU Huaikun, WANG Fan, LÜXiaoyan, WEN Yanxuan. Kinetics of Reductive Leaching of Low-grade Pyrolusite with Molasses Alcohol Wastewater in H₂SO₄[J]. , 2010, 18(5): 730 -735 .
[4]	WANG Jianlin, XUE Yaoyu, YU Tao, ZHAO Liqiang. Run-to-run Optimization for Fed-batch Fermentation Process with Swarm Energy Conservation Particle Swarm Optimization Algorithm[J]. , 2010, 18(5): 787 -794 .
[5]	SUN Fubao, MAO Zhonggui, ZHANG Jianhua, ZHANG Hongjian, TANG Lei, ZHANG Chengming, ZHANG Jing, ZHAI Fangfang. Water-recycled Cassava Bioethanol Production Integrated with Two-stage UASB Treatment[J]. , 2010, 18(5): 837 -842 .
[6]	Gao Ruichang, Song Baodong and Yuan Xiaojing( Chemical Engineering Research Center, Tianjin University, Tianjin 300072). LIQUID FLOW DISTRIBUTION IN GAS - LIQUID COUNTER - CONTACTING PACKED COLUMN[J]. , 1999, 50(1): 94 -100 .
[7]	Su Yaxin, Luo Zhongyang and Cen Kefa( Institute of Thermal Power Engineering , Zhejiang University , Hangzhou 310027). A STUDY ON THE FINS OF HEAT EXCHANGERS FROM OPTIMIZATION OF ENTROPY GENERATION[J]. , 1999, 50(1): 118 -124 .
[8]	Luo Xiaoping(Department of Industrial Equipment and Control Engineering , South China University of Technology, Guangzhou 510641)Deng Xianhe and Deng Songjiu( Research Institute of Chemical Engineering, South China University of Technology, Guangzhou 5106. RESEARCH ON FLOW RESISTANCE OF RING SUPPORT HEAT EXCHANGER WITH LONGITUDINAL FLUID FLOW ON SHELL SIDE[J]. , 1999, 50(1): 130 -135 .
[9]	Jin Wenzheng , Gao Guangtu , Qu Yixin and Wang Wenchuan ( College of Chemical Engineering, Beijing Univercity of Chemical Technology, Beijing 100029). MONTE CARLO SIMULATION OF HENRY CONSTANT OF METHANE OR BENZENE IN INFINITE DILUTE AQUEOUS SOLUTIONS[J]. , 1999, 50(2): 174 -184 .
[10]	LI Qingzhao;ZHAO Changsui;CHEN Xiaoping;WU Weifang;LI Yingjie . Combustion of pulverized coal in O₂/CO₂mixtures and its pore structure development [J]. , 2008, 59(11): 2891 -2897 .

Anomaly detection of process unit based on LSTM time series reconstruction

RichHTML

PDF (PC)