引用本文

鲍刚, 陈媛媛, 温思雨, 赖陟岑. 忆阻递归神经网络稳定性分析及其在联想记忆中的应用. 自动化学报, 2017, 43(12): 2244-2252. doi: 10.16383/j.aas.2017.e170103

Bao Gang, Chen Yuanyuan, Wen Siyu, Lai Zhicen. Stability Analysis for Memristive Recurrent Neural Network and Its Application to Associative Memory. ACTA AUTOMATICA SINICA, 2017, 43(12): 2244-2252. doi: 10.16383/j.aas.2017.e170103

http://www.aas.net.cn/cn/article/doi/10.16383/j.aas.2017.e170103

关键词

Associative memory，memristor，memristive recurrent neural network (MRNN)，stability 

摘要

Memristor is a nonlinear resistor with variable resistance. This paper discusses dynamic properties of memristor and recurrent neural network (RNN) with memristors as connection weights. Firstly, it establishes that there exists a threshold voltage for memristor. Secondly, it presents a model for memristive recurrent neural network (MRNN) which has variable and bounded coe-cients, and analyzes stability of memristive neural network by some maths tools. Thirdly, it gives a synthesis algorithm for associative memory based on memristive recurrent neural network. At last, three examples verify our results.

文章导读

Artificial neural networks are developed for solving some complex problems in control, optimal computation, pattern recognition, information processing, and associative memory [1]-[13]. American scientist Hopfield makes a great contribution for the development of neural network. That is the implementation of neural network by simple circuit devices, resistors, capacitors and amplifiers [14]. Hopfield neural network (HNN) can mimic the human's associative memory function and accomplish optimization. The key point is the weights of HNN which are implemented by resistors for simulating neuron synapse. While the bottleneck is that linear resistors cannot reflect variability of synapse for resistance of linear resistor being invariable.

Memristor [15], [16], the arising fourth circuit device, makes it better to simulate the variability of neuron synapse. Pershin and Ventra [17] gives their experimental research results that neurons with memristors as synapses can simulate the associative memory function of a dog. Hence, memristor is the advancing spot in the present physics research. Several models of memristor have been set up and its properties have been analyzed in [18]-[21]. Based on these analyses, memristor can be used to mimic synapse in neural computing architecture [22], construct memristor bridge synapse [23] and brain combined with the conventional complementary metal oxide semiconductor (CMOS) technology [24], set memristive neural network [25], [26] and implement memristor array for image processing [27] etc.

Some researchers derive mathematical model of memristive recurrent neural network (MRNN) by replacing resistors with memristors in Hopfield and cellular neural network circuit [28]-[30]. MRNN is modeled by state-dependent switched systems by simplifying the memristance as two-valued device with different terminal voltage. With differential inclusion theory, Lyapunov-Krasovskii function and some other math tools, some sufficient conditions are derived for dynamics of MRNN, such as, convergence and attractivity [31]-[33], periodicity and dissipativity [34], dissipativity for stochastic and discrete case, global exponential almost periodicity, and complete stability [35], multi-stability [36], etc. Considering the trouble from the switching property of memristor, researchers derive some interesting results about exponential stabilization, reliable stabilization, and finite-time stabilization of MRNN by designing different state feedback controllers [37], [38] and sampled-data controller [39]. All of these results make a solid foundation for MRNN's application to associative memory.

Associative memory is a distinguished function of human brain which can be simulated by recurrent neural network (RNN). The design problem is that some given prototype patterns are to be stored by RNN, and then the stored patterns can be recalled by some prompt information. In the existing literatures [40]-[46], there are two design methods for associative memory. One is that prototype patterns are designed as multiple locally asymptotically stable equilibria and initial values are the recalling probes. Another is that a prototype pattern is designed as the unique globally asymptotically stable equilibrium point with one external input as the recalling probe. Different external inputs mean different equilibrium points, i.e., different prototype patterns.

To the best of our knowledge, the bottleneck of associative memory based on RNN is that capacity of RNN is limited and different storage task needs different RNN because resistance can not be changed. Furthermore, there are few works about associative memory based on MRNN. Hence, the contribution of this paper is obtaining a threshold voltage for memristor by simulation, presenting a novel type of MRNN with infinite number of sub neural networks, and design a program for associative memory based on MRNN. Compared with MRNN models in the existing literatures, the difference is that every coefficient of MRNN has infinite number of values, not two values. Furthermore, every coefficient can be changed by the external input. So the associative memory based on MRNN seems to solve the problem of storage capacity.

The rest of this paper is organized as the following sections. Memristor property analysis and some preliminaries are stated in Section 2. Then, some sufficient conditions are given to ensure global stability and multi-stability of MRNN by some maths tools in Section 3, respectively. Next, design procedure for associative memory based on MRNN is given in Section 4. To elucidate our results, three simulation examples are presented in Section 5. At last, conclusion is drawn in Section 6.

图 2  Transient behaviors of x1(t) of MRNN (26).

图 3  Transient behaviors of x2(t)of MRNN (26).

图 4  Phase plot of x1(t) and x2(t) of MRNN (26).

In this paper, we have introduced MRNN which is a family of recurrent neural networks. Some sufficient conditions are derived to assure its mono-stability and multi-stability. In the existing literature on neural network, the largest number of equilibrium points is (4k−1)ⁿ and (2k)ⁿ of them are locally exponentially stable. In fact, associative memory output patterns are up to the activation function. This point affects the storage capacity of associative memory. Our MRNN with coefficients in intervals cannot be limited by output value of the activation. Hence MRNN can increase the storage capacity of associative memory. This is the main merit which is different from traditional artificial neural network. So self-adaptive and self-organization recurrent neural network can be realized with memristor [26] in the future.

作者简介

Yuanyuan Chen

received the B.S.degree from the College of Science and Technology, China Three Gorges University in 2016.Now she is a postgraduate student and pursuing for M.S.degree at the School of Electrical Engineering and New Energies, China Three Gorges University.Her current research interests include microgrid optimization scheduling and stability analysis.E-mail:pretty.yuanzi@qq.com

Siyu Wen 

received the B.S.degree in water resources and hydropower engineering from the College of Science and Technology, China Three Gorges University in 2016.Now, she is currently working toward the M.S.degree at the School of Electrical Engineering and New Energies, China Three Gorges University.Her current research interests include hydropower dispatching and unit commitment optimization.E-mail:215341796@qq.com

Zhicen Lai 

received the B.S.degree in electrical engineering and its automation (focus on transmission line), China Three Gorges University in 2016.She is currently working toward the M.S.degree at the School of Electrical Engineering and New Energies, China Three Gorges University, Yichang, China.Her current research interests include microgrid control and stability analysis.E-mail:2512991452@qq.com

Gang Bao 

received the B.S.degree in mathematics from Hubei Normal University, Huangshi, China, the M.S.degree in applied mathematics from Beijing University of Technology, Beijing, China, in 2000 and 2004, the Ph.D.degree from the Department of Control Science and Engineering, Huazhong University of Science and Technology, respectively.His research interests include memristor, stability analysis of nonlinear systems, and association memory.Corresponding author of this paper.E-mail:hustgangbao@ctgu.edu.cn