Introduction: Artificial intelligence machine learning related to the algorithm content, today we will focus on the long and short-term memory neural network (LSTM) algorithm.
Through the introduction of the previous article [Artificial Intelligence Recurrent Neural Network (RNN)], we know that RNN is a powerful artificial neural network algorithm. An important advantage of RNN is that it can be used between input and output sequences. Use contextual information in the mapping process. However, RNN has problems such as gradient disappearance or gradient explosion. Therefore, in order to solve the above problems, the Long Short-Term Memory Neural Network (LSTM) was born.
The long and short-term memory neural network LSTM is a special RNN that can learn long-term dependencies. Proposed by Hochreiter and Schmidhuber (1997), in the later work, many people have adjusted and popularized it (in addition to the original author, many people have contributed to modern LSTM, incomplete statistics: Felix Gers (currently in Google’s DeepMind) Appointment), Fred Cummins, SanTIago Fernandez, Felix Gers (invented the LSTM forget gate), JusTIn Bayer (automatic evolution), Daan Wierstra, Julian Togelius, FausTIan Gomez, Matteo Gagliolo and Alex Graves). LSTM works very well on a large number of problems and is now widely used.
Long Short-Term Memory (LSTM) is a time recurrent neural network, suitable for processing and predicting important events with relatively long intervals and delays in time series.
Rather than saying that the long and short-term memory neural network LSTM is a kind of recurrent neural network, it is better to say that an enhanced version of the component is placed in the recurrent neural network. Specifically, the small circle in the hidden layer of the recurrent neural network is replaced by a short-term memory module, as shown in the figure below.
The essence of LSTM:LSTM introduces the ingenious idea of ​​self-circulation to generate a long-term continuous flow path of the gradient is the core contribution of the initial LSTM model. One of the key extensions is to make the power of self-looping context dependent, rather than fixed. The weight of this self-loop (controlled by another hidden unit) is gated, and the accumulated time scale can be changed dynamically.
In addition to the external RNN cycle, the LSTM recurrent network also has an internal LSTM cell cycle (self-loop).
LSTM avoids the problem of long-term dependence through deliberate design. Remember that long-term information is the default behavior of LSTM in practice, not a capability that can be acquired at a high price.
LSTM principle:The difference between LSTM and RNN is that it adds a "processor" to the algorithm to determine whether the information is useful or not. The structure of this processor is called a cell.
Three doors are placed in a cell, which are called input gate, forget gate and output gate. When a piece of information enters the LSTM network, it can be judged whether it is useful according to the rules. Only the information that meets the algorithm authentication will be left, and the non-compliant information will be forgotten through the forget door.
It is nothing more than the working principle of one input and two output, but it can solve long-standing problems in neural networks under repeated calculations. It has been proved that LSTM is an effective technology to solve the problem of long-order dependence, and the universality of this technology is very high, resulting in many possible changes. Researchers have proposed their own variable versions based on LSTM, which allows LSTM to handle ever-changing vertical problems.
LSTM depth analysis:LSTM has the ability to remove or add information to the cell state through a well-designed structure called a "gate". A door is a way of letting information through selectively. It contains a sigmoid neural network layer and a pointwise multiplication operation.
The sigmoid layer outputs a value between 0 and 1, describing how much of each part can pass. 0 means "No amount is allowed to pass", 1 means "Allow any amount to pass"!
LSTM has three gates (input gate, forget gate, output gate) to protect and control the cell state.
Standard LSTM:1) Decide to discard information:
2) Determine the updated information:
3) Update cell status:
4) Output information:
Variants of LSTM:1) peephole connection:
2) Coupled forget gate and input gate:
3) GRU (Gated Recurrent Unit):
LSTM application scenarios:LSTM has already had many applications in the field of technology. The LSTM-based system can learn tasks such as language translation, robot control, image analysis, document summarization, speech recognition, image recognition, handwriting recognition, chat robot control, disease prediction, click-through rates and stocks, and music synthesis.
In 2015, Google greatly improved the speech recognition capabilities of Android phones and other devices through the LSTM program based on CTC training. Baidu also uses CTC; Apple's iPhone uses LSTM in QucikType and Siri; Microsoft not only uses LSTM for speech recognition, but also uses this technology for virtual dialogue image generation and programming code. Amazon Alexa communicates with users at home through two-way LSTM, and Google uses LSTM in a wider range. It can generate image subtitles and automatically reply to emails. It is included in the new smart assistant Allo, and it also significantly improves the quality of Google translation. . At present, a large part of the computing resources of Google's data center are now performing LSTM tasks.
Conclusion:The long short-term memory network LSTM is a time recurrent neural network, which is suitable for processing and predicting important events with relatively long intervals and delays in time series. LSTM is a leap forward in using RNN. LSTM algorithm has a wide range of applications in artificial intelligence machine learning, translation language, control robots, image analysis, document summarization, speech recognition, image recognition, handwriting recognition, control chat robots, disease prediction, click-through rate and stocks, synthetic music and other fields.
ZGAR AZ Vape Pods 5.0S
ZGAR electronic cigarette uses high-tech R&D, food grade disposable pod device and high-quality raw material. All package designs are Original IP. Our designer team is from Hong Kong. We have very high requirements for product quality, flavors taste and packaging design. The E-liquid is imported, materials are food grade, and assembly plant is medical-grade dust-free workshops.
From production to packaging, the whole system of tracking, efficient and orderly process, achieving daily efficient output. WEIKA pays attention to the details of each process control. The first class dust-free production workshop has passed the GMP food and drug production standard certification, ensuring quality and safety. We choose the products with a traceability system, which can not only effectively track and trace all kinds of data, but also ensure good product quality.
We offer best price, high quality Pods, Pods Touch Screen, Empty Pod System, Pod Vape, Disposable Pod device, E-cigar, Vape Pods to all over the world.
Much Better Vaping Experience!
Pods, Vape Pods, Empty Pod System Vape,Disposable Pod Vape Systems
ZGAR INTERNATIONAL(HK)CO., LIMITED , https://www.szvape-pen.com