1 Introduction
This article refers to the address: http://
Electronic Image Stabilization [1, 2] (EIS, Electronic Image Stabilization) is a technique for determining the inter-frame shift of an image sequence directly from the image plane by digital image processing and electronic techniques, and performing motion compensation to obtain a stable image sequence. Compared with the traditional optical image stabilization and mechanical image stabilization technology, the electronic image stabilization has the advantages of high image stabilization accuracy, small size, light weight, low power consumption and low cost. From the history of research on image stabilization technology, electronic image stabilization technology is the development trend of research, and it has broad application prospects in military and civilian applications.
In electronic image stabilization, global motion estimation [1] and transform reference frame strategy are two difficult points. It is the key to correctly estimate the interframe motion of image sequences. At present, there are many algorithms for electronic image stabilization motion estimation, such as block matching method [3], bit plane matching method [4], representative point matching method [5] and so on. Among them, the gray projection method [6] is simple in algorithm, small in calculation, and can maintain high accuracy. The multi-region gray projection method is validated and applied to eliminate the influence of internal moving objects on global motion estimation, and a transform reference frame strategy with correction is designed.
2, the basic process of electronic image stabilization
As shown in the figure [1] above, the electronic image stabilization process consists of image preprocessing, local motion estimation, global motion estimation and motion compensation. Pre-processing: Image acquisition and processing accuracy is affected by many factors, such as CCD spatial resolution, quantization error, system noise, A/D conversion accuracy, camera vibration, etc. These noises directly affect the processing of image sequences. And the performance of the electronic image stabilization system. Before the image stabilization operation such as motion estimation and motion compensation, the image needs to be filtered. The experiment uses median filtering. Motion estimation [7]: including local motion estimation and global motion estimation, that is, obtaining an offset between adjacent two frames of images, and obtaining a local motion vector and a global motion vector. Motion compensation [8]: The motion vector obtained by motion estimation is used to calibrate the image to the same reference to obtain a stable image sequence.
Figure 1 Electronic image stabilization flow chart
3, gray projection
The gray projection algorithm is based on the standard frame-to-frame operation. The algorithm is divided into three steps: image mapping, projection filtering and correlation calculation.
Image mapping is to map the two-dimensional gray information of the image input in each frame into two independent one-dimensional projection sequences. The formula is as follows: 
, in the formula 
Is the gray value of the i-th line of the kth frame image; 
Is the gray value of the jth column of the kth frame image; 
Is the kth frame image 
The gray value of the pixel at the location.
Projection filtering is to filter the image projection value. Because the offset information is large, the edge information will adversely affect the cross-correlation peak in the cross-correlation operation, which requires the projection value at the edge to be removed. Cosine filtering is usually used. This method can preserve the projection value of the middle region and reduce the information amplitude of the edge region, thus ensuring the correctness of the correlation calculation and improving the correction accuracy.
The correlation calculation is to perform cross-correlation calculation between the k-th frame image and the row and column projection curves of the reference frame image, and the row and column displacement vectors of the current frame image relative to the reference frame image can be determined according to the peaks of the two correlation curves. After obtaining the displacement vectors in the horizontal and vertical directions, the current image is moved to the corresponding pixel distance in the opposite direction of the displacement vector to achieve stability of the image sequence.
4, sub-region gray projection
In order to eliminate the influence of local moving objects on motion estimation and improve the reliability of gray projection, a sub-region gray projection method is adopted.
The size and number of blocks should be determined by the size of the image, the size of the internal moving object, and the effectiveness of the grayscale projection. If there are too many blocks, each block contains too little information, which will affect the accuracy of gray projection. If there are too few blocks, the local motion vector cannot be accurately determined.
Table 1 shows the results of simulation experiments in which the projection area is divided into different sizes for motion estimation. It can be seen from Table 1 that if the projection area size is less than 90×90, the range that can be accurately estimated is too small, and error is easy to occur, and the larger than 100×100 pixels can ensure the validity of the gray scale projection, and meet the needs of motion estimation. The size of the block is determined by taking into account the size of the internal moving object. The smaller the internal moving object, the easier it is to apply the block grayscale projection method to eliminate its influence on the global motion vector. In the application context, the target is a minimum of a few pixels, and the maximum is 180×120 pixels. Normally, the moving target is located in the center of the field of view. If it is divided into 4 projection areas [9], each area may contain a part of the moving target, and it is difficult to obtain the actual global motion vector, so at least the projection area is divided into 9 Blocks. Assuming that the image size is 720×576 and the target is at most 180×120, the intermediate projection area is divided into nine blocks of size 200×180 pixels, and the target occupies at most four blocks, which does not affect the global motion estimation decision. It can be seen that as long as the internal moving object occupies a small number of blocks and the validity of the gray projection is ensured, the correct global motion estimation vector can be guaranteed.
Table 1 Influence of block size of projection area on motion estimation
Figure 2 Image block diagram
5, transform the reference frame strategy
In real-time electronic image stabilization, the camera is in constant motion and the background is not static. If the first frame is always used as the reference frame, the image sequence that may be finally stabilized does not reflect the real motion, which requires updating the reference frame in real time. If each frame has a previous frame as a reference frame, it will greatly increase the amount of calculation, affecting the real-time performance of the electronic image stabilization.
In order to balance the speed and accuracy of the image stabilization, the reference frame is changed every n frames, and the value of n is determined by considering the motion of the camera. The reference frame uses a stable image. The problem here is that if the stabilized image is always selected as the reference frame, if a certain motion compensation error occurs, for example, the motion estimation error due to the actual offset exceeding the maximum offset range. Then this kind of mistake will continue. Therefore, a method of inserting an I frame after several P frames in video coding is adopted [10], and one frame of the original image is inserted as a reference frame every k frames. If the motion vector of the current frame is too large when the reference frame is exchanged, and the previous reference frame is too much, the next frame is taken as the reference frame.
Figure 3 Transform reference frame flow chart
6. Experimental results and analysis
The image used in the simulation is taken from the video captured by a bank monitoring device at the moment of the earthquake. In this video, the entire background is violently shaken. This video is subjected to image stabilization processing. Figure 4 shows the continuous 6-frame image taken from it and the image after the image stabilization processing.
Figure 4.1 Six consecutive images before image stabilization
Figure 4.2 Six consecutive images after image stabilization
The effect of the experiment is analyzed. Figure 5 shows the comparison of the row and column projection curves of the six frames before and after the image stabilization. The abscissa indicates the number of rows and columns, and the ordinate indicates the projection value. It can be seen from the figure that the projection curves of the six images before the image stabilization are greatly different. After the image stabilization, the difference of the projection curve is reduced, which is basically consistent.
Figure 5.1 Comparison of image line projections for 6 consecutive frames before and after image stabilization
Figure 5.2 Comparison of image projections of 6 consecutive frames before and after image stabilization
The PSNR (Peak Signal To Noise Ratio) is an evaluation criterion for the image stabilization effect.
The mean square error MSE is the deviation value of each pixel between two frames, and the larger the PSNR, the better the image stabilization effect. It can be seen from Table 2 that the PSNR value after image stabilization by the block gray projection method is greatly improved compared with the overall projection method, which indicates that the electronic image stabilization method achieves a better image stabilization effect.
Table 2 Comparison of PSNR values ​​before and after stabilization of the two methods
7. Conclusion
The method of global motion estimation using sub-region gray projection algorithm is deeply analyzed, which can effectively eliminate the influence of local object motion on global motion estimation. The reference frame update strategy is designed with the idea of ​​video coding, with correction function. Like the algorithm is simple to implement, easy to understand, use it to stabilize the sharply shaken seismic video, and achieve better results
BT2 Cable Trunking (75X75)
BT2 Cable Trunking (75X75),Electrical Trunking ,Cable Conduit ,Cable Ducting
Cable Tray ,Steel Channel Co., Ltd. , http://www.nbcabletray.com