Download Fast automatic target detection system based on a visible image sensor, ripple algorithm. and more Study Guides, Projects, Research Electronic Technology in PDF only on Docsity!
a visible image sensor and ripple algorithm
FAST AUTOMATIC TARGET DETECTION SYSTEM
BASED ON A VISIBLE IMAGE SENSOR AND RIPPLE
ALGORITHM
AUTOMATIC target detection (ATD) aims to detect and identify the types of targets. It can be used in a variety of applications, such as assessing the situation of battle field using robots and monitoring targets over land, sea and air. In these applications, the captured image pivots on a shaft through the optical axis of the optical sensor. To deal with this situation, the ATD system should have the ability of detecting the targets when they rotate. Also, the targets on the ground may move and have different kinds of attitude. Thus, the ATD system should be able to detect the targets from different view angles and have fast recognition speed. Furthermore, the ATD system is often required to be small in size, light in weight and of low power consumption such that they can be embedded into small and mobile carriers, such as remote-controlled robots. To overcome the aforementioned problems, a number of ATD algorithms have been reported in the literature. Engin et al surveyed the applications of neural networks technology to ATD. The difficulties associated with neural networks include dimensionality and generalization problems. To learn high-dimensional data, such as images, a huge number of training data are required. As a result, the training process takes a long time to accomplish. It is difficult to reduce the time used to train the neural networks as the usage of a small number of training data will result in poor generalization of the neural networks. Hough transforms and their generalized versions are popular methods for ATD. Hough transforms find the optimum transformation by voting for candidates in a parameter space. When there are many parameters describing shape transformation, it is computationally expensive to search for the maximum voting in the multidimensional parameter space. Thus, it cannot fulfill the requirement of real-time applications. Affine moment invariants approaches are also widely used for ATD. Affine moment invariants can be used not only for scale, position, and rotation invariant pattern identification, but also for recognition of
a visible image sensor and ripple algorithm affine-deformed objects. However, the amount of calculation increases rapidly when the order of moment increases. So it is not suitable for real-time system either. The
a visible image sensor and ripple algorithm
1. SYSTEM HARDWARE CONFIGURATION
The new ATD system is composed of a visible image sensor, the USTB- CK923 platform, a power module, and an LCD screen (see Fig 2.1). The basic system parameters are shown in Table I. The functions of the main system elements are as follows:
2.1 VISIBLE IMAGE SENSOR
The visible image sensor can capture visible images and the contour information (rather than the texture information) of the images is used for target detection. The CMOS image sensor is employed in our system since it is cheaper and faster than the CCD image sensor, and very high resolution images are not necessary for the ripple algorithm to be introduced in Section III.
2.2 USTB-CK
The is an image processing hardware platform developed by us at the University of Science and Technology Beijing, China. It contains a 600MHz digital media processor DM642 which is based on the TI’s C64xx DSP, an ultra low power PAL video decoder TVP5150A and a video encoder SAA7105H. Fig 2.2 shows the picture of the USTB-CK923 platform. The size and weight of the platform are 11cm × 8cm and 102g, respectively, and it has a power rating of 1.1W. Whilst the TI’s DM642 development board has the same performance parameters as the USTB- CK923 platform, its size, weight and power rating are 16cm×10cm, 135g and 1.5W, respectively. So, our USTB-CK923 platform is prior to the TI’S DM642 development board.
2.3 LCD SCREEN
The LCD screen is used to show the detection results. The programs are downloaded into the flash memory of the digital media processor DM642. The video signal captured by the visible image sensor is converted to digital YCbCr 4:2: component video by the video decoder TVP5150A. In the digital media processor
a visible image sensor and ripple algorithm DM642, the color images of the video are converted to gray images by abandoning the channels of Cb and Cr, and then the gray images are processed by the ripple algorithm. The output digital signal from the DSP is converted to PAL (50Hz) analog video signal by the video encoder SAA7105H. The output of the SAA7105H can be displayed on the LCD screen. The block diagram of the DSP- based system structure is shown in Fig 2.3. Fig 2.1. System configuration Fig 2.2.USTB-CK923 platform . Fig 2.3. Block diagram of DSP-based system.
a visible image sensor and ripple algorithm
[ ]
algorithm can obtain almost the same feature values when the view angles are symmetrical about the X-axis or Y-axis. For example, when the view angles are 45°, 135°, 225° and 275°, the obtained feature values will be nearly the same. Consequently, we only consider view angles in the range from 0° to 90°. Note that if the targets are not symmetrical, the range from 0° to 180° should be used. Specifically, a total of 18 reference images were taken for each target from different view angles (with an increment of 5°) on a clear background. The acquired gray-level images were converted into binary chips using threshold determined by the difference between the mean value and the standard deviations of the whole image. The holes in the binary chips are filled with “black” color. Fig 3.2 shows an example of converting an image to binary chips. The similarity between two binary images A and B are defined as
(𝐴 & 𝐵)
Similarity =
(𝐴|𝐵)+ 1
Where& and | denote AND logic operation and XOR logic operation respectively, and #(X) denotes the number of pixels that are “1” in the binary image X. The similarity definition in (3.1) can be used as a classification criterion to classify the binary chips into several groups. After obtaining the binary chips, we can compute M binary templates by using unsupervised learning approach illustrated in Fig 3.3. The procedure is shown below. Step 1: Randomly select M initial templates from the binary chips. Step 2: Cluster the binary chips into M groups using similarity features: the M similarity values between an input chip and the M initial templates are computed using (3.1). The initial template having the highest similarity determines the group of the binary chip. Hence, the binary chip can be classified into one of the M groups, each of which is represented by a template. Step 3: After clustering, the binary templates are updated for the M groups (see Fig 3.4). Let the number of binary chips in a group be Nc, where c = 1,2,...M. If the number of binary chips having target pixels at (x, y) is larger than or equal to 1 2 Nc, the specific coordinate (x, y) is regarded as a valid pixel and updated as a target pixel
a visible image sensor and ripple algorithm in the updated template. Otherwise, the coordinate (x, y) will be considered as a non- target pixel. Step 4: Replace the initial M templates with the updated templates, and repeat the steps 2 – 4 until there is no further change in the template update. Fig 3.1.Definition of view angle Fig 3.2. Conversion of image to binary image. (a) Original image. (b) Binary image.
a visible image sensor and ripple algorithm Step 1: The global adaptive threshold is used to do the initial segmentation. The threshold value for an image is determined by the difference between the mean and the standard deviations of the whole image
Threshold value = mean – std (3.2)
Step 2: The region is considered as the candidate target region if the number of the contiguous group of black pixels in the region exceeds the threshold. Step 3: Label the candidate target region. Then the initial candidate target block is expanded by a half of the initial width in the horizontal direction and a half of the initial height in the vertical direction, as shown in Fig 3.5. Step 4: Perform binarization of the original image on the expanded block with a more precise threshold value, which can effectively discriminate the real targets from the background in the expanded block area. Then compute the threshold value for the expanded block using (3.2). Step 5: Dilation and erosion are used to fill holes in the binary chips. And the 3 × 3 square is used as the structuring element. Fig 3.5. Expanded block for first candidate target.
3.3 FEATURE EXTRACTION AND TARGET RECOGNITION
Consider the binary image shown in Fig 3.6. It contains a vehicle target and a small plant which acts as a disturbing factor. The vehicle target is segmented as an individual region which is within the circumscribed rectangle. A group of four concentric circles are drawn and each circle superimposes the vehicle target. The
a visible image sensor and ripple algorithm center of mass of each connected component is computed and considered as the center of a group of concentric circles with different radii. The maximal radius is defined by the minimal distance between the center of mass and the four vertexes (such as A, B, C or D) of the rectangle. For instance, suppose O is the center of mass of the vehicle, if the distance between O and D is the shortest one, OD is selected to be the largest radius in concentric circle. The maximal radius is divided into four equal parts and four concentric circles are drawn. The radius of the j-th circle is denoted by Rj, where the unit of Rj is pixel. Obviously, the minimal radius R1 is equal to one fourth of the maximal radius R4. For the j-th concentric circle, the area of the vehicle region, denoted by X j, is calculated by counting the black pixels in the j-th concentric circle. Define the area rate Aj by
Aj =X j/ πR^2 j ×100% (1 ≤ j ≤ 4) (3.3)
Which represents one type of target feature another type of target feature is the rate of circle is defined as
Pj = Nj/2πRj ×100% (1 ≤ j ≤4) (3.4)
Where Nj represents the number of black pixels overlapping between the j-th circle and the vehicle. For example, in Fig 3.6, the arcs such as ab, cd and ef are the parts super- imposed by the circle 4 and the vehicle target. The arcs ab and cd are taken into consideration but the arc ef which is outside the circumscribed rectangle (ABCD) is not considered. Note that in this study, we only used four concentric circles. To achieve higher recognition accuracy, more concentric circles should be used. Now, let us have a look how the proposed features Aj and Pj respond to region translation, rotation and scaling. When the region rotates in any degree, the relative position of the region barycenter does not change. Since the circle characteristic is invariant to rotation, the number of black pixels overlapping between circle and region is not changed. So does the area of the region. When the target scales up (or scales down), each region acts in the same way. As a result, all circles scale up (or scale down) simultaneously. When the target translates, the position of target barycenter and the circles move with region simultaneously. Thus, Aj and Pj do not
a visible image sensor and ripple algorithm
3. EXPERIMENTATION AND RESULTS
In this section, five experimental examples are provided to demonstrate the effectiveness of the developed ATD system. The first experiment evaluates the performance of the feature vector in (6) using different number of templates. The second example assesses the performance of our system under variouslight conditions. The third example investigates whether our system is able to deal with image rotation or target translation and scaling. The fourth one verifies the ability of our system in detecting target from different view angles. The last one tests the detection rate of the system and the average processing time required to recognize one image, and compares our algorithm with the affine moment invariants approach.
4.1 PERFORMANCE OF THE FEATURE VECTOR UNDER
DIFFERENT NUMBER OF TEMPLATES
In this experiment, the single vehicle target scenario was considered. The ripple algorithm was utilized to obtain the feature vectors of the M binary templates and those of the target candidate regions. By comparing the two sets of feature vectors, we decided whether the target candidate regions represent the real target or not, and calculated the detection rate. We chose σij = εij = 0.1, and considered M = 1, 2, 3, 4, respectively. Fig 4.1 shows the detection rate versus the number of binary templates. We can see that the detection rates for M = 1, 2, 3, 4 are 35%, 66%, 95% and 96%, respectively. Since the detection rates for M =3 and M =4 are very close, one can choose M = 3 to reduce computation and processing time. For M = 3, (5) can be simplified as M0 = A 11 A 12 A 13 A= P 11 P 12 P 13 P 14 A 21 A 22 A 23 A 24 P 21 P 22 P 23 P 24 A 31 A 32 A 33 A 34 P 31 P 32 P 33 P 34 and Fig 4.2 shows the three templates obtained.
a visible image sensor and ripple algorithm Fig 4.1.Detection rate versus number of binary templates Fig 4.2.Three binary templates obtained by offline template computing
4.2 PERFORMANCE UNDER DIFFERENT LIGHT CONDITIONS
The visible image sensor was used to capture test images under different light conditions. And bright and dark test images were obtained. As shown in Fig 4. and Fig 4.4, the vehicle target is detected and labeled by the cross and the circumscribed rectangle. The non-target region is only labeled by the circumscribed rectangle. To find the relation between detection rate and light condition, we built the vehicle target image database (see the description in the subsection III-A) under common light condition. The detection rates were calculated based on the database. As shown in Fig 4.5, the detection rates under dark and bright light conditions are 91% and 93%, respectively. The detection rate increases to about 96% under common light condition. Clearly, the detection rates under dark and bright light conditions are just slightly lower than that under common light condition. This means that our system is robust to different light conditions.
a visible image sensor and ripple algorithm
4.3 CAPABILITY OF DEALING WITH IMAGE ROTATION AND
TARGET SCALING
In the application of robots, a common occurrence is that the robot swings when it walks. As a result, the images pivot on a shaft through the optical axis. At the same time, when the distance between the robot and the target changes, the size and position of the target also alter. In this experiment, an intensity image was rotated clockwise to simulate the condition of robot swing and two targets with different sizes were placed in the image. Specifically, the image was rotated by 10°, 30°, 45°, 60°, 90°, 135° and 180°, respectively, as shown in Fig 4.6. Table 4.1 shows the target features against rotation angles and scaling of image. One can see that when the image rotates in the range from 0° to 180°. For example, the maximum and minimum values of A’1 are 66.38% and 61.02%, respectively, which gives a variation of 5.36%. This means that the ripple is robust to image rotation. The robustness of the ripple algorithm to target translation and scaling has also been verified since the scale and position of the targets in these images are different from each other. Fig 4.6 Robustness to image rotation or target translation and scaling (a) 0° (b) 10° (c) 30° (d) 45° (e) 60° (f) 90° (g) 135° (h) 180°
a visible image sensor and ripple algorithm Table 4.1: Quasi invariance of target features to rotation and scaling
4.4 TARGET DETECTION FROM DIFFERENT VIEW ANGLES
In practice, the vehicle target may be viewed from different angles or rotate itself. In such a case, its contour will make projective transformation and thus the feature of the target con- tour will change. Many templates explained above were used to represent the projective transformation of the target contour. Here, we verify the validity of our method by detecting targets from different view angles in static images. It can be seen from Fig 4.7 that a total of 12 images of a vehicle model placed on sandy land were taken from different view angles with an increment of 30°. The view angles are0°, 30°, 60°, 90°, 120°, 150°, 180°, 210°, 240°, 270°, 300° and 330°, respectively. The vehicle model can be detected successfully from every view angle and is labeled by a cross and a circumscribed rectangle in each case. After that, we tested our method in the case that the images were polluted by salt and pepper noise. As shown in Fig 4.8, the vehicle targets in these images have different view angles and sizes. They are detected and labeled by the cross and the circumscribed rectangle.
a visible image sensor and ripple algorithm circumscribed rectangle. The non- target region is only labeled by the circumscribed rectangle. The experimental result based on 1000 airplane images shows that the recognition rate of the proposed algorithm reaches 93.8%. This indicates that our system is robust to different targets. Fig 4.9 Airplane target recognition
4.6 RECOGNITION RATE AND AVERAGE PROCESSING TIME
In this experiment, we examined our ATD system using a vehicle video sequence of 1000 frames to obtain the detection rate and the average processing time. The frames were captured outdoor by a visible image sensor. The size of each frame is of 576 × 720 pixels and the frame rate is 30 frames per second. The time required for inputting one frame image data is denoted by T start and the ending time of processing one frame image data is denoted by Tend. Hence Tend−T start is the time an algorithm would cost. The same video sequence was also processed by the affine moment invariants algorithm, in which the order was chosen to be 3. Table 4.2 shows the comparison results. To the original images, as we can see that a true detection rate of 94.2% has been achieved by the ripple algorithm. In contrast, the affine moment invariants algorithm achieves a detection rate of 91.2%. Also, the ripple algorithm has less missed frames and lower false recognition rate than the affine moment invariants algorithm. And the average process time is about 21ms per frame by the ripple algorithm, compared to 35ms per frame by the affine moment invariants algorithm.
a visible image sensor and ripple algorithm When the salt and pepper noise is added to the video images, the true detection rate is 93.6% by the ripple algorithm and 89.4% by the affine moment invariants algorithm. And the missed out frames by the two methods are 18 frames and 27 fames, respectively. From the Table 4.2, we know that the true detection rate of the ripple algorithm is 94.2% by the original image and 93.6% by the image with noise. The true detection rate is decreased by 0.6% due to the influence of noise. Also, the true detection rate of the affine moment invariants algorithm is 91.2% by the original image and 89.4% by the image with noise. It is decreased by 1.8% due to the influence of noise. The average process time is 21ms by the ripple algorithm and 35ms by the affine moment invariants algorithm. So, the ripple algorithm is more suitable for real-time applications and has better robustness. The reasons that the ripple algorithm outperforms the affine moment invariants algorithm are as follows. The affine moment invariants approach can obtain invariant features when the contour of the target makes affine transform. When the vehicle target is in different view angles, the contour of the target makes projective transformation, which can be approximately considered as affine transform in some situations. However, in many other cases, the projective transformation cannot be approximately considered as affine transform. Consequently, the target features obtained by the affine moment invariants approach vary considerably. On the contrary, the ripple algorithm employs several templates to represent the projective transformation of the vehicle contour, so it can achieve a better result. Moreover, although the higher-order moments can describe the contour details of the target, with the increase of the order of moment, the amount of calculation by the affine moment invariants algorithm increases rapidly.
