-
Global Histogram Equalization(GHE) is a popular technique to enhance the global contrast of images, since it is computationally fast and simple to implement[1]. Based on flattening the histogram and stretching the dynamic range of the gray levels by using the Cumulative Density Function(CDF) of image, GHE technique tends to homogenize the distribution of pixels in the image, expand the dynamic range of the original image, and improve the specific high contrast of the image. Despite its success for image contrast enhancement, this technique has a recognized drawback that it does not preserve the brightness of the input image on the output one. However, it will also lead to, such as over-enhancement, increasing the contrast of background noise, bluring the image details and reducing the contrast of useful signals[2]. It makes the use of GHE unsuitable for image contrast enhancement on consumer electronics such as camcorder, where preserving the input brightness is essential to avoid the generation of non-existing artifacts in the output image.
To overcome such drawback, many scholars proposed various effective Mean Brightness Preserving Histogram Equalization Methods(MBPHE). Kim first presented Brightness preserving Bi-Histogram Equalization(BBHE)[3], which divided the histogram into two parts with the input mean brightness and equalized the two sub histograms independently. Then, Equal Area Dualistic Sub-Image Histogram Equalization(DSIHE) used the median intensity value as the separating point, and claimed that it is better than BBHE in terms of preservation of brightness and average information content of an image[4]. Chen et al. introduced Minimum Mean Brightness Error Bi-Histogram Equalization(MMBEBHE) for preserving the mean brightness "optimally" [5]. Sim et al. presented Recursive Mean-Separate Histogram Equalization(RSIHE)[6]. This algorithm performed the division of histogram based on median value of brightness instead of mean brightness. Zuo et al. presented Range Limited Bi-Histogram Equalization(RLBHE)[7], etc. RLBHE divided the input image histogram into two parts based on the Otsu′s method. Xiu et al. presented saliency histogram, which increased the amount of calculation a lot[9]. Methods such as the retinex method for the image enhancement and the adaptive image enhancement based on NSCT coefficient histogram matching are really extremely time-consuming[10-11]. Histogram of oriented gradient mentioned by Xiao et al. did behave well, but still too redundant for the real time system[12]. Cao et al. raised a method of image enhancement using clustering and histogram equalization, but the method of image segmentation is much more complicated[13]. Modified Contrast Limited Adaptive Histogram Equalization method made the system more complicated[14-16].
However, these above discussed algorithms can preserve the mean brightness only to a certain extent. And they might generate images that do not look as natural as the input ones, which is unacceptable for consumer electronics products[2]. Furthermore, the most basic requirement of MBPHE to preserve the original mean brightness is that the input histogram has a quasi-symmetrical distribution around its separating point[8]. But most of the input histogram do not have this property, which leads to the failure of MBPHE in preserving the mean intensity in real applications.
In order to enhance contrast, preserve brightness and produce natural looking images, we put forward a new adaptive N-thresholds histogram equalization algorithm called Range Limited Adaptive Multi-Histogram Equalization(RLAMHE). First of all, we search the peaks number (N+1) in the smoothed histogram of the image. Extended N-threshold Otsu′s method is used to obtain N-threshold of the image. Then, the range of the equalized image is limited to guarantee that the mean brightness of the output image is almost consistent with the mean brightness of the input image. In this paper, GHE and RLBHE methods and their mathematical formula are reviewed in section 2 and 3. Section 4 contributes to the RLAMHE method. Section 5 lists a few experimental results to illustrate the performance of RLAMHE. Section 6 comes the conclusion of this paper.
-
Let′s suppose that f(i, j)=I={I(i, j)} stands for a digital image, where I(i, j) represents the gray level of the pixel at (i, j). n denotes the total number of the image pixels, and the image intensity is digitized and divided into L levels as {I0, I1, I2, …, IL-1}. So it is obvious that ∀I(i, j)∈{I0, I1, I2, …, IL-1}. Suppose nk stands for the total number of pixels with the gray level of Ik in the image, then the Probability Density Function(PDF) p(Ik) can be written as follow:
(1) Based on the image′s PDF, its Cumulative Distribution Function(CDF) is defined as:
(2) It is easy to know that c(IL-1)=1. The transform function T(I) can be defined below based on the CDF:
(3) Here T(I) is a linear function of I.
Then the output image of the GHE, O={O(i, j)} can be written as follow:
(4) Suppose that I is a continuous random variable, i.e., L=∞, then the output image O is also regarded as a continuous random. It is obvious that the PDF of the output gray level of O follows a uniform distribution because T(I) is a linear function, i.e., the density function of the output image would be distributed over the whole range. The mean brightness of the output image can be expressed as:
(5) where E(O) stands for a statistical expectation. Since E(O) is a constant that is irrelative to the brightness of the input image, the GHE algorithm does not take the mean brightness of the input image into consideration. The GHE algorithm cannot be applied into the electronics such as the digital camera due to the brightness change of the input image.
-
RLBHE algorithm is formally defined by the following procedures:
-
Otsu′s method is used to automatically separate the image into two parts, including the target region and the background. The algorithm assumes that the image to be thresholded contains two classes of pixels(e.g., foreground and background), then the optimum threshold is calculated to separate those two classes so that their intra-class variance is maximum.
(6) where E(XL) and E(XU) stand for the mean brightness of the two sub-images thresholded by XT, respectively. E(X) is the mean brightness of the whole image. WL and WU stand for the fractions to indicate the numbers of two classes of pixels of the whole:
(7) (8) -
The mean brightness of the output image of bi-histogram equalization using XT is as follow:
(9) The output image should keep the mean brightness of the original image as much as possible:
(10) To make Eq.(10) hold, the range of equalized image is modified. Two variables X′L-1 and X′0 are used to replace the upper bound XL-1 and the lower bound X0, where X′L-1 and X′0 are chosen to yield minimum AMBE between the equalized image and the original image.
-
Then what to do is to equalize each sub-histogram independently. It is same with all bi-histogram equalization methods except for the mapping range. That is, the output image of RLBHE, Y, is finally expressed as:
(11) -
Refering to the RLBHE algorithm, the RLAMHE algorithm can be mainly divided into the following steps:
(1) Searching the peaks number (N+1) in the smoothed histogram of the image;
(2) Choosing N-threshold to separate the input image;
(3) Determining the upper and the lower bounds for histogram equalization;
(4) Equalizing each partition independently;
Then in the following subsection, the details of each step will be discussed.
-
The histogram of an image will be consisted of many peaks or modes. Each peak of histogram cannot be easily detected since the probability of brightness levels is fluctuated. The neighborhoods averaging process is applied to smooth the histogram. Nine consecutive probabilities of brightness levels are averaged for the new probability of the central brightness level. This new probability value of the central brightness level k, defined as pn(Ik), can be obtained by the following equation:
(12) This new probability pn(Ik) is used only in the break point detection process. In order to obtain the break point of each peak, the signs of the difference between two successive probabilities in the smoothed histogram are calculated. Each break point in the smoothed histogram is detected when four successive negative signs are followed by the appearance of eight successive positive signs. That means, the break point is detected only on the downward path of probabilities. The histogram is composed of (N+1) peaks, and the number of break points N must be obtained.
Here we take the image Lena as an example. As can be seen from Fig. 1(b), the original histogram is smoothed by Eq.(12). The number of peaks in the histogram is significantly reduced, and only the big ones are reserved, which can help to seek the number of (N+1) much easier.
-
In RLBHE algorithm, using Otsu′s method to choose a single threshold for histogram separation requests that the density histogram of image has a obvious bimodal characteristics. But in fact, the density histogram of image always has more than two peaks, i.e., three or more. When dealing with such multi-objective or complex background image, the RLBHE algorithm cannot separate the image well with only a single threshold. For example, the smoothed histogram of Fig. 1(a) has a obvious five peaks characteristic. In this condition, more than one thresholds are required to divide the image into five parts.
Inspired by the single threshold Otsu′s method, we deduced the N-threshold Otsu′s method, then divided the image into (N+1) parts with the N-thresholds T1, T2, T3, …, TN. The algorithm assumes that the image to be thresholded contains (N+1) classes of pixels.
Assumed that f(i, j)=I={I(i, j)} stands for a digital image, where I(i, j) stands for the gray level of the pixel at (i, j). n denotes the total number of the image pixels, and the image intensity is digitized and divided into L levels as { I0, I1, I2, …, IL-1}. So it is obvious that ∀I(i, j) ∈{I0, I1, I2, …, IL-1}. Using the N-threshold T1, T2, …, TN, obviously, T1 < T2 < … < TN∈{I0, I1, I2, …, IL-1}, the input image I can be decomposed into (N+1) sub-images IT1, IT2, IT3, …, ITN+1 as
(13) where
(14) (15) and
(16) Then, the PDF of the sub-images IT1, IT2, IT3, …, ITN+1 can be written as follows:
(17) (18) (19) where, nk stands for the total number of pixels with the gray level of Ik in IT1, IT2, IT3, …, ITN+1, and n is the total number of pixels in I. Then the CDF for IT1, IT2, IT3, …, ITN+1 can be defined as
(20) (21) (22) Thus, the transform function can be defined as
(23) (24) (25) then the adaptive thresholds for dividing those (N+1) classes were calculated so that their inter-class variance is maximum:
(26) where T1, T2, …, TN is the N-threshold, pT1, pT2, …, pTN+1 stand for the PDF of the sub-images IT1, IT2, …, ITN+1. E(IT1), E(IT2), …, E(ITN+1) are defined as the mean brightness of the (N+1) sub-images divided by T1, T2, …, TN. E(I) is the mean brightness of the whole image.
In order to acquire the best segmentation, the Otsu′s method requests that the mean of the sub-images divided by the threshold should be far away from the center of the image. And the gray mean of the image is used to represent the target and the background. In this paper, the average variance is used instead of the mean of the image gray value because the average variance reflects the uniformity of the image gray scale distribution. The image gray scale distribution within the target and the background area is homogeneous, but the gray scale of the boundary and the surrounding pixels changes heavily. Thus, the average variance can be regarded as a reflection of the gray mutation between the boundary and the surrounding pixels. If the average variance of certain sub-image is close to that of the whole image, that is likely to divide the whole boundary and the surrounding pixels into this part, which means the wrong segmentation. According to the analysis above, it is reasonable to use the average variance instead of the mean of the image in Otsu′s method. Thus Eq.(26) can be written as
(27) where
(28) (29) (30) (31) (32) According to Eq.(27), the algorithm exhaustively searches for the thresholds that maximize the inter-class variance.
Fig. 2(b) and 2(c) show the separation result of Fig. 2(a) using the single threshold Otsu′s method. And the location of T is shown in Fig. 2(d).
Figure 2. (a) Original image of Lena. (b), (c)Separation results using the single threshold Otsu′s method. (d)Location of threshold using the single threshold Otsu′s method
Fig. 4(b), 4(c), 4(d), 4(e) and 4(f) show the separation result of Fig. 1(a) using the N-thresholds Otsu′s method. And the location of T1, T2, …, TN is shown in Fig. 3(b). It can be seen that the N-threshold Otsu′s method yields a more reasonable result than the single threshold Otsu′s method.
-
In the application such as the mobile camera, the preservation of the mean brightness is always of high requirement. Even though the thresholds searched by the N-threshold Otsu′s method can divide the input image effectively, but the mean brightness cannot be kept intact. Thus, the new upper and the lower bounds for histogram equalization should be determined to improve the defect as well as possible. The mean brightness of the output image of multi-histogram equalization using T1, T2, …, TN is as follows
(33) In order to keep the mean brightness of the original image as much as possible, the mean of the output image should meet the following equation:
(34) where I and O denote the input and output image, Im stands for the mean of the input image.
Here, let′s define
(35) (36) (37) Thus, it is obvious that
(38) By substituting Eq.(33), (35), (36), (37) into Eq.(34), we get
(39) As can be seen from Eq.(39), T1, T2, …, TN can be got by the N-threshold Otsu′s method, and
, …, aN+1= and Im can be easily got from the input image because they are irrelativant to the output image, but related to the input image only. Thus, the new lower bound I′0 and the upper bound I′L-1 are needed to substitute for the I0 and IL-1 to make Eq.(34) hold. Here we define that 0≤I′0≤T1 and TN≤I′L-1≤L-1. I′0 and I′L-1 should ensure that the AMBE is minimum between the output image and the input image:(40) In Eq.(40), since a1, a2, …, aN+1, T1, T2, …, TN andIm can be calculated beforehand, so we define that
(41) Thus Eq.(40) can be written as
(42) As can be seen from Eq.(41), it is a simple quadric optimization problem with a unique solution. Thus, the solution I′0 and I′L-1 ensure that the AMBE is minimum between the output image and the input image so that the brightness of the input image can be kept as much as possible.
-
The final step in RLAMHE is to equalize each sub-images independently. It is all the same with any bi-histogram equalization methods except for the new range I′0 and I′L-1. The transform functions Eq.(23), (24) and (25) using I′0 and I′L-1 instead of I0 and IL-1 can be expressed as follows:
(43) (44) (45) Based on the above three transform functions, we equalized the sub-images independently and finally the output image of RLAMHE is composed of the results of the equalized sub-images.
Thus, the output image O is as follows:
(46) where
(47) (48) (49) -
Besides the RLAMHE, this paper also realized the GHE and RLBHE algorithms as references. Wide varieties of standard images ranging from under exposed to over exposed low contrast to high contrast, dark background to bright background, are chosen to test the robustness and versatility of the RLAMHE method. Here we present an analysis of 3 images including Lena, House and Arplane U2. These results from Fig. 5-Fig. 7 show the superiority of RLAMHE in all the images in terms of contrast enhancement and control on over enhancement.
Figure 7. (a) Original image of Airplane U2.(b)Result of GHE.(c) Result of RLBHE.(d) Result of RLAMHE
The test image Lena(Fig. 5) is a typical example that shows the white saturation effect, which often results from GHE methods. The result of RLAMHE shows more details and the contrast of the face and the shoulder is significantly improved than that of RLBHE. The RLAMHE method generates better enhancement around the hat than the RLBHE method did, while being natural looking.
The concrete results in terms of contrast enhancement can be clearly observed in Fig. 6. It is easy to see that the tower and the sky are over enhanced by GHE. The windows of the house and the tower are obscured by RLBHE. However, RLAMHE provides control on over enhancement leading to a good appearance. Observing the house and the tower in the image, we can perceive contrast enhancement. The front edge of the tower and the squares of the house shown in the blue rectangle can be seen clearly. By observing the processed images, it is noticeable that our proposed method is the only one among the other methods that can produce natural looking images.
It is obvious that the result of GHE in the test image Airplane U2(Fig. 7) changes the intensity values abruptly, and therefore, tends to produce level saturation effect. RLBHE loses many details of the wing and the brightness is not kept well. The result of RLAMHE shows that the mean brightness is preserved well and the detail of the wing is also well enhanced.
After visually observing some processed images, we can conclude that: (1)The RLAMHE method produces images with better quality than the other methods; (2)It also presents satisfactory brightness preserving and natural looking images.
Tab. 1 shows the AMBE for the three algorithms mentioned above. From Tab. 1, we can see that the resulting AMBE for RLAMHE of Lena is 0.414 6, which is much better than the resulting AMBE for RLBHE(0.629 5).
Table 1. The resulting AMBE for GHE, RLBHE and RLAMHE
GHE RLBHE RLAMHE Lena 3.250 1 0.629 5 0.414 6 House 70.008 8 7.116 6 2.954 2 AirplaneU2 94.997 9 5.963 2 1.339 8 -
In this paper, the range limited adaptive multi-threshold histogram equalization(RLAMHE) algorithm with brightnes preserving is presented. We tested the proposed RLAMHE algorithm on an real time video processing system. The experimental results show that the proposed RLAMHE is able to achieve visually pleasant enhancement effects. The over-enhancement and level saturation artifacts are effectively avoided. Compared with many other GHE-based enhancement methods, images enhanced using the RLAMHE method show well enhanced contrast and little artifacts, while being natural looking. In addition, the RLAMHE method is computationally simple and suitable for processor-based implementation.
Range limited adaptive brightness preserving multi-threshold histogram equalization algorithm
-
摘要: 针对目前直方图均衡算法难以实现,且易造成亮度饱和等问题,本文提出了一种范围限制的自适应亮度保持多阈值直方图均衡算法。首先,对输入图像进行适当平滑,从而获得它的直方图峰值点个数(N+1)。然后,对Otsu算法进行N阈值扩展,并通过这种方法获得图像的N个分割阈值,从而按照此阈值对图像进行分割。为了能够最大程度地保持输入图像的亮度,利用输入图像和输出图像的均值亮度最小误差(AMBE)准则,重新计算了图像的均衡范围。最后,利用新的均衡范围分别对每一个子图像进行均衡。实验表明,使用本算法处理Lena图的绝对均值亮度误差为0.416 4,明显优于使用RLBHE算法的0.629 5。本算法能够获得更清晰的图像细节,同时图像的整体亮度保持的也较好。Abstract: In recent years, many histogram equalization algorithms have been proposed for the consumer electronics field. However, many of these algorithms are hard to realize. Even, for example, some algorithms may cause an effect on brightness saturation. Therefore, a range limited adaptive brightness preserving multi-threshold histogram equalization(RLAMHE) algorithm is presented in this paper. First, the input image is smoothed appropriately to obtain the number of its histogram peak points (N+1). Then the Otsu algorithm is extended by the N-threshold, and N segmentation thresholds of the image are obtained in this way, so that the image is segmented according to this threshold. In order to maximize the brightness of the input image, a range of the equalized image is recalculated according to the minimum Absolute Mean Brightness Error(AMBE) criterion of the input and the output image. Finally, all sub-images are equalized separately using the new equalization range. Test results show that the proposed algorithm is more efficient than other algorithms and can obtain sharper image details. Meanwhile, the overall brightness of the image is also ideal. Using this algorithm to process Lena graphs, the absolute mean luminance error is 0.416 4, which is obviously better than that obtained using RLBHE algorithm(0.629 5).
-
Key words:
- contrast enhancement /
- histogram equalization /
- brightness preserving /
- range limited /
- N-threshold
-
Table 1. The resulting AMBE for GHE, RLBHE and RLAMHE
GHE RLBHE RLAMHE Lena 3.250 1 0.629 5 0.414 6 House 70.008 8 7.116 6 2.954 2 AirplaneU2 94.997 9 5.963 2 1.339 8 -
[1] CHEN H O, NICHOLAS S P K, HAIDI I. Bi-histogram equalization with a plateau limit for digital image enhancement[J]. IEEE Trans. Consum. Electron, 2009, 55:2072-2080. doi: 10.1109/TCE.2009.5373771 [2] MENOTTI D, NAJMAN L, FACON J, et al.. Multi-hirtogram equalization methods for contrast enhancement and brightness preserving[J]. IEEE Trans. Consum. Electron, 2007, 53:1186-1194. doi: 10.1109/TCE.2007.4341603 [3] KIM Y T. Contrast enhancement using brightness preserving bi-histogram equalization[J]. IEEE Trans. Consum. Electron, 1997, 43:1-8. doi: 10.1109/30.580378 [4] WAN Y, CHEN Q, ZHANG B M. Image enhancement based on equal area dualistic sub-image histogram equalization method[J]. IEEE Trans. Consum. Electron, 1999, 45:68-75. doi: 10.1109/30.754419 [5] CHEN S D, RAMLI A R. Minimum mean brightness error bi-histogram equalization in contrast enhancement[J]. IEEE Trans. Consum. Electron, 2003, 49:1310-1319. doi: 10.1109/TCE.2003.1261234 [6] SIM K S, TSO C P, TAN Y Y. Recursive sub-image histogram equalization applied to gray scale images[J]. Pattern Recognit. Lett., 2007, 28:1209-1221. http://www.sciencedirect.com/science/article/pii/S0167865507000578 [7] ZUO CH, CHEN Q, SUI X B. Range limited bi-histogram equalization for image contrast enhancement[J]. Optik, 2013, 124:425-431. doi: 10.1016/j.ijleo.2011.12.057 [8] WANG CH, YE ZH F. Brightness preserving histogram equalization with maximum entropy:a variational perspective[J]. IEEE Trans. Consum. Electron, 2005, 51:1326-1334. http://ieeexplore.ieee.org/xpls/icp.jsp?arnumber=1561863 [9] XIU CH B, WEI SH A N. Camshift tracking with saliency histogram[J]. Optics and Precision Engineering, 2015, 23(6):1749-1757. http://www.opticsjournal.net/Abstract.htm?id=OJ1508250005958EaGdJ [10] LI Y, ZHANG Y F, NIAN L, et al.. [J]. Chinese J. Liquid Crystals and Displays, 2016, 31(1):104-111.(in Chinese) http://www.opticsjournal.net/abstract.htm?id=OJ160322000625x5A7D0 [11] ZHOU Y, LI Q W, HUO G Y. Adaptive image enhancement based on NSCT coefficient histogram matching[J]. Optics and Precision Engineering, 2014, 22(8):2215-2222.(in Chinese) http://en.cnki.com.cn/Article_en/CJFDTotal-GXJM201408032.htm [12] XIAO CH M, SHI Z L, LIU Y P. Metrics of image background clutter by introducing gradient features[J]. Optics and Precision Engineering, 2015, 23(12):3472-3479.(in Chinese) http://www.opticsjournal.net/abstract.htm?id=OJ160122000089jPlSoV [13] CAO J F, SHI J CH, LUO H B, et al.. Image enhancement using clustering and histogram equalization[J]. Infrared and Laser Engineering, 2012, 41(12):3436-3441.(in Chinese) http://en.cnki.com.cn/Article_en/CJFDTOTAL-HWYJ201212052.htm [14] YUN H J, WU ZH Y, WANG G J, et al.. Enhancement of infrared image combined with histogram equalization and fuzzy set theory[J]. J. Computer-Aided Design & Computer Graphics, 2015, 27(8):1498-1505.(in Chinese) [15] CHEN Y, ZHU M. Multiple sub-histogram equalization low light level image enhancement and realization on FPGA[J]. Chinese J. Optics, 2014, 7(2):225-233.(in Chinese) http://en.cnki.com.cn/Article_en/CJFDTotal-ZGGA201402006.htm [16] CAI SH D, YANG F. Image enhancement based on histogram modification[J]. Optoeletronic Technology, 2012, 32(3):155-159.(in Chinese) http://en.cnki.com.cn/Article_en/CJFDTOTAL-GDJS201203004.htm -