Global Histogram Equalization(GHE) is a popular technique to enhance the global contrast of images, since it is computationally fast and simple to implement^[1]. Based on flattening the histogram and stretching the dynamic range of the gray levels by using the Cumulative Density Function(CDF) of image, GHE technique tends to homogenize the distribution of pixels in the image, expand the dynamic range of the original image, and improve the specific high contrast of the image. Despite its success for image contrast enhancement, this technique has a recognized drawback that it does not preserve the brightness of the input image on the output one. However, it will also lead to, such as over-enhancement, increasing the contrast of background noise, bluring the image details and reducing the contrast of useful signals^[2]. It makes the use of GHE unsuitable for image contrast enhancement on consumer electronics such as camcorder, where preserving the input brightness is essential to avoid the generation of non-existing artifacts in the output image.

To overcome such drawback, many scholars proposed various effective Mean Brightness Preserving Histogram Equalization Methods(MBPHE). Kim first presented Brightness preserving Bi-Histogram Equalization(BBHE)^[3], which divided the histogram into two parts with the input mean brightness and equalized the two sub histograms independently. Then, Equal Area Dualistic Sub-Image Histogram Equalization(DSIHE) used the median intensity value as the separating point, and claimed that it is better than BBHE in terms of preservation of brightness and average information content of an image^[4]. Chen et al. introduced Minimum Mean Brightness Error Bi-Histogram Equalization(MMBEBHE) for preserving the mean brightness "optimally" ^[5]. Sim et al. presented Recursive Mean-Separate Histogram Equalization(RSIHE)^[6]. This algorithm performed the division of histogram based on median value of brightness instead of mean brightness. Zuo et al. presented Range Limited Bi-Histogram Equalization(RLBHE)^[7], etc. RLBHE divided the input image histogram into two parts based on the Otsu′s method. Xiu et al. presented saliency histogram, which increased the amount of calculation a lot^[9]. Methods such as the retinex method for the image enhancement and the adaptive image enhancement based on NSCT coefficient histogram matching are really extremely time-consuming^[10-11]. Histogram of oriented gradient mentioned by Xiao et al. did behave well, but still too redundant for the real time system^[12]. Cao et al. raised a method of image enhancement using clustering and histogram equalization, but the method of image segmentation is much more complicated^[13]. Modified Contrast Limited Adaptive Histogram Equalization method made the system more complicated^[14-16].

However, these above discussed algorithms can preserve the mean brightness only to a certain extent. And they might generate images that do not look as natural as the input ones, which is unacceptable for consumer electronics products^[2]. Furthermore, the most basic requirement of MBPHE to preserve the original mean brightness is that the input histogram has a quasi-symmetrical distribution around its separating point^[8]. But most of the input histogram do not have this property, which leads to the failure of MBPHE in preserving the mean intensity in real applications.

In order to enhance contrast, preserve brightness and produce natural looking images, we put forward a new adaptive N-thresholds histogram equalization algorithm called Range Limited Adaptive Multi-Histogram Equalization(RLAMHE). First of all, we search the peaks number (N+1) in the smoothed histogram of the image. Extended N-threshold Otsu′s method is used to obtain N-threshold of the image. Then, the range of the equalized image is limited to guarantee that the mean brightness of the output image is almost consistent with the mean brightness of the input image. In this paper, GHE and RLBHE methods and their mathematical formula are reviewed in section 2 and 3. Section 4 contributes to the RLAMHE method. Section 5 lists a few experimental results to illustrate the performance of RLAMHE. Section 6 comes the conclusion of this paper.

Let′s suppose that f(i, j)=I={I(i, j)} stands for a digital image, where I(i, j) represents the gray level of the pixel at (i, j). n denotes the total number of the image pixels, and the image intensity is digitized and divided into L levels as {I₀, I₁, I₂, …, I_L-1}. So it is obvious that ∀I(i, j)∈{I₀, I₁, I₂, …, I_L-1}. Suppose n_k stands for the total number of pixels with the gray level of I_k in the image, then the Probability Density Function(PDF) p(I_k) can be written as follow:

Based on the image′s PDF, its Cumulative Distribution Function(CDF) is defined as:

It is easy to know that c(I_L-1)=1. The transform function T(I) can be defined below based on the CDF:

Here T(I) is a linear function of I.

Then the output image of the GHE, O={O(i, j)} can be written as follow:

Suppose that I is a continuous random variable, i.e., L=∞, then the output image O is also regarded as a continuous random. It is obvious that the PDF of the output gray level of O follows a uniform distribution because T(I) is a linear function, i.e., the density function of the output image would be distributed over the whole range. The mean brightness of the output image can be expressed as:

where E(O) stands for a statistical expectation. Since E(O) is a constant that is irrelative to the brightness of the input image, the GHE algorithm does not take the mean brightness of the input image into consideration. The GHE algorithm cannot be applied into the electronics such as the digital camera due to the brightness change of the input image.

RLBHE algorithm is formally defined by the following procedures:

Otsu′s method is used to automatically separate the image into two parts, including the target region and the background. The algorithm assumes that the image to be thresholded contains two classes of pixels(e.g., foreground and background), then the optimum threshold is calculated to separate those two classes so that their intra-class variance is maximum.

where E(X_L) and E(X_U) stand for the mean brightness of the two sub-images thresholded by X_T, respectively. E(X) is the mean brightness of the whole image. W_L and W_U stand for the fractions to indicate the numbers of two classes of pixels of the whole:

The mean brightness of the output image of bi-histogram equalization using X_T is as follow:

The output image should keep the mean brightness of the original image as much as possible:

To make Eq.(10) hold, the range of equalized image is modified. Two variables X′_L-1 and X′₀ are used to replace the upper bound X_L-1 and the lower bound X₀, where X′_L-1 and X′₀ are chosen to yield minimum AMBE between the equalized image and the original image.

Then what to do is to equalize each sub-histogram independently. It is same with all bi-histogram equalization methods except for the mapping range. That is, the output image of RLBHE, Y, is finally expressed as:

Refering to the RLBHE algorithm, the RLAMHE algorithm can be mainly divided into the following steps:

(1) Searching the peaks number (N+1) in the smoothed histogram of the image;

(2) Choosing N-threshold to separate the input image;

(3) Determining the upper and the lower bounds for histogram equalization;

(4) Equalizing each partition independently;

Then in the following subsection, the details of each step will be discussed.

The histogram of an image will be consisted of many peaks or modes. Each peak of histogram cannot be easily detected since the probability of brightness levels is fluctuated. The neighborhoods averaging process is applied to smooth the histogram. Nine consecutive probabilities of brightness levels are averaged for the new probability of the central brightness level. This new probability value of the central brightness level k, defined as p_n(I_k), can be obtained by the following equation:

This new probability p_n(I_k) is used only in the break point detection process. In order to obtain the break point of each peak, the signs of the difference between two successive probabilities in the smoothed histogram are calculated. Each break point in the smoothed histogram is detected when four successive negative signs are followed by the appearance of eight successive positive signs. That means, the break point is detected only on the downward path of probabilities. The histogram is composed of (N+1) peaks, and the number of break points N must be obtained.

Here we take the image Lena as an example. As can be seen from Fig. 1(b), the original histogram is smoothed by Eq.(12). The number of peaks in the histogram is significantly reduced, and only the big ones are reserved, which can help to seek the number of (N+1) much easier.

Figure 1.  (a) Original image of Lena. (b)Comparison of the original histogram and the smoothed one of Fig. 1(a)

In RLBHE algorithm, using Otsu′s method to choose a single threshold for histogram separation requests that the density histogram of image has a obvious bimodal characteristics. But in fact, the density histogram of image always has more than two peaks, i.e., three or more. When dealing with such multi-objective or complex background image, the RLBHE algorithm cannot separate the image well with only a single threshold. For example, the smoothed histogram of Fig. 1(a) has a obvious five peaks characteristic. In this condition, more than one thresholds are required to divide the image into five parts.

Inspired by the single threshold Otsu′s method, we deduced the N-threshold Otsu′s method, then divided the image into (N+1) parts with the N-thresholds T₁, T₂, T₃, …, T_N. The algorithm assumes that the image to be thresholded contains (N+1) classes of pixels.

Assumed that f(i, j)=I={I(i, j)} stands for a digital image, where I(i, j) stands for the gray level of the pixel at (i, j). n denotes the total number of the image pixels, and the image intensity is digitized and divided into L levels as { I₀, I₁, I₂, …, I_L-1}. So it is obvious that ∀I(i, j) ∈{I₀, I₁, I₂, …, I_L-1}. Using the N-threshold T₁, T₂, …, T_N, obviously, T₁ < T₂ < … < T_N∈{I₀, I₁, I₂, …, I_L-1}, the input image I can be decomposed into (N+1) sub-images I_T1, I_T2, I_T3, …, I_TN+1 as

where

and

Then, the PDF of the sub-images I_T1, I_T2, I_T3, …, I_TN+1 can be written as follows:

where, n_k stands for the total number of pixels with the gray level of I_k in I_T1, I_T2, I_T3, …, I_TN+1, and n is the total number of pixels in I. Then the CDF for I_T1, I_T2, I_T3, …, I_TN+1 can be defined as

Thus, the transform function can be defined as

then the adaptive thresholds for dividing those (N+1) classes were calculated so that their inter-class variance is maximum:

where T₁, T₂, …, T_N is the N-threshold, p_T1, p_T2, …, p_TN+1 stand for the PDF of the sub-images I_T1, I_T2, …, I_TN+1. E(I_T1), E(I_T2), …, E(I_TN+1) are defined as the mean brightness of the (N+1) sub-images divided by T₁, T₂, …, T_N. E(I) is the mean brightness of the whole image.

In order to acquire the best segmentation, the Otsu′s method requests that the mean of the sub-images divided by the threshold should be far away from the center of the image. And the gray mean of the image is used to represent the target and the background. In this paper, the average variance is used instead of the mean of the image gray value because the average variance reflects the uniformity of the image gray scale distribution. The image gray scale distribution within the target and the background area is homogeneous, but the gray scale of the boundary and the surrounding pixels changes heavily. Thus, the average variance can be regarded as a reflection of the gray mutation between the boundary and the surrounding pixels. If the average variance of certain sub-image is close to that of the whole image, that is likely to divide the whole boundary and the surrounding pixels into this part, which means the wrong segmentation. According to the analysis above, it is reasonable to use the average variance instead of the mean of the image in Otsu′s method. Thus Eq.(26) can be written as

where

According to Eq.(27), the algorithm exhaustively searches for the thresholds that maximize the inter-class variance.

Fig. 2(b) and 2(c) show the separation result of Fig. 2(a) using the single threshold Otsu′s method. And the location of T is shown in Fig. 2(d).

Figure 2.  (a) Original image of Lena. (b), (c)Separation results using the single threshold Otsu′s method. (d)Location of threshold using the single threshold Otsu′s method

Fig. 4(b), 4(c), 4(d), 4(e) and 4(f) show the separation result of Fig. 1(a) using the N-thresholds Otsu′s method. And the location of T₁, T₂, …, T_N is shown in Fig. 3(b). It can be seen that the N-threshold Otsu′s method yields a more reasonable result than the single threshold Otsu′s method.

Figure 4.  (a) Original image of Lena.(b)~(f)Separation results using the N-threshold Otsu′s method

Figure 3.  (a) Original image of Lena. (b)Location of thresholds using the N-threshold Otsu′s method

In the application such as the mobile camera, the preservation of the mean brightness is always of high requirement. Even though the thresholds searched by the N-threshold Otsu′s method can divide the input image effectively, but the mean brightness cannot be kept intact. Thus, the new upper and the lower bounds for histogram equalization should be determined to improve the defect as well as possible. The mean brightness of the output image of multi-histogram equalization using T₁, T₂, …, T_N is as follows

In order to keep the mean brightness of the original image as much as possible, the mean of the output image should meet the following equation:

where I and O denote the input and output image, I_m stands for the mean of the input image.

Here, let′s define

Thus, it is obvious that

By substituting Eq.(33), (35), (36), (37) into Eq.(34), we get

As can be seen from Eq.(39), T₁, T₂, …, T_N can be got by the N-threshold Otsu′s method, and , …, a_N+1= and I_m can be easily got from the input image because they are irrelativant to the output image, but related to the input image only. Thus, the new lower bound I′₀ and the upper bound I′_L-1 are needed to substitute for the I₀ and I_L-1 to make Eq.(34) hold. Here we define that 0≤I′₀≤T₁ and T_N≤I′_L-1≤L-1. I′₀ and I′_L-1 should ensure that the AMBE is minimum between the output image and the input image:

In Eq.(40), since a₁, a₂, …, a_N+1, T₁, T₂, …, T_N andI_m can be calculated beforehand, so we define that

Thus Eq.(40) can be written as

As can be seen from Eq.(41), it is a simple quadric optimization problem with a unique solution. Thus, the solution I′₀ and I′_L-1 ensure that the AMBE is minimum between the output image and the input image so that the brightness of the input image can be kept as much as possible.

The final step in RLAMHE is to equalize each sub-images independently. It is all the same with any bi-histogram equalization methods except for the new range I′₀ and I′_L-1. The transform functions Eq.(23), (24) and (25) using I′₀ and I′_L-1 instead of I₀ and I_L-1 can be expressed as follows:

Based on the above three transform functions, we equalized the sub-images independently and finally the output image of RLAMHE is composed of the results of the equalized sub-images.

Thus, the output image O is as follows:

where

Besides the RLAMHE, this paper also realized the GHE and RLBHE algorithms as references. Wide varieties of standard images ranging from under exposed to over exposed low contrast to high contrast, dark background to bright background, are chosen to test the robustness and versatility of the RLAMHE method. Here we present an analysis of 3 images including Lena, House and Arplane U2. These results from Fig. 5-Fig. 7 show the superiority of RLAMHE in all the images in terms of contrast enhancement and control on over enhancement.

Figure 5.  (a) Original image of Lena.(b)Result of GHE.(c) Result of RLBHE.(d) Result of RLAMHE

Figure 6.  (a) Original image of House.(b)Result of GHE.(c) Result of RLBHE.(d) Result of RLAMHE

Figure 7.  (a) Original image of Airplane U2.(b)Result of GHE.(c) Result of RLBHE.(d) Result of RLAMHE

The test image Lena(Fig. 5) is a typical example that shows the white saturation effect, which often results from GHE methods. The result of RLAMHE shows more details and the contrast of the face and the shoulder is significantly improved than that of RLBHE. The RLAMHE method generates better enhancement around the hat than the RLBHE method did, while being natural looking.

The concrete results in terms of contrast enhancement can be clearly observed in Fig. 6. It is easy to see that the tower and the sky are over enhanced by GHE. The windows of the house and the tower are obscured by RLBHE. However, RLAMHE provides control on over enhancement leading to a good appearance. Observing the house and the tower in the image, we can perceive contrast enhancement. The front edge of the tower and the squares of the house shown in the blue rectangle can be seen clearly. By observing the processed images, it is noticeable that our proposed method is the only one among the other methods that can produce natural looking images.

It is obvious that the result of GHE in the test image Airplane U2(Fig. 7) changes the intensity values abruptly, and therefore, tends to produce level saturation effect. RLBHE loses many details of the wing and the brightness is not kept well. The result of RLAMHE shows that the mean brightness is preserved well and the detail of the wing is also well enhanced.

After visually observing some processed images, we can conclude that: (1)The RLAMHE method produces images with better quality than the other methods; (2)It also presents satisfactory brightness preserving and natural looking images.

Tab. 1 shows the AMBE for the three algorithms mentioned above. From Tab. 1, we can see that the resulting AMBE for RLAMHE of Lena is 0.414 6, which is much better than the resulting AMBE for RLBHE(0.629 5).

In this paper, the range limited adaptive multi-threshold histogram equalization(RLAMHE) algorithm with brightnes preserving is presented. We tested the proposed RLAMHE algorithm on an real time video processing system. The experimental results show that the proposed RLAMHE is able to achieve visually pleasant enhancement effects. The over-enhancement and level saturation artifacts are effectively avoided. Compared with many other GHE-based enhancement methods, images enhanced using the RLAMHE method show well enhanced contrast and little artifacts, while being natural looking. In addition, the RLAMHE method is computationally simple and suitable for processor-based implementation.