An improved fast Bayer interpolation algorithm based on the Field Programming Gate Array(FPGA) was presented to solve the problems of a traditional bilinear Bayer color image restoration on edge blurring and lumps in images. With the algorithm, a gradient algorithm was used to restore the green channel and the color groups corresponding to pixels to restore the blue channel for a image. Experimental results indicate that the algorithm can offer a better visual effect of the images, and the Signal to Noise Ratios(SNRs) of RGB channels obtained from the proposed algorithm are all higher 5 dB than those of obtained from the bilinear algorithm. Moreover, the time-consuming of the algorithm is less than that of the bilinear algorithm, and a 512× 512 full color image can be processed in 9.3 ms. These results can meet the requirements of real time image processing and have higher application values.