In order to decrease initial alignment complexity, increase alignment precision, and minimize time consumption of alignment process, a new method of the EMD filtering is adopted for SINS initial alignment. First, based on the algorithm of EMD sift process, the IMU signal can be decomposed into a cluster of IMFs. According to the criterion of CMSE minimization, the IMU signals can be reconstructed utilizing the IMFs cluster. Then IMU noise of original signal and filtering signal can be modeled by ARMA algorithm, respectively. Furthermore, the coarse initial alignment process is demonstrated by the physics characteristic of the earth, and the results are compared by the data non-filtering and filtering of the same IMU. Finally, the fine initial alignment of SINS is completed using non-filtering data, filtering data of IMU and SINS error model. Experimental results indicate that the precision of azimuth angle coarse alignment is 1.3°, fine alignment is 0.87 mrad, and Kalman filter converge time is 200 s.