Considering the visual features of eyes, À trous contrast pyramid transform combining nonsubsampled directional filter bank is proposed. The source image is decomposed into low-frequency image and high-frequency images by à trous contrast pyramid transform. Then apply directional filter bank to each high-frequency. The different fusion rule is adopted in low-frequency image and high-frequents subbands. Finally the fused image is obtained through inverse transform. Experimental results show that the proposed method is valid to fusion of infrared and visible images, and has better subjective visual effect.