In this work, we propose a novel decentralized iterative learning cooperative impedance control (ILCIC) framework to cooperatively control the impedance of a robot manipulator team that operates in an iterative manner. Using a novel notion of neighbourhood impedance error, a formation-based architecture based on an undirected communication graph is proposed so that all robots in the team can achieve the desired impedance, even though some robot manipulators may not know the desired angle profiles and their relative configuration with respect to these desired angles. Furthermore, these desired angle profiles and the desired impedance model can be iteration dependent, as the robot manipulator team may need to accomplish different task objectives. Alignment initial conditions are considered, relaxing the identical initial condition (i.i.c.) often considered in the iterative learning control literature which can be restrictive in robotic applications. Through rigourous analysis, it is shown that the impedance error for each robot manipulator converges to zero over the iteration domain, in the sense of -norm. In the end a simulation example on two 2DOF robot manipulators following a leader robotis presented to demonstrate the efficacy of the proposed framework.