After realizing a successful automatic SOM based color segmentation of static images, we tackle the subject of video sequences. We developed an algorithm that tracks color segments (or objects) through out a video sequence and indicates at each moment their position in the scene. We track color objects ever since their appearance on the scene, during their existence and until their disappearance from the scene. We lay out some new definitions and laws in the tracking domain that facilitate the matching process and the validation of that matching. We do the matching using 5 spatial descriptors of the color objects...