Mechanism with few degree-of-freedom is an economical solution for rehabilitation robot because it does not need complicated structure and control system. To provide motion data to design the robot with few degree-of-freedom, a user motion tracking method on the mobile platform is proposed. Using the data from monocular vision system on the smartphone and the KCF algorithm to track the knee and hip joints motion. To increase the stability and the precision, the maker with texture is designed to carry out the tracking. Furthermore, the transformation from the image coordinate to the world coordinate is implemented by the cross ratio invariant in the controlled tracking condition with fixed dimension makers. By this mean, the motion path in image coordinate is converted to that in the world coordinate. With the low limb motion path that provides the input of the Fourier descriptor, the mechanism parameters of the robot with few degree-of-freedom are calculated, and the 3D models of the mechanism are constructed. Finally, the prototype of the rehabilitation robot is established which can assist the users to stand from sitting posture, or work as a wheelchair with obstacle avoidance function by infrared and ultrasonic sensors.