This paper presents a novel hydrostatic actuator which is named as Linear-Driven Electro-Hydrostatic Actuator (LEHA). In an LEHA, the actuator is driven by a novel collaborative rectification pump (CRP), which incorporates two miniature cylinders and two spool valves. Specifically, the CRP is driven by two linear oscillating motors, which are designed and optimized to generate reciprocating motion at high frequency with adequate stroke. CRP offers a highly novel linear fluid pump with flexibility in bi-directionally driving. In this paper, schematic of LEHA is firstly presented and its kinematic flow rate equation is derived. Then the design of CRP, linear oscillating motor as well as the whole LEHA prototype is introduced. Performance of the LEHA is demonstrated through a series of experiments and simulation, and analysis of the results is also included.