The stability of the collocated position control of a mass is studied when a pendulum is attached to it. The simplest proportional-derivative (PD) controller is applied, but the relevant constant time delay is taken into account. The linearized governing equations of the system are investigated. Stability charts are constructed for different pendulum parameters. Closed form expression is derived for the critical time delay; for delay values larger than the critical one, the PD controller cannot stabilize the desired position of the mass. The frequencies of the self-excited vibrations at the stability boundaries have essential role in identifying the types of loss of stability.