A time domain structural damage detection method based on hierarchical Bayesian framework is proposed. Due to local stiffness reductions, the responses of damaged structures vary from those in undamaged status under the same external excitation. In this paper, the responses of damaged structures are assumed as the result of a summation of known external forces and unknown virtual forces exerted on corresponding undamaged structures. The damages can thus be detected, located, and quantified by the identification of associated virtual forces. A hierarchical Bayesian formulation considering all undetermined damage-related variables is adopted for the identification of virtual forces. The reasonable values of the variables and their uncertainties are depicted by their posterior distributions, sampled by Markov chain Monte Carlo method. Compared with traditional Bayesian formulations, manual choice of prior parameters is avoided and less prior information is required. The proposed virtual force indicator provides a more intuitive perspective for damage detection tasks and is potentially more operable in engineering practice. These advantages are illustrated by simulation of a cantilever beam under various damage conditions.