The topological design of structures to avoid vibration resonance for a certain external excitation frequency is often desired. This paper considers the topology optimization of freely vibrating bi-material structures with fixed/varying attached mass positions, targeting at maximizing the frequency band gap centering at a specified frequency. A band gap measure index is proposed to measure the size of the band gap with a specified center frequency. Aiming at maximizing this measure index, the topology optimization problem is formulated on the basis of the material-field series-expansion (MFSE) method, which greatly reduces the number of design variables and at the same time keeps the capability to describe relatively complex structural topologies with clear boundaries. As the considered optimization problem is highly non-linear and may yield multiple local minima, a sequential Kriging-based optimization solution strategy is employed to effectively solve the optimization problem. This solution strategy exhibits a relatively strong global search capability and requires no sensitivity information. With the present topology optimization model and the gradient-free algorithm, relative large band gaps with specified center frequencies have been obtained for two-dimensional (2D) beams and three-dimensional (3D) plates, without specifying the frequency orders between which the desired band gap occurs in prior.