It is well known that the biological composites have superior mechanical properties due to their exquisite multilevel structural hierarchy. However, the underlying mechanisms of the roles of this hierarchical design in the toughness of the biocomposites remain elusive. In this paper, the deformation and fracture mechanism of multilevel hierarchical structures are explored by molecular dynamics simulations. The effects of the multilevel design on fracture toughness, nonlinear deformation of soft matrix, and the crack path pattern were quantitatively analyzed. We showed that the toughness of composites is closely associated with the pattern of the crack path and the nonlinear deformation of the matrix. Additionally, the structure with a higher level of hierarchy exhibit higher toughness, which is less sensitive to the geometrical change of inclusions, such as the aspect ratio and the staggered ratio. This work provides more theoretical evidence of the toughening mechanism of the multilevel hierarchy in fracture toughness of biological materials via new methods of analyzing fracture of multilevel structures and provides guidelines for the design of high-performance engineering materials.