Multi-subsystem co-design refers to the simultaneous optimization of physical plant and controller of a system decomposed into multiple interconnected subsystems. In this paper, two decentralized (multi-level and bi-level) approaches are formulated to solve multi-subsystem co-design problems, which are based on the direct collocation and decomposition-based optimization methods. In the multi-level approach, the problem is decomposed into two bi-level optimization problems, one for the physical plant and the other for the control part. In the bi-level approach, the problem is decomposed into subsystem optimization subproblems, with each subproblem having the optimization model for physical plant and control parts together. In both cases, the entire time horizon is discretized to convert the continuous optimal control problem into a finite-dimensional nonlinear program. The optimality condition decomposition method is employed to solve the converted problem in a decentralized manner. Using the proposed approaches, it is possible to obtain an optimal solution for more generalized multi-subsystem co-design problems than was previously possible, including those with nonlinear dynamic constraints. The proposed approaches are applied to a numerical and engineering example. For both examples, the solutions obtained by the decentralized approaches are compared to a centralized (all-at-once) approach. Finally, a scalable version of the engineering example is solved to demonstrate that using a simulated parallelization with and without communication delays, the computational time of the proposed decentralized approaches can outperform a centralized approach as the size of the problem increases.