The behavior of Cross-linked Polymers in finite deformations is often characterized by nonlinear behaviour. In this paper, we propose to embed an artificial neural network (ANN) within a micro-mechanical platform and thus to enforce certain physical restrictions of an amorphous network such as directional dependency and history-dependency of the constitutive behavior of rubber-like materials during loading and unloading. Accordingly, a strain energy density function is assumed for a set of chains in each direction based on ANN and trained with experimental data set. Summation of the energies provided by ANNs in different directions can determine the strain energy density function of the matrix. Stress-strain relation is derived from strain energy density function. Polyconvexity is enforced to assure minimized potential energy, a global solution for an optimization problem, and thermodynamic consistency that show the model cannot generate energy. The model is validated against multiple sets of experimental data, e.g. uniaxial, shear, and biaxial deformation available in the literature. This model captures not only the loading and unloading paths but also the inelastic response of these materials, such as the Mullins effect and permanent set. The model can be generalized to other materials and other inelastic effects as well.