A material generally fails with elongation or with cyclic loading. The energy of the system increases during loading resulting in breaking of bonds and creation of crack surfaces. However, an interesting phenomenon of failure at constant deformation was observed in cross-linked polymers such as rubber and polyurethane. These viscoelastic materials show relaxation behaviour at a constant deformation state and the global stress level decreases with energy dissipation. However, at the microscopic level a fraction of the polymer chains may break which in turn increases the load on the remaining chains. With the relaxation of some of these chains, the microscopic load carrying capacity can decrease and a macroscopic crack initiation and propagation can occur. In this work a numerical investigation on this failure phenomenon is presented. A viscoelastic material model is coupled with a phase field model to simulate the crack propagation during relaxation. The phase field evolves with the changing mechanical energy during the relaxation of the material. A mobility constant is used to control the evolution rate. The simulations are carried out for polyurethane (PU9010) and results for failure during tensile and relaxation tests are discussed. A method to predict the time to failure during the relaxation test is proposed using the concept of a failure envelope. A study on the various parameters of the model that can influence the time to failure is presented. Thus an explanation for this failure phenomenon is presented with the help of the numerical results and a physical interpretation is proposed.