A neural network-based surrogate model for inelastic solid materials simulations is presented. The network architecture incorporates the elastoplasticity equations and has been proven to be exact in one-dimensional elastoplasticity with hardening. This strategy provides an alternative to the complex task of formulating constitutive equations for new materials due to the capability of learning directly from data. The network is constructed based on the concept of internal states, hence, the data required utilises only observable variables (strains and stresses). Moreover, instead of opting for a simple static feedforward network, a recurrent neural network (RNN) is chosen to train on data sequences since inelastic materials depend on the current stress and deformation history. A proposed gradient-free strategy is used to train the network, which confirms the exact rendering of 1D elastoplastic models, while multiaxial cases are approximated accurately. The trained network is implemented into a finite element code on a Gauss point level replacing a traditional library of algorithmic constitutive models. The updated stress is obtained from the network’s output while the input includes the current strains and internal variables. A compact expression to compute the elastoplastic modulus is presented, which depends on the network architecture selected. Finally, a number of numerical experiments is presented.