Advances in machine learning are offering accelerated material design and analysis. Recent works on the Deep Material Network (DMN) have predicted microstructures' constitutive responses in linear and nonlinear regions with similar accuracy as fast Fourier transform (FFT) analysis by using a network trained solely on linear elastic data. Here, we walk over the issue of the inevitable training and calibration errors that happen when we use randomly initialized network parameters. By visualizing the DMN training process with the newly introduced DMN RVE, we improve the explainability of the network training process and tackle those errors. Learning from the DMN RVE, we perform recursive training to construct and initialize deeper networks. The results using the recursive training have shown an improvement in the accuracy and calibration performances both in the offline and online modes, along with intuitively and physically interpreted network parameter convergence and optimization. Sandia National Laboratories is a multi-mission laboratory managed and operated by National Technology and Engineering Solutions of Sandia, LLC., a wholly owned subsidiary of Honeywell International, Inc., for the U.S. Department of Energy National Nuclear Security Administration under contract DE-NA0003525.