A multi-scale defect dynamics model (DDM) has been developed by coupling dislocation dynamics (DD) and finite element modeling. In this concurrently coupled model, the DD framework tracks the dynamic evolution of dislocation microstructure, while the stress fields with the given plastic strain (or transformation strain) field from DD are calculated in the finite element model. The proposed model has validated to handle material rotation in finite deformation and accurately calculated stress field from dislocations and interaction force between dislocations. In addition, it could provide the capability to account for complex loading and boundary conditions as well as multi-physical phenomena. In this talk, we will present a dislocation-based damage model using the developed multiscale defect dynamics model. In contrast to continuum based phenomenological models, our model could provide a unique opportunity to investigate detailed dynamic evolution of microstructure. Especially, the collective motion of dislocations could be correlated with the macroscopic damage behavior of the materials in various loading conditions and extreme environments. Additionally, micromechanical characterization results of single crystalline show good agreement with observations in terms of defect microstructure.