Ductile failure in structural metals is often associated with nucleation, growth, and coalescence of microscopic voids. In 6000-series aluminum alloys, void nucleation typically occurs from the largest intermetallic particles -- the constituent particles. These have various sizes, ranging from less than one to several micrometers. A high volume fraction of constituent particles is known to adversely affect ductility and toughness. The effect of particle size distribution on the failure properties is not fully understood. Numerical simulations based on Representative Volume Element (RVE) models with heterogeneous particle distributions offer great potential for such studies. However, statistically representative models require many particles to be analyzed, which renders models with discretely resolved particles intractable. In this work, we propose a computationally efficient numerical model to evaluate the effect of particle size distribution on the failure of aluminum alloys. To this end, we assume that voids are nucleated instantly upon loading and impose proportional loading to RVE models where the void size and volume fraction are statistically distributed within the mesh. The RVE is modeled with N x N x N elements of equal size where each element comprises a single void. The void spacing is thus uniform throughout the model. Size-dependent void growth is accounted for using the porous plasticity model proposed by Monchiet and Bonnet. The results show that the heterogeneity introduced by the void size distribution has a large influence on the failure strain of the RVE. One of the main outcomes of this numerical study is that an abrupt failure mode is obtained without introducing a coalescence model at the material point level. We believe the proposed numerical framework has great potential for future studies of ductile fracture in structural metals.