Consolidation of colloidal dispersions under external load is a complex process involving inter-particle interactions, thermal forces and hydrodynamics. Despite its importance in diverse industrial applications, past studies involving experiments, scaling approaches and simulations are yet to provide a comprehensive understanding of how the microstructure determines the mechanical response in three dimensional colloidal gels. Here, we develop a model that accounts for the microstructural details and predicts the mechanical response under slow, uniaxial compression of a strongly aggregated three dimensional colloidal gel. The particle network assumes a fractal structure that is independent of the strength of inter-particle interactions. While the yield strain changes negligibly during the entire process, the yield stress increases by several orders of magnitude. The predicted yield stress and strain are in close agreement with those observed in simulations and experiments with diverse colloidal systems, suggesting a universality in the consolidation process.