We show that the information gained in spectroscopic experiments regarding the number and distribution of atomic environments can be used as a valuable constraint in the refinement of the atomic-scale structures of nanostructured or amorphous materials from pair distribution function (PDF) data. We illustrate the effectiveness of this approach for three paradigmatic disordered systems: molecular C60, a-Si, and a-SiO2. Much improved atomistic models are attained in each case without any a priori assumptions regarding coordination number or local geometry. We propose that this approach may form the basis for a generalized methodology for structure "solution" from PDF data applicable to network, nanostructured and molecular systems alike.