Long-term overgrazing tends to cause soil phosphorus (P) deficiency in grasslands. The relationships between grazing-induced shifts in species composition and root traits associated with P acquisition involved in these shifts remain unknown. Species vary in their P-acquisition strategies, and we hypothesize that species that acquire P more efficiently are better adapted to overgrazing. We measured relative biomass, root physiological activities (e.g., rhizosheath acid phosphatase activity (Apase), and leaf manganese concentration ([Mn]) as a proxy for carboxylate concentrations in rhizosheath) and morphological traits (e.g., specific root length) of six common species in a field experiment conducted in a typical steppe of Inner Mongolia. There were two exclosure demonstration plots, i.e. exclosed since 1983 and 1996, and long-term free grazing without exclosure of sheep. Long-term overgrazing caused a reduction in bulk soil Olsen P concentration and increased community-weighted leaf nitrogen: P ratio by 27% and 37%, respectively, indicating more severe P limitation for steppe vegetation. Carex duriuscula exhibited an inherently greater specific root length, proportion of fine roots and rhizosheath Apase than other species did in both exclosure and grazing treatments. Cleistogenes squarrosa showed a greater leaf [Mn] induced by overgrazing. The increased dominance of C. duriuscula and C. squarrosa was positively correlated with finer roots, greater rhizosheath Apase or carboxylate release under long-term overgrazing. Species that had inefficient root traits for P acquisition (e.g., low specific root length and low leaf [Mn]), i.e. Stipa grandis, exhibited a decreased dominance in response to overgrazing. Dominance of species did not change under grazing which may be related to either relatively inefficient inherent morphological (i.e. in Artemisia frigida) or physiological traits (i.e. in Leymus chinensis and Agropyron michnoi) for P acquisition. Our study highlights the importance of acknowledging root traits involved in efficient P acquisition for theories on community succession induced by overgrazing.