Ni-rich layered oxides, like LiNi0.8Co0.1Mn0.1O2 (NCM811), have been widely investigated as cathodes due to their high energy density. However, gradual structural transformation during cycling can lead to capacity degradation and potential decay of cathode materials. Herein, we doped high-valence transition metal (TM) ions (V5+, Nb5+, and Zr4+) at the Ni site of NCM811 by first principles simulations and explored the mechanism of doping TMs in NCMs for enhancing the electrochemical performance. Analysis of the calculations shows that doping V, Nb and Zr has an efficient influence on alleviating the Ni oxidation, reducing the loss of oxygen, and facilitating Li+ migration. Moreover, V doping can further suppress the lattice distortion due to the radius of V5+ being close to the radius of Mn4+. In particular, compared with the barrier of the pristine NCM in Li divacancy, the barrier of V-doped NCM reaches the lowest. In conclusion, V is the most favorable dopant for NCM811 to improve the electrochemical properties and achieve both high capacity and cycling stability.