The herb Orostachys japonicus has been traditionally used to treat chronic diseases, such as hepatitis, hemorrhoids, and cancer, in Asia. In this study, we investigated the effect of Orostachys japonicus water extract (OJWE) on the receptor activator of nuclear factor-κB ligand (RANKL)-induced osteoclast differentiation and bone loss. We found that OJWE inhibited RANKL-induced osteoclast differentiation in a dose-dependent manner without affecting bone resorption in bone marrow-derived macrophage cells. Interestingly, OJWE significantly reduced serum levels of C-terminal telopeptide of type 1 collagen and tartrate-resistant acid phosphatase (TRAP) 5b, markers of bone resorption and osteoclast number, respectively, in an animal model of bone loss. Furthermore, OJWE suppressed the RANKL-induced up-regulation of nuclear factor of activated T cells cytoplasmic 1 (NFATc1) expression, and activation of the p38 signaling pathway, but prevented the RANKL-mediated down-regulation of interferon regulatory factor-8 (IRF-8), which is known to be an anti-osteoclastogenic factor that represses NFATc1 expression. We also identified gallic acid and quercetin-3-O-β-D-glucoside as the OJWE components that inhibit RANKL-induced osteoclast differentiation. These results suggest that OJWE inhibits osteoclast differentiation by inhibiting RANKL-induced NFATc1 expression, which prevents osteoclast differentiation and bone loss. The present study elucidated a mechanism of action underlying the inhibitory effect of OJWE on osteoclast differentiation. Our findings suggest that O. japonicus has therapeutic potential for use in the treatment of bone diseases.