Huperzia serrata has been used as a Chinese folk medicine for many years. It contains huperzine A, which has a protective effect against memory deficits in animal models; however, it is unclear if H. serrata extract exerts any effects in Alzheimer's disease (AD) models. We used H. serrata collected in Japan and determined its huperzine A content using HPLC. We determined its inhibitory effects on acetylcholinesterase (AChE) and butyrylcholinesterase (BuChE) activity. H. serrata extract (30 mg/kg/day) and donepezil (10 mg/kg/day) were orally administrated for 7 days. After repeated administration, we performed the Y-maze and passive avoidance tests. H. serrata extract contained 0.5% huperzine A; H. serrata extract inhibited AChE, but not BuChE. H. serrata extract ameliorated cognitive function in mice. These results indicate that Japanese H. serrata extract ameliorates cognitive function deficits by inhibiting AChE. Therefore, H. serrata extract may be valuable for the prevention or treatment of dementia in AD.