Recent reports that coffee can significantly inhibit cAMP phosphodiesterases (PDEs) in vitro, as well as in vivo, have described another beneficial effect of coffee consumption. However, the PDE-inhibiting substances remain mostly unknown. We chose activity-guided fractionation and an in vitro test system to identify the coffee components that are responsible for PDE inhibition. This approach indicated that a fraction of melanoidins reveals strong PDE-inhibiting potential (IC50 = 130 ± 42 μg/mL). These melanoidins were characterized as water-soluble, low-molecular weight melanoidins (<3 kDa) with a nitrogen content of 4.2% and a carbohydrate content lower than those of other melanoidins. Fractions containing known PDE inhibitors such as chlorogenic acids, alkylpyrazines, or trigonelline as well as N-caffeoyl-tryptophan and N-p-coumaroyl-tryptophan did not exert PDE-inhibiting activity. We also observed that the known PDE inhibitor caffeine does not contribute to the PDE-inhibiting effects of coffee.