Cow’s milk is considered a complete food, providing high-quality protein and essential micronutrients, including vitamins and minerals. For medical reasons or as a lifestyle choice, consumers are replacing cow’s milk with plant-based milk alternatives (PBMA); some perceive them as healthier alternatives to cow’s milk due to their low saturated fatty acid content and no cholesterol content. However, the nutritional composition of PBMA is quite variable between different types and even within, which makes a comparison with cow’s milk a complex issue. Furthermore, the consumption of PBMA has been associated with the development of some diseases in infants and children. Meanwhile, the consumption of cow’s milk in human health is a controversial issue since it has been associated with a favorable effect in some diseases (such as obesity, type 2 diabetes, and Alzheimer’s) and a negative effect in others (such as prostate cancer risk and Parkinson’s disease); while in some diseases, there is no consensus in the cow’s milk consumption effect. The aim of this review is to make a nutritional comparison of cow’s milk with PBMA and to clarify the potential health issues related to their consumption.