Besides protection against specific microorganisms, vaccines can induce heterologous or non-specific effects (NSE). Epidemiological data suggest that vaccination with live-attenuated vaccines such as Bacillus Calmette-Guérin (BCG), measles vaccine, and oral polio vaccine results in increased overall childhood survival, and several of these observations have been confirmed in randomized trials. Immunological mechanisms mediating NSE include heterologous lymphocyte effects and induction of innate immune memory (trained immunity). Trained immunity induces long-term functional upregulation of innate immune cells through epigenetic and metabolic reprogramming. An overview of the epidemiological evidence of non-specific effects of vaccines and the latest insights regarding the biological mechanisms behind this phenomenon is presented, and future research priorities and potential implications are discussed.