The characteristics of the primary tumor blood vessels and the tumor microenvironment drive the enhanced permeability and retention (EPR) effect, which confers an advantage towards enhanced delivery of anti-cancer nanomedicine and has shown beneficial effects in preclinical models. Increased vascular permeability is a landmark feature of the tumor vessels and an important driver of the EPR. The main focus of this review is the endothelial regulation of vascular permeability. We discuss current challenges of targeting vascular permeability towards clinical translation and summarize the structural components and mechanisms of endothelial permeability, the principal mediators and signaling players, the targeted approaches that have been used and their outcomes to date. We also critically discuss the effects of the tumor-infiltrating immune cells, their interplay with the tumor vessels and the impact of immune responses on nanomedicine delivery, the impact of anti-angiogenic and tumor-stroma targeting approaches, and desirable nanoparticle design approaches for greater translational benefit.