Multiple sclerosis (MS) is a demyelinating disease of the CNS. Epstein-Barr virus (EBV) contributes to the MS pathogenesis because high levels of EBV EBNA_386–405-specific antibodies cross react with the CNS-derived GlialCAM_370–389. However, it is unclear why only some individuals with such high autoreactive antibody titers develop MS. Here, we show that autoreactive cells are eliminated by distinct immune responses, which are determined by genetic variations of the host, as well as of the infecting EBV and human cytomegalovirus (HCMV). We demonstrate that potent cytotoxic NKG2C⁺ and NKG2D⁺ natural killer (NK) cells and distinct EBV-specific T cell responses kill autoreactive GlialCAM_370–389-specific cells. Furthermore, immune evasion of these autoreactive cells was induced by EBV-variant-specific upregulation of the immunomodulatory HLA-E. These defined virus and host genetic pre-dispositions are associated with an up to 260-fold increased risk of MS. Our findings thus allow the early identification of patients at risk for MS and suggest additional therapeutic options against MS.