The advent of silica-based low-cost standard single-mode fibers revolutionized the whole communication industry. The deployment of optical fibers in the networks induces a paradigm shift in the communication technologies used for long-haul information transfer. However, the communication using the optical fibers is affected by several linear and nonlinear effects. The most common linear effects are attenuation and chromatic dispersion, whereas the dominant nonlinear effect is the Kerr effect. The Kerr effect induces a power-dependent nonlinear distortion for the signal propagating in the optical fiber. The detrimental effects of the Kerr nonlinearity limit the capacity of long-haul optical communication systems. Fiber Kerr nonlinearity compensation using digital signal processing (DSP) techniques has been well investigated over several years. In this paper, we provide a comprehensive tutorial, including the fundamental mathematical analysis, on the characteristics of the optical fiber channel, the origin of the Kerr nonlinearity effect, the theory of the pulse propagation in the optical fiber, and the numerical and analytical tools for solving the pulse propagation equation. In addition, we provide a concise review of various DSP techniques for fiber nonlinearity compensation, such as digital back-propagation, Volterra series-based nonlinearity equalization, perturbation theory-based nonlinearity compensation, and phase conjugation. We also carry out numerical simulation and the complexity evaluation of the selected nonlinearity compensation techniques.