Pancreatic cancer is typically detected at an advanced stage, and is refractory to most forms of treatment, contributing to poor survival outcomes. The incidence of pancreatic cancer is gradually increasing, linked to an aging population and increasing rates of obesity and pancreatitis, which are risk factors for this cancer. Sources of risk include adipokine signaling from fat cells throughout the body, elevated levels of intrapancreatic intrapancreatic adipocytes (IPAs), inflammatory signals arising from pancreas-infiltrating immune cells and a fibrotic environment induced by recurring cycles of pancreatic obstruction and acinar cell lysis. Once cancers become established, reorganization of pancreatic tissue typically excludes IPAs from the tumor microenvironment, which instead consists of cancer cells embedded in a specialized microenvironment derived from cancer-associated fibroblasts (CAFs). While cancer cell interactions with CAFs and immune cells have been the topic of much investigation, mechanistic studies of the source and function of IPAs in the pre-cancerous niche are much less developed. Intriguingly, an extensive review of studies addressing the accumulation and activity of IPAs in the pancreas reveals that unexpectedly diverse group of factors cause replacement of acinar tissue with IPAs, particularly in the mouse models that are essential tools for research into pancreatic cancer. Genes implicated in regulation of IPA accumulation include KRAS, MYC, TGF-β, periostin, HNF1, and regulators of ductal ciliation and ER stress, among others. These findings emphasize the importance of studying pancreas-damaging factors in the pre-cancerous environment, and have significant implications for the interpretation of data from mouse models for pancreatic cancer.