Despite rapid advances in our knowledge of the cellular heterogeneity and molecular regulation of the mammary gland, how these relate to 3D cellular organization remains unclear. In addition to hormonal regulation, mammary gland development and function is directed by para- and juxtacrine signaling among diverse cell-types, particularly the immune and mesenchymal populations. Precise mapping of the cellular landscape of the breast will help to decipher this complex coordination. Imaging of thin tissue sections has provided foundational information about cell positioning in the mammary gland and now technological advances in tissue clearing and subcellular-resolution 3D imaging are painting a more complete picture. In particular, confocal, light-sheet and multiphoton microscopy applied to intact tissue can fully capture cell morphology, position and interactions, and have the power to identify spatially rare events. This review will summarize our current understanding of mammary gland cellular organization as revealed by microscopy. We focus on the mouse mammary gland and cover a broad range of immune and stromal cell types at major developmental stages and give insights into important tissue niches and cellular interactions.