It is well established that the formation of episodic memories requires multiple hippocampal mechanisms operating on different time scales. Early mechanisms of memory formation (synaptic consolidation) have been extensively characterized. However, delayed mechanisms, which maintain hippocampal activity as memories stabilize in cortical circuits, are not well understood. Here we demonstrate that contrary to the transient expression of early- and delayed-response genes, the expression of cytoskeleton- and extracellular matrix-associated genes remains dynamic even at remote time points. The most profound expression changes clustered around primary cilium-associated and collagen genes. These genes most likely contribute to memory by stabilizing perineuronal nets in the dorsohippocampal CA1 subfield, as revealed by targeted disruptions of the primary cilium or perineuronal nets. The findings show that nonsynaptic, primary cilium-mediated mechanisms are required for the persistence of context memory.