As the largest substructures in the nucleus, nucleoli are the sites of ribosome biogenesis. Increasing evidence indicates that nucleoli play a key role in the organization of 3D genome architecture, but systematic studies of nucleolus-associated chromatin interactions are lacking. Here, we developed a nucleolus Hi-C (nHi-C) experimental technique to enrich nucleolus-associated chromatin interactions. Using the nHi-C experiment, we identify 264 high-confidence nucleolus-associated domains (hNADs) that form strong heterochromatin interactions associated with the nucleolus and consist of 24% of the whole genome in HeLa cells. Based on the global hNAD inter-chromosomal interactions, we find five nucleolar organizer region (NOR)-bearing chromosomes formed into two clusters that show different interaction patterns, which is concordant with their epigenetic states and gene expression levels. hNADs can be divided into three groups that display distinct cis/trans interaction signals, interaction frequencies associated with nucleoli, distance from the centromeres, and overlap percentage with lamina-associated domains (LADs). Nucleolus disassembly caused by Actinomycin D (ActD) significantly decreases the strength of hNADs and affects compartment/TAD strength genome-wide. In summary, our results provide a global view of heterochromatin interactions organized around nucleoli and demonstrate that nucleoli act as an inactive inter-chromosomal hub to shape both compartments and TADs.