Hepatocellular carcinoma (HCC) represents a paradigm of the relation between tumor microenvironment (TME) and tumor development. Here, we generate a single-cell atlas of the multicellular ecosystem of HCC from four tissue sites. We show the enrichment of central memory T cells (T_CM) in the early tertiary lymphoid structures (E-TLSs) in HCC and assess the relationships between chronic HBV/HCV infection and T cell infiltration and exhaustion. We find the MMP9⁺ macrophages to be terminally differentiated tumor-associated macrophages (TAMs) and PPARγ to be the pivotal transcription factor driving their differentiation. We also characterize the heterogeneous subpopulations of malignant hepatocytes and their multifaceted functions in shaping the immune microenvironment of HCC. Finally, we identify seven microenvironment-based subtypes that can predict prognosis of HCC patients. Collectively, this large-scale atlas deepens our understanding of the HCC microenvironment, which might facilitate the development of new immune therapy strategies for this malignancy.