Lithocarpus, with >320 species, is the second largest genus of Fagaceae. However, the lack of a reference genome limits the molecular biology and functional study of Lithocarpus species. Here, we report the chromosome-scale genome assembly of sweet tea (Lithocarpus polystachyus Rehder), the first Lithocarpus species to be sequenced to date. Sweet tea has a 952-Mb genome, with a 21.4-Mb contig N50 value and 98.6% complete BUSCO score. In addition, the per-base consensus accuracy and completeness of the genome were estimated at 60.6 and 81.4, respectively. Genome annotation predicted 37,396 protein-coding genes, with repetitive sequences accounting for 64.2% of the genome. The genome did not undergo whole-genome duplication after the gamma (γ) hexaploidy event. Phylogenetic analysis showed that sweet tea diverged from the genus Quercus approximately at 59 million years ago. The high-quality genome assembly and gene annotation resources enrich the genomics of sweet tea, and will facilitate functional genomic studies in sweet tea and other Fagaceae species.