The immune microenvironment in hepatocellular carcinoma (HCC), especially T-cell infiltration, plays a key role in the prognosis and drug sensitivity of HCC. Our study aimed to analyze genes related to non-regulatory CD4⁺ and CD8⁺ T cell in HCC. Data of HCC samples were downloaded from The Cancer Genome Atlas (TCGA) database. According to stromal and immune score retrieved by Estimation of Stromal and Immune cells in Malignant Tumor tissues using Expression data (ESTIMATE) algorithm, differentiated expressed genes (DEGs) between high and low stromal/immune scoring groups were collected. Using Cibersort algorithm, abundance of immune cells was calculated and genes related with CD4⁺ and CD8⁺ T cells were selected. Protein–protein interaction (PPI) networks and networks of microRNA (miRNA)–target gene interactions were illustrated, in which CD4⁺ and CD8⁺ T cell-related core genes were selected. Finally, Cox regression test and Kaplan–Meier (K–M) survival analysis were conducted. Totally, 1579 DEGs were identified, where 103 genes and 407 genes related with CD4⁺ and CD8⁺ T cell were selected, respectively. Each of 30 core genes related to CD4⁺ T cells and CD8⁺ T cells were selected by PPI network. Four genes each related with the two types of T cells had a significant impact on prognosis of HCC patients. Amongst, KLRB1 and IL18RAP were final two genes related to both two kinds of T cells and associated with overall survival of the HCC patients.