Large amounts of metal(loid)s in abandoned non-ferrous metal smelting site are released to the surrounding environment, causing serious harm to the environment. A weighted comprehensive pollution risk score (CRS), combining two typical metal(loid)s pollution and risk index evaluation methods (speciation and total content indices), and two microbial activity evaluation methods (microcalorimetry and enzyme activities), is proposed to evaluate the pollution degree and environmental risk of metal(loid)s. In the surface soil (Site SS) of contaminated site, the CRS identified seven metal(loid)s (CRS_As (20) > CRS_Cd (19) > CRS_Sb (18) > CRS_Zn (13) = CRS_Cu (13) > CRS_Ni (10) > CRS_Pb (8)) with higher risks, and six (CRS_Cr (7) > CRS_Mn (4) = CRS_V (4) > CRS_Al (3) = CRS_Mg (3) > CRS_Ti (2)) with lower risks. Different areas were sorted by the potential ecological risk index (RI) and CRS index from large to small: the deep soil (Site SD) > Site SS > the surrounding wasteland (Site Y), indicating that smelting site was heavily polluted by metal(loid)s. In Site SS, microbial activities were low confirming the inhibition effect of metal(loid)s on soil microbial community activities. Microbial activity characteristics were correlated to metal(loid)s concentration and mobility, indicating that microbial activity represents a potential indicator of pollution. Particularly, FDA activity and TMA are sensitive indicators to estimate soil metal(loid)s pollution level. Our study provides a comprehensive assessment of the metal(loid)s pollution risk, including microbial activity, in non-ferrous metal smelting sites, useful information for the management and remediation of metal(loid)s polluted sites.