The Balia Nala landslide in Nainital has become a widely concerning issue due to its frequent reactivation and rampant construction over its slope. It has underscored the importance of characterizing the Balia Nala landslide and quantification of the physical vulnerability of the buildings. The current study incorporates the indicator-based approach with the aid of Geographic Information System (GIS) to assess the site-specific physical vulnerability of the buildings exposed to debris flow landslide. The vulnerability of the 63 household samples was computed using 15 different indicators. Empirical relationships were employed for estimating the intensity of the landslide, resistance of the buildings, physical vulnerability (PV), and site-specific vulnerability (SSV). A very high landslide intensity of value ‘1’ has been evaluated in the area. 89% and 11% of the surveyed buildings came out to be moderately (0.5 < R ≤ 0.9) and low resistant (R ≤ 0.5) respectively. The final study results show that 44% of the surveyed buildings fall under class III (0.9 < SSV ≤ 1; very severe to complete damage), and about 56% fall under class II (0.5 < SSV ≤ 0.9; moderate to severe damage). None of the buildings surveyed falls under class I (SSV ≤ 0.5), which would be subjected to low damage. This research opens a new avenue to frame or revise proper guidelines on building constructions and design, specifically concerning landslides in the hilly terrain. It would also help the State Disaster Management Authority and Irrigation Department, Nainital, take efficient disaster risk reduction measures to reduce the landslide intensity, increase buildings’ resistance, and prioritize relocation of the vulnerable households.