Fibers and parenchyma cells are two cellular units of bamboo, which possess different structures and properties from each other. However, there were rare reports on the comparison of morphology, structure, and properties between bamboo fibers and parenchyma cells that were processed by the same chemical treatment. Here, we mechanically separated bamboo fibers and parenchyma cells from the same bamboo culm, and subsequently treated them with 5% alkali solution under varying treatment time (1 h, 2 h, 4 h, 6 h, and 8 h) and temperatures (23 °C, 40 °C, 60 °C, and 80 °C), and investigated the effects of treatment time and temperature on their morphology, structure, and properties. The microstructure, chemical composition, crystallinity index and thermal stability of alkali-treated fibers and parenchyma cells were examined by scanning electron microscopy, Fourier-transform infrared spectroscopy, X-ray diffraction chemical composition analysis, and thermogravimetric analysis. The results revealed that the microstructure of bamboo fibers was affected by alkali treatment time and temperatures more pronouncedly than that of parenchyma cells. Regardless of fibers or parenchyma cells, alkali treatment temperature affected chemical compositions and crystallinity more pronouncedly than alkali treatment time. The increase in alkali treatment temperature was more likely to remove hemicellulose and lignin than extending alkali treatment time, which had a more pronounced impact on CrI. The CrI reached the maximum value at 80 °C, with a 8.3% and 20.0% increase in fibers and parenchyma cells, respectively. The thermal stability of fibers and parenchyma cells showed opposite changes after different alkali treatment time and temperatures.