Life cycle assessment (LCA) is widely used to evaluate the environmental impacts of buildings, but due to uncertainties, the final results can be unreliable. To increase the reliability of LCA results, this study identifies the building materials that have the largest relative contribution to buildings' impacts and uncertainties. To do so, the impacts of 15 single-family houses and 15 multi-family building projects situated in France are evaluated. Only the uncertainties related to input parameters for building materials are considered (service life, characterization factors and quantity). The results obtained in this study show that LCA will still be able to distinguish significantly between two projects if their difference is higher than approximately 20%. Furthermore, the impacts of the buildings' exploitation phase do not show any correlations with the impacts related to the construction materials. The exploitation phase dominates the non-renewable energy consumption while waste impacts are most influenced by building materials. The contribution to global warming potential is shared between both phases. Finally, reinforced concrete was identified as the largest contributor to the environmental impact of both building types. In contrast, insulation materials and non-structural wood were the largest contributors to the uncertainties of the final results for single-family houses and multi-family buildings, respectively.