Increasing environmental awareness in the construction sector has been the cornerstone in the research on identifying sustainable materials and improving their environmental performance. Stone is a natural material with many applications in construction, from the production of tiles and curbs to its use as massive stone. This paper presents a study on the environmental impact of the extraction and processing of stone in relation to the final product used in construction. The results of the life cycle assessment presented here show that, in contrast to other construction materials such as cement and steel, the determining factor for the environmental profile of stone is not the amount of material included in the final product but the amount subtracted from the original block during the different forms of processing and refinement. This result introduces a new approach to the environmental evaluation of natural materials and could induce a different way of designing buildings which promotes massive design for natural materials rather than a thin cover on external façades.