Rule extraction from neural networks represents a difficult research problem, which is NP-hard. In this work we show how a special Multi Layer Perceptron architecture denoted as DIMLP can be used to extract rules from ensembles of DIMLPs and Quantized Support Vector Machines (QSVMs). The key idea for rule extraction is that the locations of discriminative hyperplanes are known, precisely. Based on ten repetitions of stratified 10-fold cross validation trials and with the use of default learning parameters we generated symbolic rules from five datasets. The obtained results compared favorably with respect to another state of the art technique applied to Support Vector Machines.