ActiDote –activity as an antidote– is a system for manual wheelchair users that takes advantage of wireless sensors to recognize activities of various intensity levels in order to allow self-tracking of the physical activity. In this paper, we describe both the hardware setup and the software pipeline that enable our system to operate. Laboratory tests using multi-modal fusion and machine learning reveal promising results on classifying activity levels and assessing energy expenditure during wheelchair propulsion on ramps of dierent slopes and speeds. Our results indicate that it is possible to implement a system that uses the accelerometer of a smartphone as the only sensor in the wheelchair, i.e., by attaching it to the wheelchair frame. Additionally, the user might wear a smartwatch equipped with an accelerometer to enrich the system and enhance its performance.