Fault-tolerant streaming computation with BlockMon

As the amount of data being exchanged over the network increases, algorithms originally implemented for running on a single machine have been re-designed to work in a distributed manner, with a processing platform that splits tasks among machines and cores. Brand new frameworks have emerged for the analysis of unbound streams of data, aiming at processing data and retrieving information nearly real-time by using clusters of machines. Node failure and recovery are crucial issues related to distributed systems, especially when using commodity hardware and when continuously processing data coming real- time into the system. In this paper we present the performance of the distributed stream-processing platform Blockmon, with the novel fault-tolerant mechanism that we implement on top, and compare it against Spark, the state-of-the art in terms of fault-tolerant stream-processing platform. Our experimental results suggest that Blockmon performs around two times faster than Spark, with a twenty times reduced memory footprint, showing the feasibility of using Blockmon on popular energy- efficient architectures such as the ARM ones.