A Real-Time Massive Data Processing Technique for Densely Distributed Sensor Networks

Today, we are awash in a flood of data coming from different data generating sources. Wireless sensor networks (WSNs) are one of the big data contributors, where data are being collected at unprecedented scale. Unfortunately, much of these data are of no interest, meaningless, and redundant. Hence, data reduction is becoming fundamental operation in order to decrease the communication costs and enhance data mining in WSNs. In this paper, we propose a two-level data reduction approach for sensor networks. The first level operated by the sensor nodes consists of compressing collected data while using the Pearson coefficient. The second level is executed at intermediate nodes (e.g., aggregators, cluster heads, and so on). The objective of the second level is to eliminate redundant data generated by neighboring nodes using two adapted clustering methods: EKmeans and TopK. Through both simulations and real experiments on real telosB sensors, we show the relevance of our approach in terms of minimizing the big data collected in WSNs and enhancing network lifetime, compared to other existing techniques.