Improving big-data automotive applications performance through adaptive resource allocation

In automotive applications, connected vehicles (CVs) can collect various information (external temperature, speed, location, etc.) and send them to a central infrastructure for exploitation in a wide range of applications: Eco-Driving, Beet management, environmental monitoring, etc. Such applications are known to generate a massive volume of data that is processed in real or near real time (i.e., data streams) depending on the target application requirements. To handle this data volume, big data architectures, based on stream computing paradigm, are usually adopted. Within this paradigm, data are continuously processed by a set of operators (elementary operations) instances. Further, a streaming application can be modeled as a directed graph where vertices are operators instances and edges are data streams (i.e., continuous series of tuples, generated by an operator). The central challenge when developing streaming applications is the way to assign operators to given resources for optimal performances (i.e., resource allocation). We showed that straightforward allocation, when based on intrinsic data parallelism, do not yield satisfying results. We developed a novel approach that takes into account the specifics of the target application, on the one hand, and on the other hand the systems performance metrics we derived from the infrastructure. The proposed approach improves the throughput by 4% compared to the previous approach. In the automotive context, this number translates into 800k additional vehicles to be served by the infrastructure over the 20M of the expected number of connected vehicles by Groupe PSA.