The advent of the Big Data era and the diffusion of Cloud computing have renewed the interest in Data Stream Processing (DSP) applications, which can timely extract useful information from distributed data sources. Due to the unpredictable rate at which the sources may produce data, DSP applications demand high dynamism. Storm has emerged as a widely adopted DSP system, which, although having many desirable features, shows some limitations due to the lack of adaptation capabilities. In this paper, we extend Storm with two mechanisms that support the run-time adaptation of DSP applications. Specifically, we introduce new components that allow automatic elasticity and stateful migration of the application components. The experimental results show the benefits of the newly introduced functionalities that, albeit equipped with proof of concept policies, allow to properly cope with workload variations while improving the resource utilization of the underlying infrastructure.
Cardellini, V., Nardelli, M., Luzi, D. (2016). Elastic stateful stream processing in storm. In Proceedings of the 2016 International Conference on High Performance Computing & Simulation (HPCS 2016) (pp.583-590). IEEE [10.1109/HPCSim.2016.7568388].
Elastic stateful stream processing in storm
CARDELLINI, VALERIA;
2016-01-01
Abstract
The advent of the Big Data era and the diffusion of Cloud computing have renewed the interest in Data Stream Processing (DSP) applications, which can timely extract useful information from distributed data sources. Due to the unpredictable rate at which the sources may produce data, DSP applications demand high dynamism. Storm has emerged as a widely adopted DSP system, which, although having many desirable features, shows some limitations due to the lack of adaptation capabilities. In this paper, we extend Storm with two mechanisms that support the run-time adaptation of DSP applications. Specifically, we introduce new components that allow automatic elasticity and stateful migration of the application components. The experimental results show the benefits of the newly introduced functionalities that, albeit equipped with proof of concept policies, allow to properly cope with workload variations while improving the resource utilization of the underlying infrastructure.File | Dimensione | Formato | |
---|---|---|---|
hpcs2016.pdf
solo utenti autorizzati
Tipologia:
Documento in Post-print
Licenza:
Copyright dell'editore
Dimensione
1.17 MB
Formato
Adobe PDF
|
1.17 MB | Adobe PDF | Visualizza/Apri Richiedi una copia |
I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.