Skip to Main content Skip to Navigation

Experimental Methods for the Evaluation of Big Data Systems

Abdulqawi Saif 1, 2 
1 RESIST - Resilience and Elasticity for Security and ScalabiliTy of dynamic networked systems
Inria Nancy - Grand Est, LORIA - NSS - Department of Networks, Systems and Services
Abstract : In the era of big data, many systems and applications are created to collect, to store, and to analyze massive data in multiple domains. Although those – big data systems – are subjected to multiple evaluations during their development life-cycle, academia and industry encourage further experimentation to ensure their quality of service and to understand their performance under various contexts and configurations. However, the experimental challenges of big data systems are not trivial. While many pieces of research still employ legacy experimental methods to face such challenges, we argue that experimentation activity can be improved by proposing flexible experimental methods. In this thesis, we address particular challenges to improve experimental context and observability for big data experiments. We firstly enable experiments to customize the performance of their environmental resources, encouraging researchers to perform scalable experiments over heterogeneous configurations. We then introduce two experimental tools: IOscope and MonEx to improve observability. IOscope allows performing low-level observations on the I/O stack to detect potential performance issues in target systems, convincing that the high-level evaluation techniques should be accompanied by such complementary tools to understand systems’ performance. In contrast, MonEx framework works on higher levels to facilitate experimental data collection. MonEx opens directions to practice experiment-based monitoring independently from the underlying experimental environments. We finally apply statistics to improve experimental designs, reducing the number of experimental scenarios and obtaining a refined set of experimental factors as fast as possible. At last, all contributions complement each other to facilitate the experimentation activity by working almost on all phases of big data experiments’ life-cycle.
Document type :
Complete list of metadata

Cited literature [152 references]  Display  Hide  Download
Contributor : Thèses UL Connect in order to contact the contributor
Submitted on : Thursday, March 5, 2020 - 4:09:12 PM
Last modification on : Friday, February 4, 2022 - 3:34:50 AM
Long-term archiving on: : Saturday, June 6, 2020 - 4:12:56 PM


Files produced by the author(s)


  • HAL Id : tel-02499941, version 1


Abdulqawi Saif. Experimental Methods for the Evaluation of Big Data Systems. Computer Science [cs]. Université de Lorraine, 2020. English. ⟨NNT : 2020LORR0001⟩. ⟨tel-02499941⟩



Record views


Files downloads