Skip to Main content Skip to Navigation
Theses

Experimental Methods for the Evaluation of Big Data Systems

Abdulqawi Saif 1, 2
1 RESIST - Resilience and Elasticity for Security and ScalabiliTy of dynamic networked systems
Inria Nancy - Grand Est, LORIA - NSS - Department of Networks, Systems and Services
Abstract : In the era of big data, many systems and applications are created to collect, to store, and to analyze massive data in multiple domains. Although those – big data systems – are subjected to multiple evaluations during their development life-cycle, academia and industry encourage further experimentation to ensure their quality of service and to understand their performance under various contexts and configurations. However, the experimental challenges of big data systems are not trivial. While many pieces of research still employ legacy experimental methods to face such challenges, we argue that experimentation activity can be improved by proposing flexible experimental methods. In this thesis, we address particular challenges to improve experimental context and observability for big data experiments. We firstly enable experiments to customize the performance of their environmental resources, encouraging researchers to perform scalable experiments over heterogeneous configurations. We then introduce two experimental tools: IOscope and MonEx to improve observability. IOscope allows performing low-level observations on the I/O stack to detect potential performance issues in target systems, convincing that the high-level evaluation techniques should be accompanied by such complementary tools to understand systems’ performance. In contrast, MonEx framework works on higher levels to facilitate experimental data collection. MonEx opens directions to practice experiment-based monitoring independently from the underlying experimental environments. We finally apply statistics to improve experimental designs, reducing the number of experimental scenarios and obtaining a refined set of experimental factors as fast as possible. At last, all contributions complement each other to facilitate the experimentation activity by working almost on all phases of big data experiments’ life-cycle.
Document type :
Theses
Complete list of metadatas

Cited literature [152 references]  Display  Hide  Download

https://hal.univ-lorraine.fr/tel-02499941
Contributor : Thèses Ul <>
Submitted on : Thursday, March 5, 2020 - 4:09:12 PM
Last modification on : Thursday, April 30, 2020 - 10:01:53 PM
Long-term archiving on: : Saturday, June 6, 2020 - 4:12:56 PM

File

DDOC_T_2020_0001_SAIF.pdf
Files produced by the author(s)

Identifiers

  • HAL Id : tel-02499941, version 1

Citation

Abdulqawi Saif. Experimental Methods for the Evaluation of Big Data Systems. Computer Science [cs]. Université de Lorraine, 2020. English. ⟨NNT : 2020LORR0001⟩. ⟨tel-02499941⟩

Share

Metrics

Record views

204

Files downloads

508