. ?-?-?-?-?-?-?-?-?-?-?-?-?-?-?-?-?-?-?-?-?-?-?-?-?,

. ?-?-?-?-?-?-?-?-?-?-?-?-?-?-?-?-?-?-?-?-?-?-?-?-?,

. ?-?-?-?-?-?-?-?-?-?-?-?-?-?-?-?-?-?-?-?-?-?-?-?-?,

. ?-?-?-?-?-?-?-?-?-?-?-?-?-?-?-?-?-?-?-?-?-?-?-?-?-?-?-?-?-?-?-?-?-?-?-?-?-?-?-?-?-?-?-?-?-?-?-?-?-?,

. ?-?-?-?-?,

J. B-i-b-l-i-o-g-r-a-p-h-y-[-]-veronika-abramova and . Bernardino, NoSQL databases: Mon-goDB vs cassandra, Proceedings of the international C* conference on computer science and so tware engineering

K. Sungyong-ahn, J. La, and . Kim, Improving I/O Resource Sharing of Linux Cgroup for NVMe SSDs on Multi-core Systems

A. and T. Magedanz, Monitoring and measurement architecture for federated Future Internet experimentation facilities, European Conference on Networks and Communications (EuCNC). IEEE

Y. Al, -. Hazmi, and T. Magedanz, MOFI: Monitoring ontology for federated infrastructures, IEEE International Workshop on. IEEE

Y. Al, -. Hazmi, and T. Magedanz, Towards semantic monitoring data collection and representation in federated infrastructures, rd International Conference on Future Internet of Things and Cloud

O. Paul, D. Yoo, S. Muhaidat, K. George, K. Karagiannidis et al., E ficient machine learning for big data: A review, In: Big Data Research

A. Alexandrov, C. Brücke, and V. Markl, Issues in big data testing and benchmarking, Proceedings of the Sixth International Workshop on Testing Database Systems

, Jiju Antony. Design of experiments for engineers and scientists

D. Balouek, Adding Virtualization Capabilities to the Grid' Testbed, Communications in Computer and Information Science. , pp. -. isbn
URL : https://hal.archives-ouvertes.fr/hal-00946971

C. Baru and T. Rabl, Application-Level Benchmarking of Big Data Systems, Big Data Analytics

C. Baru, M. Bhandarkar, and R. Nambiar, Setting the direction for big data benchmark standards, Technology Conference on Performance Evaluation and Benchmarking

C. Baru, M. Bhandarkar, R. Nambiar, M. Poess, and T. Rabl, Benchmarking big data systems and the bigdata top list, In: Big Data

C. John, H. Bertot, and . Choi, Proceedings of the th Annual International Conference on Digital Government Research. dg.o '

E. Betke and J. Kunkel, Real-Time I/O-Monitoring of HPC Applications with SIOX, Elasticsearch, Grafana and FUSE

M. Bjørling, J. González, and P. Bonnet, LightNVM: The Linux Open-Channel SSD Subsystem, In: FAST

M. Bjørling, Open-Channel Solid State Drives, Vault, Mar (, p.p

E. P. George, J. Box, and . Hunter, The k-p fractional factorial designs, In: Technometrics . (

M. Brattstrom and P. Morreale, Scalable Agentless Cloud Network Monitoring

P. Bühlmann and S. Van-de-geer, Statistics for big data: A perspective, In: Statistics & Probability Letters

J. Michael, D. J. Carey, . Dewitt, J. Michael, N. E. Franklin et al., Shoring up persistent applications

R. Cattell, Scalable SQL and NoSQL data stores, In: Acm Sigmod Record

A. Sheri-fo-ceesay, B. Barker, and . Varghese, Plug and play bench: Simplifying big data benchmarking using containers, IEEE International Conference on Big Data (Big Data)

D. Chahal, R. Virk, and M. Nambiar, Performance extrapolation of io intensive workloads: Work in progress, Proceedings of the th ACM/SPEC on International Conference on Performance Engineering

A. Charbon, B. Harrington, . Fields, . Myklebust, J. Jayaraman et al., NFSv Test Project, Proceedings to the Linux Symposium

H. Chen, R. Kazman, and S. Haziyev, Agile big data analytics development: An architecture-centric approach, th Hawaii International Conference on System Sciences (HICSS). IEEE

H. Chen, R. Kazman, and S. Haziyev, Strategic prototyping for developing big data systems

M. Chen, S. Mao, and Y. Liu, Big data: A survey

M. Chen, Linux NFSv . Performance Under a Microscope

M. Chen, Newer Is Sometimes Better: An Evaluation of NFSv, SIGMETRICS

B. Chun, Planetlab: an overlay testbed for broad-coverage services, In: ACM SIGCOMM Computer Communication Review

F. Clouet, A unified monitoring framework for energy consumption and network tra fic

F. Brian, A. Cooper, E. Silberstein, R. Tam, R. Ramakrishnan et al., Benchmarking cloud serving systems with YCSB, Proceedings of the st ACM symposium on Cloud computing. ACM

J. H. Piet, . Daas, J. Marco, B. Puts, . Buelens et al., Big data as a source for o ficial statistics, In: Journal of O ficial Statistics

H. Daoud and . Michel-r-dagenais, Recovering disk storage metrics from low-level trace events

A. Mauro, M. Greco, and M. Grimaldi, What is big data? A consensual definition and a review of key research topics, AIP conference proceedings

A. Mauro, M. Greco, and M. Grimaldi, A formal definition of Big Data based on its essential features, In: Library Review

E. Dede, B. Sendir, P. Kuzlu, J. Hartog, and M. Govindaraju, An Evaluation of Cassandra for Hadoop

E. Dede, M. Govindaraju, D. Gunter, R. Shane-canon, L. Ramakrishnan et al., Performance evaluation of a mongodb and hadoop platform for scientific data analysis, In: Proceedings of the th ACM workshop on Scientific cloud computing. ACM

L. Citeseer and . Symposium,

M. Thaer, K. Dieb, and . Tsuda, Machine Learning-Based Experimental Design in Materials Science, In: Nanoinformatics. Springer

C. Christopher, C. Drovandi, . Holmes, M. James, K. Mcgree et al., Principles of experimental design for big data analysis, In: Statistical science: a review journal of the Institute of Mathematical Statistics, p.p

. Gc-fox, . Jha, . Qiu, A. Ekanazake, and . Luckow, Towards a comprehensive set of big data benchmarks, Big Data and High Performance Computing (, p.p

C. Fredriksson, F. Mubarak, M. Tuohimaa, and M. Zhan, Big data in the public sector: A systematic literature review, In: Scandinavian Journal of Public Administration

A. Gandini, M. Gribaudo, J. William, R. Knottenbelt, P. Osman et al., Performance evaluation of NoSQL databases, European Workshop on Performance Engineering. Springer

J. Gantz and D. Reinsel, Extracting value from chaos, In: IDC iview

N. Garg, S. Singla, and S. Jangra, Challenges and Techniques for Testing of Big Data, Procedia Computer Science (

D. Giatsios and A. Apostolaras, Methodology and tools for measurements on wireless testbeds: The nitos approach, In: Measurement Methodology and Tools

M. Gudipati, S. Rao, D. Naju, N. Mohan, and . Gajja, Big data: Testing approach to overcome quality challenges, In: Big Data: Challenges and Opportunities

J. Gustedt, E. Jeannot, and M. Quinson, Experimental methodologies for large-scale systems: a survey, In: Parallel Processing Letters
URL : https://hal.archives-ouvertes.fr/inria-00364180

R. Han, J. Lizy-kurian-john, and . Zhan, Benchmarking big data systems: A review, IEEE Transactions on Services Computing

R. Han, Z. Jia, W. Gao, X. Tian, and L. Wang, Benchmarking Big Data Systems: State-of-the-Art and Future Directions

S. Harizopoulos, J. Daniel, S. Abadi, M. Madden, and . Stonebraker, OLTP through the looking glass, and what we found there, Proceedings of the ACM SIGMOD international conference on Management of data

B. Hindman, A. Konwinski, M. Zaharia, A. Ghodsi, D. Anthony et al., Mesos: A platform for fine-grained resource sharing in the data center, In: NSDI

K. Hinkelmann and O. Kempthorne, Design and Analysis of Experiments

D. Huang, J. Wang, Q. Liu, X. Zhang, X. Chen et al., DFS-container: achieving containerized block I/O for distributed file systems, Proceedings of the Symposium on Cloud Computing

K. Huppler, The art of building a good benchmark, Technology Conference on Performance Evaluation and Benchmarking

T. Ivanov, T. Rabl, M. Poess, A. Queralt, J. Poelman et al., In: Technology Conference on Performance Evaluation and Benchmarking

B. Jacob, P. Larson, B. Leitao, and S. Silva, SystemTap: instrumenting the Linux kernel for analyzing performance and functional problems, In: IBM Redbook (

J. Hv-jagadish, A. Gehrke, Y. Labrinidis, . Papakonstantinou, M. Jignesh et al., Big data and its technical challenges, Communications of the ACM

R. Jain, The art of computer systems performance analysis

E. Jeannot, Experimental validation of grid algorithms: A comparison of methodologies, IEEE International Symposium on Parallel and Distributed Processing
URL : https://hal.archives-ouvertes.fr/inria-00333898

S. Jeong, K. Lee, J. Hwang, S. Lee, and Y. Won, AndroStep: Android Storage Performance Analysis Tool, In: So tware Engineering (Workshops)

I. Jimenez and M. Sevilla, The Popper Convention: Making Reproducible Systems Evaluation Practical, Parallel and Distributed Processing Symposium Workshops (IPDPSW)

. Bpf-manual-page-on-linux,

B. Zhen-hua-liu, D. Hammerschmidt, Y. Mcmahon, H. Liu, and . Chang, Closing the functional and Performance Gap between SQL and NoSQL, Proceedings of the International Conference on Management of Data. ACM

S. Lohr, The age of big data, New York Times

X. Luo, F. Mueller, P. Carns, J. Jenkins, R. Latham et al., Hpc i/o trace extrapolation, Proceedings of the th Workshop on Extreme Scale Programming Tools

X. Luo, F. Mueller, P. Carns, J. Jenkins, R. Latham et al., ScalaIOExtrap: Elastic I/O Tracing and Extrapolation, Parallel and Distributed Processing Symposium (IPDPS)

A. , K. Grolinger, F. Hany, M. Elyamany, and . Capretz, Machine learning with big data: Challenges and approaches, In: IEEE Access

S. Ma, J. Jiang, B. Li, and B. Li, Custody: Towards data-aware resource sharing in cloud-based big data processing, IEEE International Conference on Cluster Computing (CLUSTER). IEEE

M. Sushil-govindnarayan, E ficient In-Depth IO Tracing and its application for optimizing systems

A. Ovidiu-cristian-marcu, G. Costan, M. Antoniu, and . Pérez, Spark versus flink: Understanding performance in big data analytics frameworks, Cluster -The IEEE International Conference on Cluster Computing

P. Richard, D. E. Martin, and . Culler, NFS sensitivity to high performance networks, In: SIGMETRICS Performance Evaluation Review

L. Matthew and . Massie, The ganglia distributed monitoring system: design, implementation, and experience, In: Parallel Computing (

M. Mattoso, C. Werner, G. Horta-travassos, V. Braganholo, E. Ogasawara et al., Towards supporting the life cycle of large scale scientific experiments, In: International Journal of Business Process Integration and Management

R. Mcdougall, J. Mauro, and B. Gregg, Solaris performance and tools: DTrace and MDB techniques for Solaris and OpenSolaris

Z. Ming, C. Luo, W. Gao, R. Han, Q. Yang et al., BDGS: A scalable big data generator suite in big data benchmarking, Advancing Big Data Benchmarks

M. P. Ashlesha-s-nagdive, R. M. Tembhurkar, and . Tugnayat, Overview on performance testing approach in big data, In: International Journal of Advanced Research in Computer Science

D. Nelubin and B. Engber, Ultra-High Performance NoSQL Benchmarking, White paper in Aerospike website

C. Newcombe, T. Rath, F. Zhang, B. Munteanu, M. Brooker et al., How Amazon web services uses formal methods, Communications of the ACM

L. Nussbaum and O. Richard, Lightweight emulation to study peer-to-peer systems, Concurrency and Computation: Practice and Experience
URL : https://hal.archives-ouvertes.fr/hal-00097816

T. Oetiker and D. Rand, MRTG: The Multi Router Tra fic Grapher, In: LISA

, Rihards Olups. Zabbix . network monitoring

Z. Ou, Z. Hwang, A. Ylä-jääski, F. Chen, and R. Wang, Is Cloud Storage Ready? A Comprehensive Study of IPbased Storage Systems

C. Pahl and B. Lee, Containers and clusters for edge cloud architectures-A technology review

C. Pahl, A. Brogi, J. Soldani, and P. Jamshidi, Cloud container technologies: a state-of-the-art review, IEEE Transactions on Cloud Computing

F. Paraiso, C. Stéphanie, A. Yahya, and P. Merle, Model-Driven Management of Docker Containers, th IEEE International Conference on Cloud Computing (CLOUD)
URL : https://hal.archives-ouvertes.fr/hal-01314827

S. Patil, M. Polte, K. Ren, W. Tantisiriroj, L. Xiao et al., YCSB++: benchmarking and performance debugging advanced features in scalable table stores, Proceedings of the nd ACM Symposium on Cloud Computing

E. Pesce, E. Riccomagno, and H. Wynn, Experimental Design Issues in Big Data. The Question of Bias

X. Qin and X. Zhou, A survey on benchmarks for big data and some more considerations, International Conference on Intelligent Data Engineering and Automated Learning

T. Rabl and H. Jacobsen, In: Specifying Big Data Benchmarks

P. Radkov, L. Yin, P. Goyal, P. Sarkar, and P. Shenoy, A Performance Comparison of NFS and iSCSI for IP-networked Storage

A. Rajasekar, iRODS Primer: integrated rule-oriented data system, Synthesis Lectures on Information Concepts, Retrieval, and Services

M. Thierry-rakotoarivelo, G. Ott, I. Jourjon, and . Seskar, OMF: a control and management framework for networking testbeds, In: ACM SIGOPS Operating Systems Review

O. Rensfelt, L. Larzon, and S. Westergren, Vendetta -a tool for flexible monitoring and management of distributed testbeds

C. Ruiz, E. Jeanvoine, and L. Nussbaum, Porting the Distem Emulator to the CloudLab and Chameleon testbeds, Research Report RR-. Inria, Sept
URL : https://hal.archives-ouvertes.fr/hal-01372050

A. Saif and L. Nussbaum, Performance Evaluation of NFS over a Wide-Area Network, COMPAS -Conférence d'informatique en Parallélisme
URL : https://hal.archives-ouvertes.fr/hal-01327272

A. Saif, L. Nussbaum, and Y. Song, MongoDB I/O Access Patterns are under the Microscope. Annual PhD students conference IAEM Lorraine . Poster
URL : https://hal.archives-ouvertes.fr/hal-01793531

A. Saif, L. Nussbaum, and Y. Song, Performance Evaluation of MongoDB I/O access patterns
URL : https://hal.archives-ouvertes.fr/hal-01793479

, Action transverse Virtualisation et Cloud du GdR RSD

A. Saif, L. Nussbaum, and Y. Song, IOscope: A Flexible I/O Tracer for Workloads' I/O Pattern Characterization, Abdulqawi Saif, Lucas Nussbaum, and Ye-Qiong Song. On the Impact of I/O Access Patterns on SSD Storage
URL : https://hal.archives-ouvertes.fr/hal-01828249

A. Saif, A. Merlin, L. Nussbaum, and Y. Song, In: P-RECS : st International Workshop on Practical Reproducible Evaluation of Computer Systems

A. Saif, A. Merlin, L. Nussbaum, and Y. Song, In: Grid -FIT Spring School. SILECS project, INRIA

A. Saif, A. Merlin, O. Dautricourt, M. Houbre, L. Nussbaum et al., Emulation of Storage Performance in Testbed Experiments with Distem, CNERT -IEEE INFOCOM International Workshop on Computer and Networking Experimental Research using Testbeds
URL : https://hal.archives-ouvertes.fr/hal-02078301

H. Saissi, P. Bokor, C. Arda-mu-tuoglu, N. Suri, and M. Serafini, E ficient verification of distributed protocols using stateful model checking, IEEE nd International Symposium on Reliable Distributed Systems. IEEE

A. Vikram, K. Saletore, V. Krishnan, M. E. Viswanathan, and . Tolentino, HcBench: Methodology, development, and full-system characterization of a customer usage representative big data/hadoop benchmark, Advancing big data benchmarks

T. Salmito, L. Ciu-fo, and I. Machado, FIBRE-an international testbed for future internet experimentation, Simpósio Brasileiro de Redes de Computadores e Sistemas Distribuídos
URL : https://hal.archives-ouvertes.fr/hal-00990392

R. Sandberg, D. Goldberg, S. Kleiman, D. Walsh, and B. Lyon, Design and implementation of the Sun network filesystem, In: Summer USENIX conference

L. Sarzyniec, T. Buchert, E. Jeanvoine, and L. Nussbaum, Design and Evaluation of a Virtual Experimental Environment for Distributed Systems, PDP -st Euromicro International Conference on Parallel
URL : https://hal.archives-ouvertes.fr/hal-00724308

L. Sarzyniec, T. Buchert, E. Jeanvoine, and L. Nussbaum, Design and evaluation of a virtual experimental environment for distributed systems
URL : https://hal.archives-ouvertes.fr/hal-00724308

V. Schia-fonati and M. Verdicchio, Computing and experiments, Philosophy & Technology

D. Elizabeth, J. Schifano, C. Wu, J. Wang, M. Yan et al., Online updating of statistical inference in the big data setting, In: Technometrics . (

J. Schindler, I/O characteristics of NoSQL databases, Proceedings of the VLDB Endowment

J. Schindler, Profiling and analyzing the I/O performance of NoSQL DBs, In: ACM SIGMETRICS Performance Evaluation Review

A. ,

J. Schindler, A. Ailamaki, and G. Ganger, Matching Database Access Patterns to Storage Characteristics

J. Schindler and . Gregory-r-ganger, Matching application access patterns to storage device characteristics

J. Schulist, A. Borkmann, and . Starovoitov, Linux socket filtering aka Berkeley packet filter (BPF)

G. Shapira and Y. Chen, Common Pitfalls of Benchmarking Big Data Systems, IEEE Transactions on Services Computing

D. Suchakrapani, M. Sharma, and . Dagenais, Enhanced Userspace and In-Kernel Trace Filtering for Production Systems, In: Journal of Computer Science and Technology

S. Shepler, D. Eisler, and . Noveck, Network file system (NFS) version minor version protocol, The Internet Engineering Task Force (IETF) (

S. Shepler, M. Eisler, D. Robinson, B. Callaghan, R. Thurlow et al., Network file system (NFS) version protocol

J. Shi, Y. Qiu, L. Umar-farooq-minhas, C. Jiao, B. Wang et al., Clash of the titans: MapReduce vs. Spark for large scale data analytics, Proceedings of the VLDB Endowment

M. Singh, M. Ott, I. Seskar, and P. Kamat, ORBIT Measurements framework and library (OML): motivations, implementation and features

H. Song, X. Sun, and Y. Chen, A Hybrid Sharednothing/Shared-data Storage Architecture for Large Scale Databases, th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing

A. Starovoitov,

A. S. Maarten-van-steen and . Tanenbaum, A brief introduction to distributed systems, In: Computing . (

M. Strohbach, J. Daubert, H. Ravkin, and M. Lischka, Big Data Storage, New Horizons for a Data-Driven Economy

P. Sukheepoj and N. Nupairoj, Adaptive I/O Bandwidth Allocation for Virtualization Environment on Xen Platform

C. Byung-chul-tak, H. Tang, L. Huang, and . Wang, Pseudoapp: Performance prediction for application migration to cloud, Integrated Network Management (IM ), IFIP/IEEE International Symposium on. IEEE

V. Tarasov, D. Hildebrand, G. F. Kuenning, and E. Zadok, Virtual Machine Workloads: The Case for New NAS Benchmarks

. Mininet-team and . Mininet,

M. Tedre and N. Moisseinen, Experiments in computing: A survey, The Scientific World Journal

S. Rakshit, . Trivedi, J. Nilavalagan, and . Haritsa, CODD: constructing dataless databases, Proceedings of the Fi th International Workshop on Testing Database Systems

. Hal-r-varian, Beyond big data, Business Economics

V. Kumar-vavilapalli, C. Arun, C. Murthy, S. Douglas, M. Agarwal et al., Apache hadoop yarn: Yet another resource negotiator, Proceedings of the th annual Symposium on Cloud Computing

M. Vef, V. Tarasov, D. Hildebrand, and A. Brinkmann, Challenges and Solutions for Tracing Storage Systems: A Case Study with Spectrum Scale, In: ACM Trans. Storage

B. Videau, C. Touati, and O. Richard, Toward an experiment engine for lightweight grids
URL : https://hal.archives-ouvertes.fr/hal-01398153

R. Virk and D. Chahal, Trace replay based i/o performance studies for enterprise workload migration, nd Annual Conference of CMG India, p.page Online

K. Wang, N. Liu, I. Sadooghi, X. Yang, X. Zhou et al., Overcoming hadoop scaling limitations through distributed task execution, IEEE International Conference on Cluster Computing

L. Wang, J. Zhan, C. Luo, Y. Zhu, Q. Yang et al., Bigdatabench: A big data benchmark suite from internet services, High Performance Computer Architecture (HPCA)

J. S. Ward and A. Barker, Undefined by data: a survey of big data definitions

A. Sage, S. A. Weil, E. L. Brandt, D. D. Miller, C. Long et al., Ceph: A Scalable, High-performance Distributed File System

Y. Xu and M. Zhao, Ibis: interposed big-data i/o scheduler, Proceedings of the th ACM HPDC

V. Marvin, D. Zelkowitz, and . Wallace, Experimental validation in so tware engineering, In: Information and So tware Technology

J. Zhang, A. Sivasubramaniam, A. Riska, Q. Wang, and E. Riedel, An interposed -level I/O scheduling framework for performance virtualization, In: ACM SIGMETRICS Performance Evaluation Review

J. Zhang, A. Sivasubramaniam, Q. Wang, A. Riska, and E. Riedel, Storage performance virtualization via throughput and latency control, ACM Transactions on Storage (TOS)