Skip to main content

Über dieses Buch

Understand the fundamental factors of data storage system performance and master an essential analytical skill using block trace via applications such as MATLAB and Python tools. You will increase your productivity and learn the best techniques for doing specific tasks (such as analyzing the IO pattern in a quantitative way, identifying the storage system bottleneck, and designing the cache policy).

In the new era of IoT, big data, and cloud systems, better performance and higher density of storage systems has become crucial. To increase data storage density, new techniques have evolved and hybrid and parallel access techniques—together with specially designed IO scheduling and data migration algorithms—are being deployed to develop high-performance data storage solutions. Among the various storage system performance analysis techniques, IO event trace analysis (block-level trace analysis particularly) is one of the most common approaches for system optimization and design. However, the task of completing a systematic survey is challenging and very few works on this topic exist.

Block Trace Analysis and Storage System Optimization brings together theoretical analysis (such as IO qualitative properties and quantitative metrics) and practical tools (such as trace parsing, analysis, and results reporting perspectives). The book provides content on block-level trace analysis techniques, and includes case studies to illustrate how these techniques and tools can be applied in real applications (such as SSHD, RAID, Hadoop, and Ceph systems).

What You’ll Learn

Understand the fundamental factors of data storage system performance

Master an essential analytical skill using block trace via various applications

Distinguish how the IO pattern differs in the block level from the file level

Know how the sequential HDFS request becomes “fragmented” in final storage devices

Perform trace analysis tasks with a tool based on the MATLAB and Python platforms

Who This Book Is For

IT professionals interested in storage system performance optimization: network administrators, data storage managers, data storage engineers, storage network engineers, systems engineers



Chapter 1. Introduction

The chapter provides the background of data storage systems and general trace analysis. I will show that wide applications of block storage devices motivate the intensive study of various block-level workload properties. I will also list the objectives and contributions of this book in this chapter.
Jun Xu

Chapter 2. Trace Characteristics

Trace is usually classified as three levels: block level, file level, and object level. They share many common metrics, although each has its own unique properties. In this chapter, I will discuss block-level trace metrics in detail since the block-level trace provides more fundamental information on storage systems than other two levels. For simplicity of representation, I divide the metrics into two categories: the basic ones and the advanced ones. The meanings and performance impacts of these metrics are explained in detail.
Jun Xu

Chapter 3. Trace Collection

Trace quality is one of the essential requirements for analysis. Low quality traces may lead to complex and wrong conclusions for trace analysis. There are two main issues in trace quality. One is timing drift, which is when the actual event arrival time is earlier than the collected arrival time. The other is a missing event, such as when the tool cannot capture all the required events. Thus, proper tools shall be applied to guarantee the correctness of the traces. Both software tools and hardware devices are introduced in this chapter.
Jun Xu

Chapter 4. Trace Analysis

Trace analysis provides insights into workload properties and IO patterns, which are essential for storage system tuning and optimizing. This chapter discusses how the workload interacts with system components, algorithms, structures, and applications.
Jun Xu

Chapter 5. Case Study: Benchmarking Tools

Benchmark tools are useful to provide some “standard” performance indexes for storage systems with specific requirements. This chapter shows how to identify the access pattern of benchmark results. The first tool is SPC-1C from the Storage Performance Council (SPC). After capturing the pattern, I developed a synthetic emulator to match the real traces. The second tool is PCMark from FutureMark. I illustrate how to use gain-loss analysis to improve cache algorithm efficiency.
Jun Xu

Chapter 6. Case Study: Modern Disks

Modern disks implement many different features, such as media-based cache (e.g., using a portion of disk space to log some random write accesses), DRAM protection (e.g., using a small-size NVM to temporarily store some data in DRAM cache during a power loss such that write-cache can be always enabled), hybrid structure (e.g., migrating hot data to high-speed devices and cold data to low-speed devices so that the overall access time is reduced), etc. A hybrid disk (e.g., SSHD), one of the hybrid structures, has advantages in some scenarios where data hotness is significant. Some emerging and future techniques like SMR, HAMR, and BPR favor sequential access in order to diminish garbage collection, reduce energy consumption, and/or improve the device life. This chapter shows how trace analysis can help to identify these mechanisms via workload property analysis using two examples: SSHD and SMR drives.
Jun Xu

Chapter 7. Case Study: RAID

RAID is one of the most widely applied data-protection strategies in the world [23, 60, 61, 62, 63]. It has unique features compared with single disk access, such as file synchronization, recovery, etc. Therefore, it leads to some unique IO patterns compared with others. This chapter analyzes two examples based on RAID 5 from two application scenarios. Large differences are observed between two traces. This chapter also analyzes whether the workloads are suitable for SMR drives. In addition, some suggestions are provided in order to improve system performance.
Jun Xu

Chapter 8. Case Study: Hadoop

Hadoop is one of the most popular distributed big data platforms in the world. Besides computing power, its storage subsystem capability is also a key factor in its overall performance. In particular, there are many intermediate file exchanges for MapReduce. This chapter presents the block-level workload characteristics of a Hadoop cluster by considering some specific metrics. The analysis techniques presented can help you understand the performance and drive characteristics of Hadoop in production environments. In addition, this chapter also identifies whether SMR drives are suitable for the Hadoop workload.
Jun Xu

Chapter 9. Case Study: Ceph

Ceph, an open-source distributed storage platform, provides a unified interface for object-, block-, and file-level storage [33, 80, 34, 81]. This chapter presents the block-level workload characteristics of a WD WASP/EPIC microserver-based Ceph cluster. The analysis techniques presented can help you to understand the performance and drive characteristics of Ceph in production environments. In addition, I also identify whether SMR, hybrid disk, and SSD drives are suitable for the Ceph workload.
Jun Xu


Weitere Informationen

Premium Partner

BranchenIndex Online

Die B2B-Firmensuche für Industrie und Wirtschaft: Kostenfrei in Firmenprofilen nach Lieferanten, Herstellern, Dienstleistern und Händlern recherchieren.



Best Practices für die Mitarbeiter-Partizipation in der Produktentwicklung

Unternehmen haben das Innovationspotenzial der eigenen Mitarbeiter auch außerhalb der F&E-Abteilung erkannt. Viele Initiativen zur Partizipation scheitern in der Praxis jedoch häufig. Lesen Sie hier  - basierend auf einer qualitativ-explorativen Expertenstudie - mehr über die wesentlichen Problemfelder der mitarbeiterzentrierten Produktentwicklung und profitieren Sie von konkreten Handlungsempfehlungen aus der Praxis.
Jetzt gratis downloaden!