Benchmarks/Metrics

Several tools test system performance. Some are specific to an application or environment, while others are more general. Whenever a tool is used, it is critical to understand for what the tool was designed and how it operates in different environments and with different storage array features such as deduplication. The following are several tools and the associated use case for each:

• IOmeter. IOmeter is an I/O subsystem measurement and characterization tool for single and clustered systems. It was originally developed by Intel Corporation. For more information about IOmeter, refer to http://iometer.org.

• OLTP-A. OLTP-A consists of a single workload with 8K blocks and a 40%/60% read/write mix with both random and sequential patterns. Though most databases are more read intensive, this benchmark was selected because of the write environment, which in an all-SSD system tends to be the bottleneck. The workload simulates a write-heavy online transaction-processing database and executes both queries and updates to the database during operation.

• SLOB (Silly Little Oracle Benchmark). SLOB is a complete toolkit for generating I/O through an Oracle database and is used to analyze the I/O capabilities of the database. SLOB is designed to generate high I/O levels without causing application bottlenecks.

• VMmark. VMmark is a free tool from VMware to measure performance for applications running in VMware environments. For example, this tool helps to identify the number of applications that can be supported using a single storage system.

• sqlio. sqlio is a tool provided by Microsoft that can also be used to determine the I/O capacity of a configuration.

• Vdbench. Vdbench is a command-line tool to generate disk I/O for validating storage performance.

• STAC-M3. The STAC Benchmark Council developed the STAC-M3 Benchmarks. These benchmarks are primarily used in the financial community to measure performance associated with financial applications.

Metrics Terminology

The terms used to describe performance can be thought of in three pillars: I/Os per second, latency, and throughput or bandwidth. Depending on the workload, the performance test values obtained are dramatically different by design intent. For example, if the workload is predominantly small random reads, the IOPS values are high, but the throughput values are relatively low in comparison. Conversely, if the workload is mostly large sequential I/O, especially 1M I/O, the measured IOPS values and throughput values are close to the same. Both of these circumstances are expected behaviors. In general terms, the relationship between IOPS and throughput can be expressed as:

Throughput (MBps) = IOPS multiplied by block size (MB)

The following sections explore each performance measure in more detail.

I/Os per Second (IOPS)

I/Os per second is the measure of how many input/output communications pass between an initiator (host server) and a target (E-Series storage system) in one second. Based on the protocol that manages the path between the initiator and target, the maximum level of IOPS achievable varies significantly. For example, a 10Gbps iSCSI link cannot process the same level of I/Os as a 32Gb FC link. Factors external to the storage system also introduce overhead to communications and affect IOPS resulting from multiple network settings and host settings on HBAs, HCAs, and NICs to settings in OSs and multiple other host-side or application-related issues. Therefore, achievable IOPS for a given system depends on many factors and must be understood holistically for you to achieve the best performance profile for a given environment.

Latency

The second pillar is latency: the time required to move an I/O from the initiator to the target or back in the other direction. All of the same factors that affect IOPS have a related impact on latency. However, the other measures ultimately top out to the limits of the hardware and protocols involved. Latency spikes and grows dramatically as hardware and protocol limits are exceeded. As a result, latency is the measure used to define operating ranges that apply to normal data center workloads.

Throughput/Bandwidth

Throughput is the third pillar of storage performance and is a measure of how much data can pass between host initiators and storage system targets in one second. Like IOPS, throughput is heavily related to the type of workflow. For example, from a throughput perspective, it takes many small I/Os to equal one large IOPS. As a result, the type of I/O transferred as well as the bandwidth (size of links) and protocol used play significant roles in the amount of data that can be transferred in one second. With this fact in mind, some host OS suppliers have built in tunable settings to allow hosts to use larger I/Os or block sizes when transferring data to and from storage. As a result, just like IOPS, the ability to achieve certain throughput requirements is a multilevel activity and depends on factors both inside the storage system and within each unique customer data center.

Bye...