NVMe Partitioning Guidelines – Open-E’s Best Practices

The latest version of Open-E JovianDSS Up30, launched on January 2024, comes with many sophisticated features and enhancements designed to boost your data storage’s performance, reliability, safety, and cost-effectiveness. A standout feature in this release is the introduction of ZFS Special Devices, a functionality that enables to store the particular data types on high-speed storage devices. Other functions worth mentioning and provided with this update of Open-E JovianDSS are MacOS Time Machine Support, Active Directory with RID Range, Active Directory support for rfc2307 (AD backend for Samba), SED support for non-shared storage clusters, or Samba Recycle Bin Support for Windows among others.

Open-E JovianDSS Up30 also provided another great feature of NVMe partition utilization, which many IT administrators highlight as extremely helpful and significantly increasing the system performance if NVMe drives are used in the storage matrix. It allows for segmenting NVMe disk space and using a single device for various functions, such as read cache, write log, or Special Devices. This feature enables users to maximize their disk space usage, reduce costs, and streamline their data storage administration and upkeep.

Write Log / Read Cache / Special Devices – Open-E Partitioning Guidelines

However, during a webinar recently organized to demonstrate the possibilities of Open-E JovianDSS Up30, questions were raised about partitioning itself. One of the participants asked us if there were any guidelines from Open-E regarding the size distribution for Write Cache / Read Cache / Special Devices partitions. Yes, there are, and below, you can find out which and what to consider when planning the volume distribution of each partition and its respective function.

Write Log Partition Guidelines

When deciding on the disk size for the write log, the key factor to consider is the potential volume of data that could be sent to the server in three successive ZFS operations. This could be determined, for instance, by the data transfer bandwidth of the network card. If the duration is set to the default 5 seconds, the write log device should be capable of holding the volume of data that can be transferred within three operation groups, which equates to 15 seconds of writing. Economically, using a larger disk isn’t justified, while a smaller one could potentially hinder performance during synchronous writes. In practical terms, a write log of 100 GB should be more than adequate.

Also we would like to highlight that we recommend configuring the Write Log with redundancy, i.e,. as a mirrored Write Log. This provides a type of of data security when one of the disks used as Write Log fails.

Read Cache Partition Guidelines

To determine the best size of the read cache, taking into account the RAM needed, the bytes reserved by single read cache header structure size (the l2hdr structure), and volblocksize (or recordsize), we can use the following formula to make an approximate calculation:

Read cache size = (RAM owned × volblocksize or recordsize / bytes reserved by l2hdr structure) 

Volblocksize is a fixed value that ensures any data written to a ZFS volume (zvol) will be stored in blocks that match the specified volblocksize.

The bytes reserved by single read cache header structure size (l2hdr structure) is the part for each cached record that must be stored in RAM.

Let’s use this new formula for:

57981809664B – 54GB RAM owned

70B – bytes reserved by the single read cache header structure size (l2hdr structure)

8192B – volblocksize or recordsize.

Size of Read Cache = (57981809664B * 8192B / 70B)

You can calculate the exact value using these numbers. Please note that the result will be in bytes, and you may want to convert it to a more readable unit like GB or TB.

Special Devices Guidelines

In the case of Special Devices, it’s not that simple. However, we are able to provide some basic guidelines to help you understand what to consider when creating the NVMe partition for this purpose.

Understand Workload and Data Type:

• The performance improvement percentage provided by ZFS special devices can vary significantly based on the workload and data type. Consider the specific use case, whether it involves heavy metadata operations, large file transfers, or a mix of both.
• Remember that there isn’t a universal percentage increase; it depends on the specifics of your workload

Hardware Considerations:

• Evaluate the performance characteristics of the storage devices used for the special vdev. If possible, use high-performance NVMe for special devices
• Contrast this with the primary pool storage. For instance, if your main pool consists of slower HDDs, the performance boost for metadata operations and small file access can be substantial when using NVMe for the special vdev

Benchmark Testing:

• Conduct benchmark tests that mimic your actual use case to measure the performance difference accurately
• Compare performance with and without the special vdev under identical conditions and workloads. It will provide insights into the impact of special devices on overall system performance

Special Devices Group:

Devices placed in the special devices group serve specific purposes:

• Metadata: Store metadata efficiently. Metadata-intensive workloads benefit from faster access to metadata
• Indirect Blocks: These blocks are crucial for data access Optimize their storage within the special devices group
• Deduplication Tables: Decide whether to include deduplication tables in the special devices group or place them in a separate group – the deduplication group

Provisioning for Small File Blocks:

• Configure special devices to accept small file blocks. Adjust the block size as needed using the relevant settings

As can be concluded from the above guidelines, there is no specific numerical data on partition capacity. However, it should be taken into account that it may require between 1% and 10% of the volume of the entire data storage capacity.

Remember that the effectiveness of special devices depends on your unique environment, workload, and hardware configuration. 

Video Tutorial – Setup in Practice

If you’re interested in maximizing the efficiency of your NVMe disks using Open-E JovianDSS Up30, check out the video below. This recent update allows you to create partitions on your NVMe drives for read cache, write log, and special devices. Doing so can save costs, reduce hardware requirements, and gain greater flexibility in designing and maintaining your data storage system!