The Journey of Progress: ZFS features in Open-E JovianDSS Up29 – Up30

Open-E JovianDSS continues to evolve as a robust ZFS-based data storage solution, with a clear focus on improving performance, security, and manageability to meet the needs of our customers. We have recently received some questions about the development of the system in terms of ZFS features. So, we decided to answer them and summarise the latest Open-E JovianDSS updates in the context of the implemented ZFS features. From Open-E JovianDSS Up29 to the Open-E JovianDSS Up30 released this year, the system has significantly advanced its ZFS-based features, strengthening its position as a leading choice for enterprises across diverse industries. Knowing how much ZFS – an open-source filesystem, has to offer, we strive to use its advantages to improve the performance and capabilities of our flagship product.

So, let’s see what improvements from the ongoing development of ZFS Open-E JovianDSS has to offer in its update releases.

Open-E JovianDSS Up29 + Up29r1

Sequential Data Scrubbing Algorithm: This algorithm speeds up the data scrubbing, which is essential for maintaining data integrity. Previously, scrubbing large datasets could take months, but with a two-step algorithm, the process only takes a few days. The first step scans metadata to create a list of data blocks, which are then sorted by size and offset, reflecting their physical disk location. This enables sequential data access during error checking.

Sequential Data Resilvering Algorithm: A similar sequential approach has been implemented for data resilvering triggered during disk failures, removals, or temporary unavailability. Unlike the previous method using random I/O operations, this algorithm scans metadata to identify the data blocks needing restoration, sorts them by size and offset, and creates a sequential queue for rebuilding the new disk. This approach significantly enhances resilvering efficiency.

NVMe Disk Hot-Plugging: This enables adding or replacing NVMe drives without shutting down or rebooting the system. This is particularly beneficial in high-availability environments, allowing drive replacement without system interruption.

ZFS Update: Upgraded to version 2.0.0-1, bringing various bug fixes and performance enhancements.

For more information about the Open-E JovianDSS Up29 and Up29r1, check the below articles:

Open-E JovianDSS Up29r2

RDMA Protocol Support: Introduced RDMA (Remote Direct Memory Access) support for mirroring path connections, especially for Mellanox and ATTO 100GbE network cards. RDMA enhances cluster node performance by using RoCE (RDMA over Converged Ethernet). Open-E JovianDSS employs RDMA with iSER (iSCSI Extensions for RDMA) over RoCE to accelerate data replication, improving overall cluster efficiency.

Support for Self-Encrypting Drives (SED): Adds SED support for single nodes and shared storage clusters. SED encrypts and decrypts data directly on the drive, requiring no user intervention or additional encryption software. Open-E can provide a unique key for SSDs and HDDs with SED support, increasing data security. A compatible disk list is continually updated.

SSD TRIM Support: Support for the TRIM command on SSDs, improved performance, and extended the lifespan of the data storage solutions. Tests by Open-E showed that TRIM ensures stable SSD performance over time, particularly in high-load environments.

NVMe Write Log Mirroring Over Ethernet for High Availability Shared Storage Clusters: Enables the use of high-performance NVMe devices built into cluster nodes in High Availability Shared Storage configurations.

Custom OU Parameter in Active Directory: Allows administrators to change the default location to create group policy objects and assign permissions to users or groups in Active Directory. This provides greater control and simplifies permission management.

ZFS Update: Upgraded to version 2.1.1-1.

For more information about the Open-E JovianDSS Up29r2, check the below article:

Open-E JovianDSS Up29r2

Open-E JovianDSS Up29r3

Advanced Scrub Scheduler: Introduced flexible scheduling for data scrubbing tailored to system load, work hours, or maintenance tasks. It prevents system overloads and performance drops while protecting data from silent corruption.

ZFS Update: Upgraded to version 2.1.5.

For more information about the Open-E JovianDSS Up29r3, check the below article:

Open-E JovianDSS Up29r3

Open-E JovianDSS Up30

Introduction of ZFS Special Devices: Allows administrators to assign specific data types (e.g., metadata or small file blocks) to dedicated high-performance data storage devices like SSDs or NVMe. This “data type tiering” significantly boosts system performance by storing critical data on faster media.

NVMe Partitioning: Provides flexibility by allowing NVMe disk space to be partitioned. For example, one NVMe partition can serve as a read cache, another as a write log, and another as a ZFS special device. This optimization reduces hardware requirements and costs while simplifying architecture and maintenance.

SED Support for Non-Shared Storage Clusters: Extends SED support to non-shared storage cluster configurations. A compatible disk list is continually updated.

Active Directory RID Range Support: Previously, only the autorid method was available for assigning User ID (UID) and Group ID (GID), which is suitable for single-domain environments. Update 30 introduces RID range support, allowing efficient management and allocation of Security Identifier (SID) ranges for users and groups in multi-domain environments with trust relationships.

Active Directory RFC2307 Support: Enables using the RFC2307 standard schema for representing Unix-specific attributes like UID and GID in Active Directory. This ensures seamless integration with Unix-based systems and allows unified infrastructure management for Windows and Unix environments.

For more information about the Open-E JovianDSS Up29r3, check the below article:

Open-E JovianDSS Up30

The Journey of Progress: Open-E JovianDSS Enhancements

As you can see, over the past few years (Open-E JovianDSS Up29 was released in 2022), Open-E have been enriching the system by taking advantage of its open-source nature’s benefits. All this is to provide a product that meets the latest market requirements and the expectations of future buyers and current users and to offer a product that answers the challenges of the data storage market. Without sacrificing the ease of use and intuitiveness of its operation.