Sync always for zvol is the safest option and is set by default. However, it can cause write performance decreases since all operations are written and flushed directly to the persistent storage. In case of using sync=always, it is strongly recommended using mirrored write log devices (very fast random writes devices).
Sync=standard or sync=disabled zvol options provide huge performance improvement but the most recent cached data can be lost in case of a sudden power failure. Use this option only in environments equipped with UPS.
For NFS shares the Synchronous data record is enabled by default. This option causes performance to be worse, but data can be safely written. In order to improve the NFS performance you can use Asynchronous data record but in such case, it is strongly recommended to use UPS.
It is strongly recommended to use Mozilla Firefox browser to navigate the system's GUI. When using other browsers some slight problems with displaying content may occur.
After updating from previous versions, some problems with WebGUI content and navigation may occur. To resolve this problems, please clear Web browser cache.
In case of installing the system as a Hyper-V guest please use the following settings:
In case of installing the system as a VMware ESXi guest please use the following settings:
In case of deleting large amounts of data, reclaimed deleted blocks on thin-provisioned LUNs in Windows 2012 can significantly slow down system performance. If you predict frequent deletions of large amounts of data, we recommend turning off the automatic reclaim function in Windows 2012. This can be done by disabling the "file-delete notification" feature in the system registry. To do so, follow the steps below:
In order to reclaim the free space in Windows 2012 please change the HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\FileSystem\DisableDeleteNotification key value back to 0 and use "Optimize" tool located in Disc Management->[disk]->Properties->Tools. As the operation can generate a very high load in the system, it is recommended to perform it after-hours.
In case of VMware ESXi, the automatic reclaim feature is disabled by default. To reclaim the space of deleted blocks on thin-provisioned LUNs, please use vmkfstools. For details, please refer to the VMware Knowledge Base:
For VMware ESXi 5.0: http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2014849
For VMware ESXi 5.5 and newer: https://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2057513
In case of using Windows 2008 there is no possibility to reclaim the space released by deleted data of thin-provisioned LUNs.
Please be aware that deleting the zvol with deduplication enabled can generate a very high load in the system and lead to unstable behavior. It is strongly recommended to perform such operation only after-hours. To avoid this issue, please use (if possible) single zvol on zpools dedicated for deduplication and delete the zpool which includes the single zvol.
To determine the amount of System RAM required for deduplication, use this formula:
(Size of Zvol / Volume block size) * 320B / 0.5 / 0.25
320B - size of entry in DDT table
0.5 - Percentage RAM reservation for ARC (50%)
0.25 - Percentage DDT reservation in ARC (25%)
Example for 1TB data and 64KB Volume block size:
(1099511627776B / 65536B) * 320B / 0.5 / 0.25 = 42949672960B
42949672960B / 1024 / 1024 / 1024 = 40GB
so for every extra 1TB of storage, system needs extra 40GB RAM
Example for 1TB data and 128KB Volume block size:
(1099511627776B / 131072B) * 320B / 0.5 / 0.25 = 21474836480B
21474836480B / 1024 / 1024 / 1024 = 20GB
so for every extra 1TB of storage, system needs extra 20GB RAM.
Example for 1TB data and 1MB Volume block size:
(1099511627776B / 1048576B) * 320B / 0.75 / 0.25 = 1789569706,66B
1789569706,66B / 1024 / 1024 / 1024 = 1.66GB
so for every extra 1TB of storage, system needs extra 1.66GB RAM.
IMPORTANT: The above calculations only apply to the worst case scenario, when data is completely unique and will not be deduplicated. For the deduplicable data, the need for RAM drastically decreases. If SSD based Read Cache is present, part of deduplication table will be moved to the SSD and deduplication will work with good performance using less RAM.
IMPORTANT: With SAN (iSCSI) it is CRITICAL to match User-File-System format block size with the zvol volume-block-size. A simple example is a Windows file system NTFS with default format block size 4k and zvol default volume-block-size is 128k. With defaults like this deduplication will mostly NOT match because files can be aligned in 32 (128/4) different positions on the pool. If the NTFS format is increased to 64k and the zvol default volume-block-size is 128k, deduplication match can fail only one time because a file can be aligned to 2 (128/64) different positions on the pool. Every next write will match already as both alignment options already exist on the pool. In order to achieve all files matching and efficient memory usage NTFS must use 64k format block size and the zvol volume-block-size must equal 64k. Another option is that the NTFS=32k and zvol=32k, but in this case the deduplication table will be twice as large. That is why the NTFS=64k and zvol=64k is the most efficient setting for deduplication.
IMPORTANT: With NAS (NFS, SMB/CIFs) deduplication matching works always due to the data blocks being aligned by ZFS natively.
IMPORTANT: De-duplication is working on the pool level in the pool-range. This is why zvol-Physical size cannot show de-duplication benefit. In order to prove that deduplication saved space run the scrub and notice the current physical data space on the pool reported by the scrub. Next copy of new data and run the scrub again. Now scrub will show new physical data space. Comparing the data size from storage client side with the data space growth from the scrub will give the deduplication advantage. The exact pool of the deduplication ratio can be found in LOGs in zfs.log.
It is strongly recommended to set the client file system block size same as the zvol volume block size. For example, when using 64k zvol volume block size, the Windows Allocation unit size of NTFS should be set to 64k.
In case of more than 60 targets, GUI will not be displayed correctly.
This issue will be fixed in the next releases.
There is no possibility to install the system on disks containing LVM metadata. You will need to clear those disks before installation. To do so, use the "Remove ZFS data structures and disks partitions" function located in the Extended tools. To access this function, boot the system from a temporary media like a USB drive or DVD.
There is no option to import Zpool with a broken write log disk using the system's functions. This is why it is STRONGLY recommended to use mirrored disks for write logs. In case it is necessary to import Zpool with a broken write log, please contact technical support.
In case of replacing damaged disks for larger ones, the size of the entire Zpool will increased. Make sure that the new size will not exceed your purchased storage license.
After performing some operations, e.g. resilvering, the GUI will show outdated information. In this case refresh the web page manually by pressing F5 on your keyboard. This issue will be fixed in next releases.
Operation of replacing a disk in a data group for a smaller one will cause an error "zpool unknown error, exit code 255", and the disk will become unavailable. In order to reuse this disk, please use function "Remove ZFS data structures and disks partitions" located in the Extended tools on the Console screen.
Smaller than 64KB block sizes used with deduplication or read cache will cause very high memory consumption.
To determine how much System RAM is required for Read Cache, use the following formula:
RAM needed = (Size of Read Cache - reserved size and labels) * bytes reserved by l2hdr structure / Volume block size
For 8KB Volume block size and 1TB Read Cache:
RAM needed = (1099511627776B - 4718592B) * 432B / 8192B = 57981809664B
57981809664B / 1024 / 1024 / 1024 = 54GB
1099511627776B - 1TB Read Cache
4718592B - reserved size and labels
432B - bytes reserved by l2hdr structure
8192B - Volume block size
For 64KB Volume block size and 1TB Read Cache:
RAM needed = (1099511627776B - 4718592B) * 432B / 65536B = 7247726208B
7247726208B / 1024 / 1024 /1024 = 6.75GB
For 128KB Volume block size and 1TB Read Cache:
RAM needed = (1099511627776B - 4718592B) * 432B / 131072B = 3623863104B
3623863104B / 1024 / 1024 /1024 = 3.37GB
Multiple operations of adding and detaching disks from groups can cause that the next operation of detaching will fail, but the disk will be shown on a list of available disks. When trying to add this disk to one group it will fail with the following error "[zfslib-wrap-zpool-ZpoolCmdError-1] invalid vdev specification". In this case, detach this disk once again.
Sometimes after removing disks from groups, Spare/Read Cache/Write Log disks are displayed on a list of unassigned disks, but they are not on a list of available disks. In this case, click the rescan button located in the adding group form.
After deleting an exported Zpool, not all disks which were a part of a Zpool become immediately available. Before you can reuse disks, which were previously used as a Spare or a Read Cache, you must first clean them. Use "Remove ZFS data structures and disks partitions" function located in the "Extended tools".
For some network interfaces, the negotiated speed field may display an incorrect value in GUI and Console. This issue will be fixed in next releases.
After creating multiple snapshots, clones or zvols some forms in GUI work very slow. If you need to create many snapshots, clones or zvols, it is strongly recommended to use CLI in order to perform operations on them.
Open-E VSS Hardware Provider does not work. This problem will be resolved in the future releases.
Files located on datasets with exceeded quota cannot be removed. In this case, please resize quota and then remove unnecessary files.
Zpool with more than 20 datagroups causes that some forms on WebGUI work very slow. If you need to create many datagroups, it is strongly recommended to use CLI API.
More than 25 datasets cause that WebGUI works slow.
For Open-E JovianDSS users, it is recommended to upgrade Zpools to the latest ZFS file system. Although the file system upgrade is absolutely safe for your data, and takes only few minutes, please be aware that this operation cannot be undone. In order to upgrade a single Zpool, please use "WebGUI -> Zpool options -> Upgrade file system" from Zpool's option menu.
In case of using Open-E JovianDSS with Intel® Ethernet Controller XL710 Family, it is necessary to update firmware's network controller to the version: f4.33.31377 a1.2 n4.42 e1932.
In case of using a motherboard with x2APIC technology enabled, it is necessary to disable x2APIC in BIOS. Otherwise, problems with CPU cores will occur.
One of the factors that have been taken into account when NFS FSIDs are generated is Zpool name. It indicates that when Zpool name is changed, e.g. during export and import with different names, FSIDs for NFS Shares located on this Zpool will also be changed.
Due to technical reasons the High Availability shared storage cluster does not work properly when using the Infiniband controllers for VIP interface configuration. This limitation will be removed in the future releases.
Starting from up10, there is no possibility to configure static routing in TUI. In case the static routing was configured in previous versions, this configuration will be removed from the system.
Attempt to create Zpool with drives that contain LVM data will fail with the following error:
"cannot open 'lvm-pv-uuid-R25lTS-kcDc-eiAN-eAlf-ppgi-rAqu-Oxy1Si': no such device in /dev must be a full path or shorthand device name"
In this case, if you want use those disks, please use "Remove ZFS data structures and disks partitions" function located in "Extended tools".
In some environments, under heavy load a problem with too long time of cluster resources switching may occur. This situation can especially occur after disconnecting the SAS cables from one of the nodes. If the switching time exceeds the iSCSI initiator timeout, the timeout may be increased up to 600 seconds.
In case of using Windows, to increase iSCSI initiator timeout, please perform following steps:
1. Run regedit tool and find: HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Class\...\Parameters\MaxRequestHoldTime registry key
2. Change value of the key from default 60 sec to 600 sec (decimal)
In case of using VMware, to increase iSCSI initiator timeout, please perform following steps:
1. Select the host in the vSphere Web Client navigator
2. Go to Settings in the Manage tab
3. Under System, select Advanced System Settings
4. Choose the Misc.APDTimeout attribute and click the Edit icon
5. Change value from default 140 to 600 sec
In some environments, after update to up11 system may require re-activation. This issue will be removed in the future releases
In case of using JovianDSS as Hyper-V or VMware guest, bonding ALB is not supported. Please use another type of bonding
Using ODPS on zvol/dataset with VMware guest where many I/O operations are performed can cause that the process of deleting VMware snapshot can take long time. Please take this into consideration while you set up the scheduler for Off-site Data Protection Service task
Enabling quota functionality on a dataset can cause file transfer interrupt. Before using it in production environment, please enable quota on dataset, or make sure that no file transfers are active
If JovianDSS nodes are connected to the same AD server, they cannot have the same Server names
In case of share with the same name as Pool connections problem will occur. Please use different names
Changing settings of virtual network cards (delete, changing MAC, etc.) can cause unstable system behaviour. Please do not change settings on production system. This issue will be fixed in next releases
Starting from up18 bootable medium has always SW RAID structure. Attempt to come back to earlier version is impossible. If you need come back to earlier version, you must reinstall version again.
This issue will be fixed in next releases.
Performing operation like: reboot, shutdown, power off, etc. during mirroring data on new added disk can cause that system will be booted from new disk which has incomplete data. In this case, SW RAID function shows empty status and wrong number of RAID members. To resolve this issue, please plug off disk which has incomplete data, boot system, plug in disk and add it once again to SW RAID.
It is strongly not recommended to use Cluster over Ethernet with SAS-MPIO functionality. Such a configuration can lead to a very unstable cluster behavior.
Microsemi Adaptec RAID is not supported as HBA in cluster environment.
In case of using On- & Off-site Data Protection functionality in up21 or earlier, it is strongly recommended to remove all backup tasks created by CLI API and re-create it using GUI.
In FC Target HA environment, power cycle of both nodes simultaneously may lead to a situation when VMware is not able to restore proper state of the storage devices. In vSphere GUI LUNs are displayed as Error, Unknown or Normal,Degraded. Moving affected pools to another node and back to its native node should bring LUNs back to normal. Afterwords refresh vSphere’s Adapters and Devices tab.
In case of disk failure, please remove the damaged disks from the system, before starting administrative work to replace the disk. The order of actions is important
In HA cluster environment after updating of one node from JovianDSS up24 to JovianDSS up25 the other node can fall into separated mode and the mirror path might indicate disconnected status. In such a case go to Failover Settings and in the Failover status section select Stop Failover on both nodes. Once this operation is finished select Start Failover.
In the early beta versions of JovianDSS up25 the default value of the Write Cache Log bias of zvols was set to “In Pool (Throughput)”. In the final release of Jovian up25 the Log bias is set to “Write log device (Latency)”. Please note, that “In Pool (Throughput)” setting may cause a drop in performance in environments with a lot of random access workloads which is a common factor for a majority of production environments.
In order to configure FC Target, kernel module parameter qlini_mode should be set to “exclusive” (in some early beta versions of Jovian up25 qlini_mode was set up to “enabled”). In order to verify the value of this parameter open Jovian TUI and use CTRL+ALT+W key combination to launch Hardware configuration. Press "Yes" to acknowledge the initial warning message. Type in the password. Choose option: Kernel module parameters. Select qla2xxx_scst QLogic Fibre Channel HBA Driver and make sure the value of this parameter is set to “exclusive”.
Please note that in order to change this parameter Failover must be stopped first.