As we’ve already discussed in a previous article, entitled “How Do ZFS Snapshots Really Work”, Open-E JovianDSS has the ability to create snapshots and can also back them up on another storage server. Moreover, there’s a possibility to set up a retention plan as well, which would allow you to automatically create and remove snapshots on-site. This retention plan feature also allows you to set the off-site backup process (an inseparable part of On- and Off-site data protection) according to a particular schedule.
There are several questions that people tend to have regarding retention plans. Questions like, how does a retention plan work? How exactly does it add value to the data storage process? What do these retention rules actually even mean? In this article, our Open-E Team will discuss these questions as well as things like:
Retention plan intervals
If you’re not familiar with the topic of snapshots, we’d recommend starting with our previous article, “How Do ZFS Snapshots Really Work?”, and coming back to this one afterwards.
Retention Plan Intervals
Intervals are the first element of our retention rules’ formula and a requirement for each retention plan you set. They determine a snapshots’ frequency, i.e. how often a snapshot of the state of the system should be taken and then sent to a pre-selected destination. In the configurations menu of Open-E JovianDSS, you can set a minimal interval of once per every minute. You should decide which value to set according to your business needs, taking into account the importance of the data, how often it’s updated, the amount of the updated data buildup over time, and how much of it you can afford to lose in case of a disaster (i.e. is losing a couple of hours of data alright or not?). Multiple rules with different interval values can also be configured for a better fit to your specific business needs. These can be set for both production and backup servers.
When it comes to actually deciding how long you’d like to keep the snapshots on the server, you have plenty of choice. Snapshots can be kept for as little as an hour up to for as long as a couple of years, depending on the server’s size and use (if it’s a production server or an off-site backup). Moreover, the retention period can also be configured, just as with intervals, for both, the production (on-site) and backup (off-site) servers individually. For instance, if you’d like to keep the data on the local server for only 30 minutes but want it on the off-site server for 3 years, this is completely doable. What would happen in this particular case is that, when the 30 minute retention period elapses on the local server, then the local snapshots are erased, but not before they are transferred over to the long-term off-site backup server (assuming the appropriate settings have been set). This means that you’ll still have a chance to go back to your previous versions should you want to due to them being on the off-site backup.
The retention plan should also be configured according to the backup requirements of each enterprise. As previously stated, the retention period can allow you to keep the snapshot data for up to several years before it automatically gets removed from the server to make space for newer snapshots. This feature makes full customization of the data flow quite a real option. For example, if we’re trying to resist ransomware attacks, we could set the plan to create snapshots more often and be kept for shorter periods of time (short retention intervals/ short retention periods). If we’d like to mostly use the feature to avoid natural disasters, thefts, or human errors however, we can create rule sets with longer retention intervals (anywhere from once per hour to once per day) and retention periods (upwards of one week or more) instead. What’s exceptional about this feature is that all these rules can be combined together in order to make one advanced plan. Just as in the illustration below shows:
Make sure to keep in mind however that the first snapshot sent to the backup server has to be sent with all the system data (it will take up the same capacity as the initial data). That being said, it’s worth it in the long run, as all the following snapshots will just add updates to the first one. Thanks to that, the snapshots can be asynchronously replicated to local and/or remote locations according to the set retention plan. This lets you set more frequent intervals with a shorter retention period on the on-site server and rarer intervals with long retention periods on the off-site server.
Lastly, do keep in mind that it’s essential to always evaluate the data workflow of your enterprise as well as your Business Continuity plan before setting up your retention plan for the Open-E JovianDSS software solution.