{"id":51217,"date":"2021-09-02T10:42:00","date_gmt":"2021-09-02T08:42:00","guid":{"rendered":"https:\/\/www.open-e.com\/blog\/?p=51217"},"modified":"2025-04-14T11:19:20","modified_gmt":"2025-04-14T11:19:20","slug":"atomicity-copy-on-write-and-data-integrity-a-primer","status":"publish","type":"post","link":"https:\/\/www.open-e.com\/blog\/atomicity-copy-on-write-and-data-integrity-a-primer\/","title":{"rendered":"Atomicity, Copy-on-Write and Data Integrity: A Primer\u00a0"},"content":{"rendered":"

\t\t\t\tWere you ever in the middle of saving some changes when the power went out? Did you lose both the changes and the original version? This typically happens when the data is being saved via a read-modify-write (RMW) operation and something goes wrong. Before getting into why this happens, let\u2019s quickly review what a read-modify-write operation actually is.<\/span><\/p>\n

Read-Modify-Write?<\/span><\/h2>\n

A read-modify-write operation typically reads the data, modifies it by making the requested changes and then writes over the original, applying said changes. This operation is great as it allows changes to be made to all sorts of data, big or small. That being said, it\u2019s not without its <\/span>downsides<\/span>.\u00a0<\/span><\/p>\n

The aforementioned major downside is the simple fact that if there is ever an interruption while changes are ongoing for any reason then all the new data as well as the original data could be corrupted or lost.\u00a0<\/span><\/p>\n

Open-E JovianDSS<\/a> has several features in place that prevent data loss like this from occurring. These features include things like atomic transactions and advanced ways of performing write operations, which we\u2019ll be discussing in this article.<\/span><\/p>\n

Solving the RMW Problem: Atomic Transactions<\/span><\/h2>\n

Atomic transactions<\/a> are a relatively straightforward way that Open-E JovianDSS helps ensure your company\u2019s data integrity. Basically the way atomic transactions work is that either the transaction completes in full or it doesn\u2019t complete at all, in which case the system would just keep on using the data from its previous state as though no transaction ever took place. Atomicity ensures that the system checks every transaction to make sure that it was fully completed before allowing the changes to take place.<\/span><\/p>\n

This is important as it eliminates a plethora of problems that could otherwise occur if the transaction wasn\u2019t conducted using atomicity. These problems include things like lost and\/or uncommitted updates, inconsistent analyses and transactions generally preventing other transactions from functioning appropriately.\u00a0<\/span><\/p>\n

So how can we ensure that all the transactions are atomic? Well, in our case it\u2019s done by using redirect-on-write, a variant of copy-on-write that both Open-E JovianDSS and ZFS use. Let\u2019s start by explaining what copy-on-write is generally.\u00a0<\/span><\/p>\n

Solving the RMW Problem: Copy-on-Write<\/span><\/h2>\n

So what is advanced write technology like copy-on-write<\/a>, generally speaking? Well traditionally the way copy-on-write worked and still works in a lot of systems is similar to the way read-modify-write works during the first few stages. Both operations read the data but when a request that uses copy-on-write occurs, the system makes a copy of the original data and sends it to a different data block before then proceeding to the modify and write portions of the operation.<\/span><\/p>\n

Therefore, copy-on-write actually reads the data and performs two writes, one to copy the original data before sending it off to be stored to a different block and a second write to change the original to the new version. The writes can also be set to use the aforementioned atomic transaction process to limit data corruption and ensure data integrity if programmed that way.<\/span><\/p>\n

This differentiates it from the original read-modify-write operation where no copy of the original data is made and the original is modified directly at the exact place it\u2019s stored. This is why both the original and new data are lost should a disruption occur while the read-modify-write operation is applying changes to the original copy. The same isn\u2019t true if using copy-on-write due to the original data already being safe and secure in another block before the changes start to take effect. That is copy-on-write in its \u201cpurest\u201d, or some would say, most generally understood, form.\u00a0<\/span><\/p>\n

Examples of where copy-on-write is still used prevalently in its truest form include the Logical Volume Manager (LVM), a volume manager that\u2019s still found in a lot of Linux and Unix systems currently used. It should be noted though that not all LVM versions out there still use the traditional copy-on-write technique as even LVM has started to transition to redirect-on-write.\u00a0\u00a0<\/span><\/p>\n

The Rise of Redirect-on-Write<\/span><\/h2>\n

So at this point you might be thinking, \u201cWell, if that is copy-on-write then what\u2019s this redirect-on-write that both Open-E JovianDSS and ZFS use\u201d?\u00a0<\/span><\/p>\n

Well, redirect-on-write<\/a> happens to be another prominent variant of copy-on-write that has even occasionally eclipsed the most common one. So how do the two differ? Redirect-on-write differs from the original in that instead of making a copy of the original and then sending it to a different block, the original is left untouched in its block. The changes are instead stored in a different block with a pointer system that lets the system know exactly where the changes and the original are so that the correct data can be shown upon request. This is also the original that the system can use should it ever need to due to a failed atomic transaction verification.\u00a0<\/span><\/p>\n

This, in effect, means that the operation is even further streamlined, only requiring one write operation to traditional CoW\u2019s two, and that the original copy is potentially even more secure than with traditional copy-on-write. This is one of the reasons why redirect-on-write is gaining ground on traditional copy-on-write or being used in addition to copy-on-write in hybrid configurations to maximize the gains that combining the technology allows.\u00a0<\/span><\/p>\n

We experienced this ourselves personally at Open-E when we switched from the more traditional CoW method used in Open-E DSS V7<\/a>, which uses a LVM based volume manager, to a more RoW based method in our newest product, Open-E JovianDSS, which is based off of ZFS. That being said, redirect-on-write <\/span>isn\u2019t perfect<\/span> and it does have issues with things like data fragmentation<\/a>. These issues are currently being addressed by the ZFS file system that Open-E JovianDSS is based on. Nevertheless, redirect-on-write does have it\u2019s problems and it really is up to the individual to decide which CoW they\u2019d prefer.\u00a0<\/span><\/p>\n

The Wrap Up<\/span><\/h2>\n

In any case, by combining atomicity and software that boosts the boons provided by copy-on-write whilst minimizing its downsides, Open-E JovianDSS can ensure that your company\u2019s data will be as safe as can be should something unexpected happen while it\u2019s being updated. So what do you think, are atomic transactions the way of the future? Can RoW and CoW be used interchangeably? What\u2019s the most important document you\u2019ve ever lost due to a failed RMW operation? Let us know in the comments below!<\/span>\t\t<\/p>\n","protected":false},"excerpt":{"rendered":"

Were you ever in the middle of saving some changes when the power went out? Did you lose both the changes and the original version? This typically happens when the …<\/p>\n","protected":false},"author":2,"featured_media":55462,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[27,765],"tags":[],"class_list":["post-51217","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-open-e-joviandss","category-open-e-legacy-products"],"acf":[],"aioseo_notices":[],"_links":{"self":[{"href":"https:\/\/www.open-e.com\/blog\/wp-json\/wp\/v2\/posts\/51217","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.open-e.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.open-e.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.open-e.com\/blog\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.open-e.com\/blog\/wp-json\/wp\/v2\/comments?post=51217"}],"version-history":[{"count":1,"href":"https:\/\/www.open-e.com\/blog\/wp-json\/wp\/v2\/posts\/51217\/revisions"}],"predecessor-version":[{"id":55160,"href":"https:\/\/www.open-e.com\/blog\/wp-json\/wp\/v2\/posts\/51217\/revisions\/55160"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.open-e.com\/blog\/wp-json\/wp\/v2\/media\/55462"}],"wp:attachment":[{"href":"https:\/\/www.open-e.com\/blog\/wp-json\/wp\/v2\/media?parent=51217"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.open-e.com\/blog\/wp-json\/wp\/v2\/categories?post=51217"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.open-e.com\/blog\/wp-json\/wp\/v2\/tags?post=51217"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}