0 Liked

    Criteria for selecting files when performing full, incremental and differential backup

    A fundamental function of backup and restore software is the ability to copy data from one place to another. No matter if the backup data are stored on volumes, files, images or on magnetic tapes – we assume that data are being copied from location A to location B. In case of data loss in location A, a backup program should change the backup location and copy the data from location B back to location A. This is the simplest description of backup and restore. Shortly speaking, a backup program is a data copying program.

    Just like all software for data copying, a backup program has to know what and where to copy. In this post, I would like to take a closer look at the first option. Precisely speaking, how a backup program knows what to copy.

     When considering which files to back up, I would like to base it on three main types of backup: a full backup, incremental backup, and a differential backup.

    Introduction to CTIME and MTIME timestamps

    In UNIX systems, the selection of files for backup usually takes place based on CTIME (change time) and MTIME (modification time) timestamps. Those terms refer alternately to: date and time of the last modification of a file or its attributes (CTIME) and the date and time of the last modification of the content of this file (MTIME). Changing the content of a file means yielding a new CTIME and MTIME. By changing only attributes of a file (e.g. its privileges), only the CTIME is changed.

    All in all, changing the content of a file results in the change of CTIME and MTIME, while changing attributes in a file results in the change of CTIME. For the more curious, I recommend doing a research on changes of CTIME and MTIME when changing the content of a file and its attributes. You can use e.g. a UNIX stat command to read those timestamps.

     

    The role of CTIME and MTIME during backup operations

    Backup software does not need any information about CTIME and MTIME to determine which files are to be copied to a full backup. It just copies all files from the location assigned to it.

    In an incremental backup, only the files that have changed since the time of making a full, incremental or differential backup are copied. In this case, the algorithm for file selection looks more or less like this:

    • start  XYZ backup
    • download the CTIME and MTIME for a file
    • check whether the CTIME or MTIME is newer than the time of making the last copy of any type of XYZ backup
    • if yes, include the file in a copy; if not, omit it
    • points from 2 to 4 are repeatable for each processed file

    In case of a differential copy, only the files that have changed since the time of making a full backup are being copied. The algorithm for selecting files for a differential copy is presented below:

    • start XYZ backup
    • take the CTIME and MTIME for a file
    • check whether the CTIME or MTIME is newer than the time of making the last full copy of XYZ backup
    • if yes, include the file in a copy; if not, omit it

     

    Archive bit

    Selecting a file for backup seems a bit different in Windows systems. Often the so-called archive bit is used. It is a property of a file that works in a zero-one system. If archive bit is set to a file, then this file will be taken into consideration in the backup. In contrast, if archive bit is not set the file will be omitted.

    A full backup made by software that uses archive bit proceeds similarly to UNIX systems – all files are copied from the specified locations. For each file that is contained in a full backup, an archive bit is reset.

    For an incremental backup, only those files are copied which have a set archive bit. After making an incremental backup, an archive bit is reset for each file that is provided for a copy.

    When it comes to differential backup – similarly to incremental – only those files are copied where archive bit is set. After saving a file in a backup, archive bit is not reset.

    This way of backing up,  based on archive bit, enables you to achieve the effect of a full backup (all files), incremental backup (files that were changed since the time of the last copy of any type) and differential backup (files that were changed since the time of making a full backup) – same as in UNIX systems.

     

    Summary

    In the end, I would like to underline that CTIME, MTIME and Archive bit are properties of file systems. Backup programs use these properties to determine which files to backup. For this reason I think it’s good to know the file systems where backup files are placed.

    In the above article, I presented only a base overview of the available options used for backup. Of course, there are many more criteria for selecting backup files – depending on your file system and your backup program capabilities.

    Rating: / 5.

    No votes yet

    1 Comment

    • Deandra

      February 09, 02 2016 06:58:55

      That’s a clever answer to a tricky quoisten

      VA:F [1.9.22_1171]
      Rating: 0 (from 0 votes)

    Leave a Reply