fsckpt_intro

NAME

fsckpt_intro - introduction to the VERITAS File System Checkpointing API

AVAILABILITY

VRTSvxfs

DESCRIPTION

A file system Storage Checkpoint is a snapshot of all user files in a file system at a moment in time. Checkpoints are objects that are managed and controlled by the file system and therefore are persistent through reboots and system crashes. As with most objects, Storage Checkpoints have a name associated with them and can be managed as independent entities, that is, Storage Checkpoints can be created, removed, renamed, and so forth.

One of the most important properties of a Storage Checkpoint is that they can be created very quickly. This is done by creating a Storage Checkpoint object which consists of all user files mapping the original file system with the exception that their data is shared, not copied. A unique relationship is therefore established between the Storage Checkpoint object and the file system. The image of a Storage Checkpoint is preserved by using copy-on-write semantics. After a Storage Checkpoint is taken, files can continue to be created, removed or updated without affecting the logical image of the Storage Checkpoint. A Storage Checkpoint therefore, preserves not only the name space (directory hierarchy) of the file system, but also the user data as it was at the moment of its creation.

The file system in this context is also known as the primary image or fileset. In fact, the primary fileset does not need to be structurally much different than a checkpoint fileset, the exception being that the primary fileset will always contain a full copy of its image.

Since Storage Checkpoint filesets are updated on demand by using copy-on-write semantics, a map of all changed blocks is maintained by the Storage Checkpoint object. These changed blocks can be extremely valuable for backup applications and replication solutions since only the changed data may be retrieved. Access to this information may provide incredible advantages to these solutions by minimizing data movement many times. This information may additionally lead to higher availability and data integrity since the frequency of these backup/replication solutions may increase many times as well.

The VERITAS File System Checkpointing API allows applications to completely manage all Storage Checkpoints in the file system. Additionally, the API provides all the necessary functions to view a Storage Checkpointed name space and retrieve all blocks changed. Finally, a set of functions is provided to gather information and statistics from Storage Checkpointed filesets and files to properly manage and tune these file systems.

FILE SYSTEM SPACE CONSIDERATIONS

Since Storage Checkpoints are a resource of the file system, they require additional disk space to maintain their image. Administrators should be aware of the demands placed on these file system and the types of operations that require added space due to the presence of Storage Checkpoints. This API provides space counters that can be monitored to best tune and control disk usage.

There are a couple of interesting situations that may arise due to the existence of Storage Checkpoints in file systems where the disk space becomes scarce. These are:

When over-writing a file, the image of the Storage Checkpoint needs to be updated. This means that a disk allocation may be needed. If the file system however, does not have enough disk space to accommodate the Storage Checkpoint update, then we have a problem. Either the application can get the ENOSPC error or the Storage Checkpoint cannot be updated. If the Storage Checkpoint is not updated, then we have lost its image at the point in time when it was created. On the other, some applications may not be prepared to receive the ENOSPC error since they may have gone through the trouble of preallocating the file. Since this is an over-write, applications should not get ENOSPC errors.

A similar situation occurs when a directory update is needed due to a file being created or removed. Before the directory update is allowed, the Storage Checkpoint must be updated. Again, this may result in an allocation which fails due to lack of disk space. This condition may be particularly bothersome since we may be trying to remove a file to free up disk space, yet we cannot because there is not enough disk space to update the Storage Checkpoint.

The File System Checkpointing API does not intend to solve these problems. Instead, it provides the necessary information to tune and monitor the file system and the disk space demands from all Storage Checkpoints in the system. The API also provides a mechanism by which Storage Checkpoints can be disabled, making sure that applications never get ENOSPC errors. See the fsckpt_create() and fsckpt_cntl() man pages for more information.

EXAMPLE 1 FULL BACKUP

The following pseudo code example shows how a consistent full backup may be taken from an online file system. This example may also leave a (nodata) Storage Checkpoint on the system so that a block level incremental may be taken at some point in the future.

The following procedure is used in this example:

Create a Storage Checkpoint named full.

Set the Storage Checkpoint context to full.

Do a file system backup (read name-space and data from full ).

Remove the Storage Checkpoint named full.

"" 5 or

Rename full to incr1 and convert incr1 to nodata (optional)

Use 4 to do a full backup. Use 5 to do a full backup followed by a block incremental backup (BLI).

     full(mountpoint)
         char        *mountpoint;
     {
         void            *chp, *ch[2];
         void            *fhp;
         struct c_extent        ce;
         off_t            off;
         char            *buf;
         int            error;
         if (error = fsckpt_fsopen(mountpoint, FSCKPT_VERSION, &chp)) {
             return error;
         }
         ch[0] = chp;
         ch[1] = NULL;
         fsckpt_create(&ch[0], "full", 0);
         fsckpt_setcontext(chp, "full");
         chdir(mountpoint);
         for (each file) {
             fd = open(file, O_RDONLY);
             fsckpt_fopen(chp, NULL, file, &fhp);
             for (off = 0; off < file.size; off += ce.ce_len) {
                 fsckpt_fbmap(fhp, off, &ce);
                 /*
                  * Skip holes in sparse files
                  */
                 if (ce.ce_flags & CE_HOLE) {
                     continue;
                 }
                 lseek(fd, off, SEEK_SET);
                 buf = malloc(ce.ce_len);
                 read(fd, *buf, ce.ce_len);
                 backup(file, off, ce.ce_len, buf);
                 free(buf);
             }
             fsckpt_fclose(fhp);
             close(fd);
         }
         fsckpt_clearcontext(chp);
         /*
          * If an incremental will be needed then
          */
         if (incremental needed) {
             fsckpt_rename(chp, "full", "incr1");
             fsckpt_cntl(chp, "incr1", CC_SET, CC_NODATA);
         } else {
             fsckpt_remove(chp, "full", 0);
         }
         fsckpt_fclose(chp);
     }

EXAMPLE 2 DIFFERENTIAL AND CUMULATIVE BLIS

The following procedure describes how to use the Storage Checkpoint API for block level incremental backups. A BLI would be done some time after first doing a full backup (as shown in Example 1, step 5 above) when the system is ready for an incremental backup.

Create another Storage Checkpoint named tmp to stabilize the name space.

Set Storage Checkpoint context to tmp

Do BLI backup

"" 10 -Obtain blocks changed from incr1
-Read stable name space and changed data from tmp.

For differential backups:

"" 10 -Remove incr1
-Rename tmp to incr2
-Convert incr2 to nodata (optional)

For cumulative backups:

"" 10 -Remove tmp
-Rename incr1 to incr2

     inc_bi(mountpoint)
         char        *mountpoint;
     {
         void            *chp, *ch[2];
         void            *fhp;
         struct c_extent        ce;
         struct c_info        cinfo;
         off_t            off;
         int            error;
         char            *buf;
         if (error = fsckpt_fsopen(mountpoint, FSCKPT_VERSION, &chp)) {
             return error;
         }
         /*
          * Make sure "incr1" still exists and is valid
          */
         if (error = fsckpt_info(chp, "incr1", 0, &cinfo)) {
             return error;
         }
         ch[0] = chp;
         ch[1] = NULL;
         fsckpt_create(&ch, "tmp", 0);
         fsckpt_setcontext(chp, "tmp");
         chdir(mountpoint);
         for (each changed file) {
             fd = open(file, O_RDONLY);
             fsckpt_fopen(chp, "incr1", file, &fhp);
             for (off = 0; off < file.size; off += ce.ce_len) {
                 fsckpt_fbmap(fhp, off, &ce);
                 /*
                  * If this file section has not changed or is
                  * a hole in a sparse file, then skip
                  */
                 if (!(ce.ce_flags & CE_CHANGED) ||
                     ce.ce_flags & CE_HOLE) {
                     continue;
                 }
                 lseek(fd, off, SEEK_SET);
                 buf = malloc(ce.ce_len);
                 read(fd, *buf, ce.ce_len);
                 backup(file, off, ce.ce_len, buf);
                 free(buf);
             }
             fsckpt_fclose(fhp);
             close(fd);
         }
         fsckpt_clearcontext(chp);
         /*
          * leave Storage Checkpoints setup for next incremental
          *
          * at this point we may want the next block level incremental
          * to be another differential or a cumulative incremental.
          */
         if (cumulative) {
             fsckpt_remove(chp, "tmp", 0);
             fsckpt_rename(chp, "incr1", "incr2");
         } else {
             fsckpt_remove(chp, "incr1", 0);
             fsckpt_rename(chp, "tmp", "incr2");
             fsckpt_cntl(chp, "incr2", CC_SET, CC_NODATA);
         }
         fsckpt_fclose(chp);
     }

EXAMPLE 3 LIST CHECKPOINTS

The following example describes how to use the the API to list all Storage Checkpoints.

     list(mountpoint)
         char        *mountpoint;
     {
         void                    *chp;
         struct c_info           cinfo;
         int                     error, index;
         if (error = fsckpt_fsopen(argv[1], FSCKPT_VERSION, &chp)) {
             return error;
         }
         for (index = 1; ; index++) {
             if (error = fsckpt_info(chp, "primary", index, &cinfo)) {
                 break;
             }
             print_cinfo(&cinfo);
         }
         if (error == ENOENT) {
             error = 0;
         }
         (void) fsckpt_fsclose(chp);
         return error;


^ Return to Top	< Previous \| Next >

Product: File System Manual Pages for Storage Foundation
Manual: Library Functions (3)
VERITAS Software Corporation www.veritas.com