AdvFS provides a number of ways to tune your file system's performance. You can configure your filesets and domains to optimize performance. You can run utilities that defragment, balance, stripe, and migrate your files. You can adjust how AdvFS performs logging, caching, and storage allocation.

This chapter includes the following sections:

• Fileset and File Limits
• File-Domain Limits
• Volume Configuration

Utilities for Tuning

• Defragment Utility
• Choosing to Defragment
• Defragment Example
• Balance Utility
• Choosing to Balance
• Balance Example
• Stripe Utility
• Choosing to Stripe
• Stripe Example
• Choosing Between AdvFS and LSM Striping
• Migrate Utility
• Choosing to Migrate
• Migrate Example

To ensure optimal performance, AdvFS limits the number of active filesets, file domains, and volumes that it supports. The following sections outline AdvFS file-system limits.

While AdvFS allows an unlimited number of filesets per system, only 512 filesets minus the number of active file domains can be mounted simultaneously.

The number of files per fileset is limited to 2³², if not first limited by disk space or by quotas.

The currently tested maximum file size is 128 gigabytes. Larger file sizes are allowed, but have not been tested. Sparse files can be addressed to 16 terabytes.

AdvFS supports a maximum of 100 active file domains per system. A file domain is active when at least one fileset within that domain is mounted. The maximum file-domain size is 128 terabytes.

Each file domain retains its own transaction log. If the file domain has a large number of volumes or includes large numbers of filesets, the log can create a bottleneck because of high activity. See Improving the Transaction Log Performance.

It is important to consider the number of filesets in your domain when setting up your system because each active file domain counts against the mounted-fileset limit. Table 9 shows how the mounted fileset count increases as the number of active file domains increases.

Table 9 Fileset Count

You can add up to 250 volumes to an AdvFS file domain. However, without disk mirroring, it is inadvisable to add more than three volumes. If you lose a volume, the entire domain becomes inaccessible. The risk of losing a volume, and thus losing access to your file domain, increases as the number of volumes increases.

There is a small performance advantage to dividing disks on different SCSI chains. However, if you must purchase additional controllers, this can be an expensive way to provide a minimal performance improvement in data access.

The theoretical maximum AdvFS volume size is 1 terabyte.

AdvFS provides utilities for keeping your file system running as efficiently as possible. These tools improve read/write performance by altering the way files are mapped on the disk. They can be run while the system is on line and are transparent to system users and to applications.

You can improve file read/write performance on your system with utilities that:

• Defragment a file domain.
• Balance a multivolume file domain to even the storage distribution between volumes.
• Stripe files across volumes in a multivolume file domain.
• Migrate files to a different volume within a multivolume file domain.

AdvFS attempts to store file data in contiguous blocks on a disk. This collection of contiguous blocks is called a file extent. If all data in a file is stored in contiguous blocks, that file has one file extent. However, as files grow, contiguous blocks on the disk may not be available to accommodate the new data. The system must spread the file over discontiguous blocks. As a result, the file is fragmented on the disk and consists of multiple file extents. File fragmentation degrades the read/write performance because many disk addresses must be examined to access a file.

Figure 7 Defragment File Domain

The defragment utility reduces the amount of file fragmentation in a file domain by attempting to make the files more contiguous so that the number of file extents is reduced. Defragmentation is an iterative, two-step process that operates on the file domain, as follows:

1. Files are moved out of a region to create an area with contiguous, unallocated space.
2. Fragmented files are written into a region that has more contiguous space so they are less fragmented.

In addition to making files contiguous so that the number of file extents is reduced, defragmenting a file domain often makes the free space on a disk more contiguous so files that are created later will also be less fragmented.

You can improve the efficiency of the defragment process by deleting any unneeded files in the file domain before running the defragment utility.

The following restrictions apply to running the defragment utility:

1. You must have root user privileges to access the defragment command.
2. All filesets in the file domain must be mounted. If you try to defragment an active file domain that includes unmounted filesets, you will get an error message.
3. A minimum free space amount of 1% of the total space or 5 megabytes per volume (whichever is less) must exist in order to run.
4. The defragment utility cannot be run while the addvol, rmvol, balance, or rmfset command is running in the same file domain.
5. Striped files are not defragmented.

To determine the amount of file fragmentation that exists in a file domain, use the defragment command with the -v and -n flags. This will show how fragmented the file domain is without starting the defragment utility.

Table 10 describes the information displayed by the defragment utility when you run the command with the verbose (-v) flag.

Table 10 Defragment Utility Output

You can also use the showfile command to check the number of file extents of individual files. The following example shows the attributes and extent information for the mail file:

# showfile -x mail
       Id Vol PgSz Pages XtntType Segs SegSz Log Perf  File

4198.800d   2   16    27   simple   **    ** off  66%  tutorial

        extentMap: 1

            pageOff    pageCnt    vol    volBlock    blockCnt

                  0          5      2      781552          80

                  5         12      2      785776         192

                 17         10      2      786800         160

               extentCnt: 3

The following example defragments the accounts_domain file domain. A time limit of 15 minutes is imposed. Verbose mode is requested to display the fragmentation data at the beginning of each pass made through the file domain and at the end of the defragmentation process.

# defragment -v -t 15 accounts_domain

defragment:  Defragmenting domain 'accounts_domain'



Pass 1; Clearing

  Volume 1: area at block    11680 (  103072 blocks): 81% full

  Domain data as of the start of this pass:

    Extents:                  10432

    Files w/extents:           4305

    Avg exts per file w/exts:  2.42

    Aggregate I/O perf:         52%

    Free space fragments:      2743

                    <100K    <1M    <10M    >10M

     Free space:     38%      0%      0%     62%

     Fragments:     2742       0       0       1

Filling

.

.

.

Pass 13; Clearing

  Volume 1: area at block   559744 (   62736 blocks):  0% full

  Volume 2: area at block    76640 (   24624 blocks): 18% full

  Domain data as of the start of this pass:

    Extents:                   4306

    Files w/extents:           4305

    Avg exts per file w/exts:  1.00

    Aggregate I/O perf:        100%

    Free space fragments:        23

                    <100K    <1M    <10M    >10M

     Free space:      0%      9%     27%     64%

     Fragments:        6      10       5       2

Filling

Current domain data:

   Extents:                   4305

   Files w/extents:           4305

   Avg exts per file w/exts:  1.00

   Aggregate I/O perf:        100%

   Free space fragments:        17

                   <100K    <1M    <10M    >10M

    Free space:      0%      6%     29%     65%

    Fragments:        3       8       4       2



Defragment: Defragmented domain 'accounts_domain'

Information displayed before each pass and at the conclusion of the defragmentation process indicates the amount of improvement made to the file domain. A decrease in the Extents and Avg exts per file w/extents values indicates a reduction in file fragmentation. An increase in the Aggregate I/O perf value indicates improvement in the overall efficiency of file-extent allocation.

The balance utility distributes the percentage of used space evenly between volumes in a multivolume file domain. This improves performance and evens the distribution of future file allocations.

Figure 8 Balance File Domain

Files are moved from one volume to another until the percentage of used space on each volume in the domain is as equal as possible. (Because the balance utility does not generally split files, file domains with very large files may not balance as evenly as file domains with smaller files.)

The following restrictions apply to running the balance utility:

1. You must have root user privileges to access the balance command.
2. All filesets in the file domain must be mounted. If you try to balance an active file domain that includes unmounted filesets, you will get an error message.
3. A minimum free space amount of 1% of the total space or 5 megabytes per volume (whichever is less) must exist in order to run.
4. The balance utility cannot run while the addvol, rmvol, defragment, or rmfset command is running in the same file domain.

To determine if you need to balance your files across volumes, use the showfdmn command to display file-domain information. From the % used field you can determine if the files are evenly distributed.

In the following example, the usr_domain file domain is not balanced. Volume 1 has 85% used space while volume 2 has 0% used space (it has just been added).

# showfdmn usr_domain

               Id      Date Created         LogPgs  Domain Name

2dcab512.000d1ed0  Fri May  6 14:22:26 1994    512  usr_domain



  Vol   512-Blks    Free  % Used  Cmode  Rblks  Wblks  Vol Name 

   1L     819200  126848     85%     on    128    128 /dev/rz8g

   2      768281  768080      0%     on    128    128 /dev/rz9d

      ----------  -------  ------

         1587481  894928     44%

Use the balance utility to even file distribution after you have added a volume with the addvol command or removed a volume with the rmvol command (if there are multiple volumes remaining).

The following example balances the multivolume domain usr_domain, examined above:

# balance usr_domain

 balance: Balancing domain 'usr_domain' 

 balance: Balanced domain 'usr_domain' 


# showfdmn usr_domain

               Id      Date Created         LogPgs  Domain Name

2dcab512.000d1ed0  Fri May  6 14:22:26 1994    512  usr_domain



  Vol   512-Blks    Free  % Used  Cmode  Rblks  Wblks  Vol Name 

   1L     819200  459248     44%     on    128    128 /dev/rz8g

   2      768281  433632     44%     on    128    128 /dev/rz9d

      ----------  -------  ------

         1587481  892880     44%

The balance utility moved files from volume 1 to volume 2 in order to even the percentage of used space between the two volumes.

Striping increases sequential read/write performance by allocating storage in segments across more than one volume. AdvFS allows you to stripe individual files according to your needs. For example, if you stripe files that have heavy input/output requirements, the I/O is spread across the volumes, so that read/write requests to the different disk drives can be overlapped. The AdvFS striping utility does not require you to stripe all your files.

With the stripe utility, you direct a file to distribute segments across specific volumes within a file domain. You first create a new zero-length file, stripe it, then write your data to the striped file and delete the original file. You can choose the number of volumes on which to stripe a file.

Figure 9 Stripe Files

As the file is appended, AdvFS determines the number of pages per stripe segment; the segments alternate among the disks in a sequential pattern. For example, the file system allocates the first segment of a two-disk striped file on the first disk and the next segment on the second disk. This completes one sequence, or stripe. The next stripe starts on the first disk, and so on.

You cannot use the stripe utility to modify the number of disks that an already striped file crosses or to restripe a file that is already striped. To change the configuration of a striped file, you must repeat the striping process.

Before you use the stripe utility, run the iostat utility to determine if disk I/O is the bottleneck operation. The blocks per second and transactions per second should be cross checked with the drive's sustained transfer rate. If the disk access is slow, then striping will improve performance. See Checking Disk Activity.

The following example creates an empty file, stripes it, copies data into the striped file, then shows the extents of the striped file:

1. Create the empty file file_1 and stripe it across three volumes in a domain:

   # touch file_1
   
   # ls -l file_1
   
   -rw-r--r-- 1 root system 0 Aug 1 05:50 file_1
   
   # stripe -n 3 file_1

2. Copy the data from the original file to the striped file:

   # cp orig_file_1 file_1

3. Examine the extents of the new striped file:

   # showfile -x file_1
   
   Id            Vol PgSz Pages XtntType  Segs  SegSz   Log   Perf   File
   
   1ed.8053        1   16    115  stripe    3       8    off   75%   file_1
   
     extentMap: 1
   
        pageOff   pageCnt   volIndex  volBlock   blockCnt
   
              0         8          2       160        384
   
             24         8
   
             48         8
   
        extentCnt: 1
   
      extentMap: 2
   
        pageOff   pageCnt   volIndex   volBlock   blockCnt
   
              8         8          3        128        336
   
             32         8
   
             56         5
   
        extentCnt: 1
   
     extentMap: 3
   
        pageOff   pageCnt   volIndex   volBlock  blockCnt
   
             16         8          1       8528       256
   
             40         8
   
        extentCnt: 1
   
      extentCnt: 1

AdvFS implements file striping at an individual file level. Therefore, if only one or a few files are large and have heavy I/O requirements, use the AdvFS stripe utility to stripe these files.The Logical Storage Manager (LSM) provides volume-level striping. With LSM you preconfigure your volumes for file striping. Then, all files located on the volumes configured for striping will be striped.

The migrate utility improves file I/O performance by altering the way files are mapped on the disk. You can use the migrate utility to move heavily accessed or large files to a different volume in the file domain. You can specify the volume where a file is to be moved or allow the system to pick the best space in the file domain. You can migrate either an entire file or specific pages to a different volume.

Figure 10 Migrate Files

A file that is migrated will be defragmented in the process if possible. This means that you can use the migrate command to defragment selected files.

The following restrictions apply to the migrate utility:

• You must have root-user privilege to access this utility.
• You can only perform one migrate operation at a time on the same file.
• When you migrate a striped file, you can only migrate one source volume at a time.
• The migrate utility does not evaluate your migration decisions. For example, you can move more than one striped file segment to the same disk, which defeats the purpose of striping the file.

If you suspect that a fileset or file domain is straining system resources, run the iostat utility. If the filesets or file domains are located on devices that appear to be a bottleneck, you can migrate files or pages of files to equalize the load. If a high-performance device is available, you can move an I/O-intensive fileset to the more efficient volume. See Checking Disk Activity.

Choose the migrate utility over the balance utility when you want to control the files that are moved. The balance utility moves files only to optimize distribution. For example, it might move many small files when moving a single larger one would be a better solution for your system.

You can use the showfile command with the -x flag to check the performance percentage and extent map of a file. A low performance percentage (less than 80%) indicates that the file is fragmented on the disk. The extent map shows whether the entire file or a portion of the file is fragmented.

The following example displays the extent map of a file called src. The file, which resides in a two-volume file domain, has an 18% performance efficiency. The showfile command with the -x flag lists the extent map for the file src:

# showfile -x src

       Id Vol PgSz Pages XtntType  Segs  SegSz  Log  Perf  File
2e1b.8004   1   16    11   simple    **     **  off   18%  src
    extentMap: 1
        pageOff    pageCnt     vol    volBlock    blockCnt
              0          1       1      684704          16
              1          1       1      684896          16
              2          1       1      684928          16
              3          1       1      685280          16
              4          1       1      685312          16
              5          1       1      685488          16
              6          1       1      686432          16
              7          1       1      686608          16
              8          1       1      686784          16
              9          1       1      686960          16
             10          1       1      687168          16
        extentCnt: 11

The file src consists of 11 file extents. This file would be a good candidate to move to another volume to reduce the number of file extents.

The following example migrates the file src examined above. The number of file extents is decreased:

# migrate -d 2 src

# showfile -x src

       Id Vol PgSz Pages XtntType Segs SegSz  Log  Perf  File
2e1b.8004   1   16    11   simple   **    **  off  100%  src
    extentMap: 1
            pageOff    pageCnt     vol    volBlock    blockCnt
                  0         11       2        1280         176
            extentCnt: 1

The file src now resides on volume 2, consists of one file extent, and has a 100% performance efficiency. Note that in the output above, the first data line of the display lists the metadata. The metadata does not migrate to the new volume. It remains in the original location. The extentMap portion of the display lists the migrated files.

You can tailor the migrate utility to the needs of your system. The following examples illustrate some possibilities.

You can migrate file abc and let the system pick a new location in the file domain:

# migrate abc

You can migrate pages 10 through 99 of file abc to volume 2 of the file domain:

# migrate -p 10 -n 90 -d 2 abc

You can move the pages of a striped file to different volumes within a file domain. For example, if the abc file is striped across three volumes (volumes 1, 2, 3) of a 6-volume file domain, you can use the migrate utility to move the pages from volume 2 to volume 4. You must specify the page offset and the page count for the pages you want to move in addition to the source volume information. Use the showfile command to determine the page count. In the following example, volume 2 contains pages between 8 and 39 of the striped file. To migrate the pages, enter:

# migrate -p 8 -n 32 -s 2 -d 4 abc

The abc file is now striped across volumes 1, 3, and 4.

Each file domain has a transaction log that keeps track of fileset activity for all filesets in the file domain. If the log resides on a congested disk or bus, or the file domain contains many filesets, system performance can degrade.

Use the showfdmn command to determine the location of the log. The letter L after the volume number indicates the volume on which the log resides. You can monitor performance of this volume with the iostat utility. If the volume containing the log appears overloaded, you can:

• Divide the file domain into several smaller domains. Then, each transaction log will handle transactions for fewer filesets.
• Use the switchlog command to move the log to a faster or less congested volume.

The switchlog command allows you to move the transaction log to any volume in the domain. Refer to the switchlog(8) reference page for more information. Moving the log to a faster device can improve performance. Issue the switchlog command with the following format:

switchlog domain_name new_volume_number

Moving the transaction log may also be useful when you are using LSM and wish to increase reliability by placing your transaction log on a volume that is mirrored.