Loading...

Data Aging - DB2 iDataAgent

Table of Contents

Getting Started

Data Aging is the process of removing old data from secondary storage to allow the associated media to be reused for future backups.

You can change the retention of your data based on your needs.

Setting Up the Basic Retention Rule

  1. From the CommCell Browser, expand Policies > Storage Policies > storage_policy.
  2. Right-click the appropriate storage policy copy, and then click the Properties.
  3. In the Copy Properties dialog box, on the Retention tab, under Basic Retention Rule for All Backups, click the Retain For.
    • Enter number of days to retain the data.
    • Enter number of cycles to retain the data.
  4. Click OK.
  5. In the Confirm Basic Retention dialog box, click Yes.
  6. On the ribbon in the CommCell Console, on the Reports tab, click Forecast, and then click Run.

  7. The Data Retention Forecast and Compliance report displays the data to be pruned when a data aging job is run.

    Note:

    • To ensure only data intended for aging is actually aged, it is important to identify the data that will be aged based on the retention rules you have configured. Hence, ensure this report includes only the data you intend to age.

      If necessary, fine-tune your rules so that only the intended data is aged.

Running the Data Aging Job

  1. From the CommCell Console, right click the CommServe node, click All Tasks > Data Aging.
  2. In the Data Aging Options dialog box, in the Job Initiation area, define whether the data aging job runs immediately or if it will be scheduled.

  3. Click OK.

    If you chose to run the job immediately, the data aging job starts now.

    If you chose to run the job according to a schedule, the data aging job runs according to the schedule that you defined.

    After data aging job is run, the data will be pruned from the storage.

Retention Rules for Data Only Backups

If a data backup job does not have a linked log backup, it will be retained based on the retention days only and does not follow the retention cycles. However, if the data backup job is linked to a log backup job, the retention will be based on both the number of days as well as the retention cycles.

Retention Rules for Command Line Full Backups

DB2 command line full backups are pruned based on the number of retention days.

Incremental backups performed through the CommCell Console after the command line full backup are kept, but will be orphaned when the command line full backup is aged. The DB2 history file has a record for the  incremental backups performed through the CommCell Console which depend on the command line full but the database is not recoverable from the incremental.

Command line jobs and incremental backups performed through the CommCell Console must not be combined on the same database because they will not be recoverable.

Extended Retention Rules

Extended retention rules allow you to keep specific full (or synthetic full) backups for an additional period of time.

Extended retention rules can be used in the following circumstances:

Extended retention rules allow you to define three additional "extended" retention periods for full (or synthetic full) backups. For example:

  • You may want to retain your weekly full backups for 30 days.
  • You may want to retain your monthly full backup for 90 days.
  • You may want to retain your yearly full backup for 365 days.

A backup job will be selected for extended retention based on its start time. For example: If a backup job starts at 11:55 pm on August 31st and ends at 1 am on September 1st, then it will be selected as the last full backup for the month of August and will be picked up for extended retention.

In all other cases, we recommend you to use Auxiliary Copy for extended storage as it actually creates another physical copy of the data, thereby reducing the risk of data loss due to media failure.

Online data only full backups from that you perform from the CommCell Console on a DB2 database that is version 9.5 or above are eligible for extended retention rules. 

Note the following regarding extended retention rules:

  • During a full backup, if the data and logs use the same storage policy, the extended retention rules will apply to both the data and the logs.
  • Extended retention rules can be applied to data only full backup jobs only after the subsequent log backup job is linked to the data backup job.
  • Extended retention rules do not apply for log only backups. However, if the log backup job is linked to a data backup job, and uses the same storage policy used by data backups, then the extended retention rules of the data backup job is also applied to the linked log backup job.
  • If the data and linked logs use different storage policy, the extended retention rules will be applied to the data backups only.

Setting Up the Extended Retention Rules

Use the following steps for setting up the extended retention rules:
  1. Right-click the storage policy copy and click Properties.
  2. Click the Retention tab.
  3. Set the basic retention rules by clicking Retain for and entering the number of days and cycles appropriate for your organization.
  4. Set the extended retention rules as follows:
    1. Click the For button.
    2. Enter the number of Days Total to retain the backup.
    3. Click the Keep drop-down list, and select the desired backup criteria (e.g., Monthly Full, Weekly Full).
    4. Click the Grace Days drop-down list and select the number of days (e.g., 2).

      This allows you to consider the additional number of days along with the Extended Retention rule. For example, if the last full backup job fails with in the defined extended retention criteria, then the next full backup job that ran in the specified grace days will be selected for retention.

  5. Repeat Step 4 to configure additional extended retention.
  6. Click OK.

Data Aging for Transaction, Archive, and Logical Log Backups

Log Backups (transaction, archive, or logical logs) are not considered part of the backup cycle. Therefore, storage policy cycle retention parameters do not apply to them. However, log backups may be linked to data backup operations, which can affect their retention as follows:
  • Log backups are linked to a full backup if they are run at the same time. This is regardless of whether the full backup included data only or data and logs. Such backups follows the standard data aging rules.
  • If a full backup job is run on data, then the next log backup job will be linked to this full backup job.

    These are considered as linked or chained log backups and are not aged until the linked data is aged. In addition, the following is also considered:

    • Logs that need to be copied to secondary copies will not be aged both on primary and non-primary source copy
    • Logs that exist only on one copy will be aged when they are older than the oldest data
    • Logs that exist on multiple copies will be aged according to copy retention days
    • Logs that exist on multiple copies with the longest retention days will be aged when they are older than the oldest data
    • Partial, disabled logs will be aged when they are older than the oldest data
  • If a full backup job is run on data and logs, then the next log backup will not be linked to this full backup job.

    As this is an unlinked log backup, by default, this will follow the unique data aging rules for log backups. If you want such log backups to be aged according to the defined days retention rule for the data, you can do so as follows:

    1. From the CommCell Console, click the Storage tab.
    2. Click the Media Management icon.
    3. Click the Data Aging tab.
    4. In the Prune All Database Agent Logs Only By Days Retention Rule, type 1 to enable pruning for all database agent logs based on the days retention.
    5. Click OK.

Data Aging Rules for Command Line Backups

  • The third party command line log backups can be linked to third party command line data backups as well as any other kind of backup data as per regular data link rule.
  • Data from third-party command line backups is aged differently than data from backups initiated through the CommCell Console. Retention cycles are not used for copies involved in operations from the third-party command line. For such operations, data is aged according to the associated retention time. However, you can manually set the retention time for each third party command line job from the storage policy copy. The command line log backups will be aged according to the retention time set for its associated command line data backup job.
  • The qoperation agedata command can age data and logs simultaneously based on the Job ID, and it is especially useful for aging each of these items separately.
  • Command line full jobs will be aged only when all of the incremental data backups in that cycle are eligible for pruning.

Data Aging Rules for Jobs Completed with Errors

A DB2 backup job will be marked as completed with error even if it fails on log phase. Jobs that are completed with errors are treated as a valid full backup jobs. They are pruned based on basic days retention rules and ignore cycle retention rules. However, extended retention rules will exclude these completed with error jobs and Selective copy.

Data Aging Rules for Selective Copy

A selective copy allows you to copy specific full backup jobs from a source copy. The source copy can be either a primary or a synchronous copy. Selective copy facilitates better tape rotation. Consider the following for DB2:

  • A DB2 full backup will be copied to a selective copy if the log job is done after the data job. If this is not the case, the data job is not copied to the selective copy and will have a "Log Link Pending" status.
  • A full backup has a one hour delay before it is copied to the  selective copy.
  • Log jobs are not copied if they are using a different storage policy. The log jobs will be retained by the log policy rules.
  • Online data only full backups from that you perform from the CommCell Console on a DB2 database that is version 9.5 or above are eligible for selective copy.

Additional details on Selective Copy can be found here.

Pruning a Backup Image

You must enable the DB2 triggered pruning for backup images for SnapProtect using the sCvDB2EnablePruning Additional Setting.

  • Set sCvDB2EnablePruning Additional Setting to y in the registry under <SnapProtect Instance>/Db2Agent

    Example:

    Instance001/Db2Agent

  • Now, you must run the following command on a DB2 Client to prune the backup image:

    $ db2 prune history <timestamp> with force option and delete

    Example:
    [db2inst1@standard ~]$ db2 prune history 20130801192856 with force option and delete
    DB20000I The PRUNE command completed successfully.

    When you prune a backup image which includes data and log files, the backup history will still show this backup job until it meets SnapProtect's retention policy. In such a case, you must restore only the log files during Restore To Disk job.

Advanced Topics

Data Aging - Advanced

Provides comprehensive information on additional Data Aging capabilities.