Loading...

Data Aging - Oracle RAC iDataAgent

Table of Contents

Getting Started

Data Aging is the process of removing old data from secondary storage to allow the associated media to be reused for future backups.

You can change the retention of your data based on your needs.

Setting Up the Basic Retention Rule

  1. From the CommCell Browser, expand Policies > Storage Policies > storage_policy.
  2. Right-click the appropriate storage policy copy, and then click the Properties.
  3. In the Copy Properties dialog box, on the Retention tab, under Basic Retention Rule for All Backups, click the Retain For.
    • Enter number of days to retain the data.
    • Enter number of cycles to retain the data.
  4. Click OK.
  5. In the Confirm Basic Retention dialog box, click Yes.
  6. On the ribbon in the CommCell Console, on the Reports tab, click Forecast, and then click Run.

  7. The Data Retention Forecast and Compliance report displays the data to be pruned when a data aging job is run.

    Note:

    • To ensure only data intended for aging is actually aged, it is important to identify the data that will be aged based on the retention rules you have configured. Hence, ensure this report includes only the data you intend to age.

      If necessary, fine-tune your rules so that only the intended data is aged.

Running the Data Aging Job

  1. From the CommCell Console, right click the CommServe node, click All Tasks > Data Aging.
  2. In the Data Aging Options dialog box, in the Job Initiation area, define whether the data aging job runs immediately or if it will be scheduled.

  3. Click OK.

    If you chose to run the job immediately, the data aging job starts now.

    If you chose to run the job according to a schedule, the data aging job runs according to the schedule that you defined.

    After data aging job is run, the data will be pruned from the storage.

Extended Retention Rules

Extended retention rules allow you to keep specific full (or synthetic full) backups for an additional period of time.

Extended retention rules can be used in the following circumstances:

Extended retention rules allow you to define three additional "extended" retention periods for full (or synthetic full) backups. For example:

  • You may want to retain your weekly full backups for 30 days.
  • You may want to retain your monthly full backup for 90 days.
  • You may want to retain your yearly full backup for 365 days.

A backup job will be selected for extended retention based on its start time. For example: If a backup job starts at 11:55 pm on August 31st and ends at 1 am on September 1st, then it will be selected as the last full backup for the month of August and will be picked up for extended retention.

In all other cases, we recommend you to use Auxiliary Copy for extended storage as it actually creates another physical copy of the data, thereby reducing the risk of data loss due to media failure.

Enabling Automatic Eligibility of Full Backups for Selective Copy and Extended Retention Rules

When you install SnapProtect Version 11, all consistent and self-contained full backups are automatically eligible for selective copy and extended retention rules. You can change this option by modifying the Honor Extended Retention Rules for Oracle/RAC/SAP regular Full Jobs Media Management parameter.

When you upgrade to SnapProtect Version 11, you must set the  Honor Extended Retention Rules for Oracle/RAC/SAP regular Full Jobs Media Management parameter to 1 to have all consistent and self-contained full backups to be automatically eligible for selective copy and extended retention rules.

For information on configuring the Media Management parameter, see Media Management Configuration.

Data Aging for Transaction, Archive, and Logical Log Backups

Log Backups (transaction, archive, or logical logs) are not considered part of the backup cycle. Therefore, storage policy cycle retention parameters do not apply to them. However, log backups may be linked to data backup operations, which can affect their retention as follows:
  • Log backups are linked to a full backup if they are run at the same time. This is regardless of whether the full backup included data only or data and logs. Such backups follows the standard data aging rules.
  • If a full backup job is run on data and logs, then the next log backup will not be linked to this full backup job. These are unlinked log backups and by default, this will follow the unique data aging rules for log backups as given below:
    • Logs that need to be copied to secondary copies will not be aged both on primary and non-primary source copy
    • Logs that exist only on one copy will be aged when they are older than the oldest data
    • When logs exist on multiple copies, the logs on the copy with longest retention days will be retained with the data and will be aged after the oldest data. The log jobs on the remaining copies will be aged according to copy retention days without checking if the oldest data exists or not.
    • Partial, disabled logs will be aged when they are older than the oldest data
  • If a full backup job is run on data, then the next log backup job will be linked to this full backup job. These are considered as linked or chained log backups and are not aged until the linked data is aged. In addition, these log backups will also follow the unique data aging rules for log backups.

Pruning All Log Backups by Days Retention Rule

Use the following steps to disable log retention rule and enable unlinked log backups to be aged according to the defined days retention rule for the data. Note that when you define the days retention rule, use the same storage policy or retention criteria used for traditional rman backup, backup copy, and  traditional rman log backup operations.

For example, consider the following subclients:

  • Clone subclient using clone storage policy
  • Snap subclient using the snap storage policy
  • Log subclient using the log storage policy

The clone jobs using backup copy are run daily, snap jobs without backup copy are run every hour, and regular log backup jobs are run every 15 minutes.

When you disable log rule, make sure the retention for log storage policy primary copy is same or higher than the retention set for clone storage policy primary copy. Alternatively, you can also assign the same clone storage policy for archive log backups.

  1. From the CommCell Console, click the Storage tab.
  2. Click the Media Management icon.
  3. Click the Data Aging tab.
  4. In the Prune All Database Agent Logs Only By Days Retention Rule, type 1 to enable pruning for all database agent logs based on the days retention.
  5. Click OK.

Data Aging of the Oracle Recovery Catalog Database

When a Data Aging job is run, the BackupPieceName UNAVAILABLE command is automatically issued to RMAN to disable specific backup pieces in the Oracle Recovery Catalog database that were pruned from the Media Manager CommServe tables. Any backup pieces that were aged from the system's database that have exceeded their retention criteria will be marked as unavailable in the Oracle Recovery Catalog database through this methodology. You can delete these specific backup pieces by creating and enabling the OracleDeleteAgedBackupPiece additional setting.

Data Aging Rules for Oracle Archive Index

Oracle archive index is deleted when the associated backup data is deleted. This applies to SnapProtect Backup and Table Level Backup.

Disable Oracle RMAN Crosschecks during Data Aging

By default, during a data aging operation, an Oracle CROSSCHECK is performed by the system to synchronize the entries in the CommServe database with the RMAN catalog. If required, you can disable this CROSSCHECK operation using the Disable RMAN Cross Check option in the Instance Properties (Details) for the specific Oracle RAC instance. For instructions, see Disable RMAN Cross Check.

Data Aging Rules for Selective Online Full Backups

A selective online full operation that consists of archive logs and oracle data can also be linked to the logs of a separate job, which was initiated within the time frame of the selective online full operation. These logs and the selective online full are then considered as one entity within the software, regardless of whether or not separate jobs have the same job ID. Therefore, they are copied to synchronous and selective copies together during auxiliary copy operations and are aged together. If any part of the selective online full is missing from a copy, the full will not be considered as a valid full and will not be counted as a cycle during data aging. Consider the following:

  • Data from selective online full backups are considered the same as regular full and offline full backups for each subclient in terms of basic retention rules of cycles and days. However, if any logs on a primary copy have not been fully copied to a secondary copy, the selective online full cannot be aged.
  • Data from selective online full backups are considered the same as offline full backups for each subclient in terms of extended retention rules of days. Selective online full backups and all logs linked with it must be retained together on the same storage policy copy.
  • Those Logs that are linked with a selective online full (and the logs of the selective online full) can be aged only if they are older than the oldest data that can be aged and the corresponding data of the selective online full that can be or have been aged.
  • Selective online full backup jobs that are completed with errors will not be retained by extended retention rules during data aging operation.

Data Aging Rules for Command Line Backups

  • The third party command line log backups can be linked to third party command line data backups as well as any other kind of backup data as per regular data link rule.
  • Data from third-party command line backups is aged differently than data from backups initiated through the CommCell Console. Retention cycles are not used for copies involved in operations from the third-party command line. For such operations, data is aged according to the associated retention time. However, you can manually set the retention time for each third party command line job from the storage policy copy. The command line log backups will be aged according to the retention time set for its associated command line data backup job.
  • The qoperation agedata command can age data and logs simultaneously based on the Job ID, and it is especially useful for aging each of these items separately.
  • Command line full jobs will be aged only when all of the incremental data backups in that cycle are eligible for pruning.

Data Aging Rules for On-Demand and Customized RMAN Script Backups

Data Aging for Oracle On Demand and Customized RMAN Script backup jobs uses days/time, and ignores cycles, as the determining factor for pruning the data. Therefore, once the retention time criteria has been met, all data (for both data and logs) is pruned that was backed up using the storage policy specified in the RMAN script that was run through the Command Line Interface.

An effective storage policy strategy for Oracle On Demand and Customized RMAN Script backups is as follows:

  • The same storage policy should not be used for regular Oracle backups and Oracle On Demand backups or Customized RMAN Script backups.
  • The storage policy copy containing logs of Oracle On Demand backups or Customized RMAN Script backups should have a much longer retention time than other storage policies used by regular Oracle backups for the same instance. This is to prevent the logs of Oracle On Demand backups from being pruned before the data of regular Oracle backups, and allow the database to be fully restored and recovered using the data of old regular Oracle backups and logs afterwards.

Oracle RMAN Retention Policy

An Oracle RMAN retention policy can be configured for each database. When RMAN retention rules are in effect, RMAN considers the backup jobs comprising data files and control files as obsolete, that is, no longer needed for recovery, according to criteria that you specify in the CONFIGURE RETENTION POLICY command. When you run DELETE OBSOLETE or CROSS CHECK operations, RMAN ages data by freeing disk and tape space used by backups that are no longer needed.

Do not configure RMAN retention policy if you want to retain data using the data aging feature provided in the CommCell console. To disable the RMAN retention policy, use the following command: CONFIGURE RETENTION POLICY TO NONE. This ensures that data will only be aged according to the retention rules specified in the associated storage policy copy.

Data Aging Rules for Jobs Completed with Errors

Cycle retention is not honored on selective online jobs that complete with errors.  The job gets pruned after the configured storage policy retention (the Basic Retention Rule for All Backups on the Retention tab of the Storage Policy Properties dialog) expires.

To apply extended retention rules to these jobs, exclude jobs that completed with errors during extended retention calculations. This option is applicable only for Selective Online full backup jobs.

  1. From the CommCell Console, click the Storage tab.
  2. Click the Media Management icon.
  3. Click the Data Aging tab.
  4. Change the value for the Ignore Completed With Errors job option for Extended Retention calculations option from 1 to 0.
  5. Click OK.

Advanced Topics

Data Aging - Advanced

Provides comprehensive information on additional Data Aging capabilities.