Troubleshooting - DB2 iDataAgent

Table of Contents

Backup

DB20006: Log only backup fails after a DB2 upgrade from 32-bit to 64-bit

Issue

Log Only Backups fail after a DB2 upgrade from 32-bit to 64-bit.

Symptom

Log Only Backups fail after a DB2 upgrade from 32-bit to 64-bit.

Resolution

The  sqllib/lib library is not linked to the correct 64-bit library.  Link the sqllib/lib library to a correct lib64 for 64bit DB2 .

DB20007: Log backup fails

Issue

If you create the directories, subdirectories or files under SnapProtect Log Archive/Retrieve/Audit Error paths, the log backups may fail.

Symptom

Log Backups fail.

In the example, the "db2inst1/DB5290/NODE0000/C0000000//lost+found" directory exists under "/home/db2inst1/db2Archive" path. Hence, the log backups failed. The following error message is displayed in ClDb2Agent.log:

11443 40476940 11/12 16:20:52 192887 Db2UserExit::OpenLogFile() - 1: fopen(/home/db2inst1/db2Archive/db2inst1/DB5290/NODE0000/C0000000//lost+found) 11443 40476940 11/12 16:20:52 192887 Db2Util::Db2StringToULong() - 0: ERROR: db2str.size()=10 < i
11443 40476940 11/12 16:20:52 192887 Db2UserExit::PopulateLogFileHeader() - 0: Util.Db2StringToULong(lost+found) failed.
11443 40476940 11/12 16:20:52 192887 Db2UserExit::BackupLogsFromPath() - 0: PopulateLogFileHeader() failed.
11443 40476940 11/12 16:20:52 192887 Db2UserExit::BackupFromAD() - 0: BackupLogsFromPath failed.
11443 40476940 11/12 16:20:52 192887 Db2UserExit::Backup() - 0: BackupFromAD() failed: .
11443 40476940 11/12 16:20:52 192887 ClDb2Agent::LogsBackup() - 0: Before delete ue...: ue->Backup() failed: errorHelpString=<>.
11443 40476940 11/12 16:20:52 192887 ClDb2Agent::LogsBackup() - 1: SetPendingCauseAndEvent:DB2_LOGBACKUP_FAILED
11443 40476940 11/12 16:20:52 192887 Db2JobObject::Db2JobObject::SetPendingCauseAndEvent() - 1: Setting the pending cause successful.

Resolution

If you have created any directories, subdirectories or files under SnapProtect Log Archive/Retrieve/Audit Error paths, move them to another location..

DB20008: Backup jobs fail with one or more errors

Issue

The backup job fails.

Symptom

Backup jobs from DB2 iDataAgent will be displayed as "Completed w/ one or more errors".

Resolution

This happens in the following cases:

  • Job is killed after creating some valid data archive files. For example: after running a database backup, if job is killed during the logs phase.
  • When an online/offline/command line backup job completes the data phase but fails on the log phase.
  • When a LOGS only backup job fails or is killed after you backup some logs.

DB20009: Database is in the BACKUP_PENDING state

Issue

The database is left in a BACKUP_PENDING state

Resolution

If you have modified he LOGARCHMETH1 parameter without performing an offline full backup after, the database is left in the BACKUP_PENDING state.

Perform an offline full backup.

DB20010: A DB2 Backup Fails with Error Code 19:1334

Issue:

The DB2 backup fails with the following error.

ERROR CODE [19:1334]: DB2 Restore [Error: wrong message=282 from PL.]
Source: noandvdb03a, Process: DB2SBT

Solution:

  1. Restart the DB2 services.

DB20012 Target Database Cannot Access Latest Log Chain During a Cross-Machine Restore

Issue:

When the DB2 log chain is being used during a cross-machine restore and the target database does not have access to the latest log chain, the log retrieval restore fails during the backup.

Solution:

Prior to running the restore:

  1. Ensure the latest DB2 FixPack is installed on the target DB2 database.
  2. In the CommCell Console, delete the target database instance, which also removes references to previous data.
  3. Delete the database on the target server and recreate it.
  4. In the CommCell Console, re-create the instance and the backup set for the target database.
  5. Re-run the cross-machine restore to refresh the production database to the target database.

Vendor Code Troubleshooting

Reason Code 1: Troubleshooting backup failures after a new DB2 Agent install

Issue:

  1. The backup may fail due to insufficient permissions for accessing the SnapProtect registry, log files and base directories.
  2. The semaphore, file limits are not set properly.
  3. The Reason Code 1 is displayed with the following

[IBM][CLI Driver] SQL2071N An error occurred while accessing the shared library "C:\Program Files\CommVault\Simpana\Base\Db2Sbt.dll". Reason code: "1".

Solution

  1. Use the cvpkgchg utility to set correct group, access permissions for the Simpana installation. See Changing Group Ownership for Simpana Files and Directories After Installing a Software Component on UNIX for more information.
  2. Set the semaphore, file limits as per the install prerequisites. See Configuring the Kernel Parameters for more information.
  3. On Windows check the PATH environment variable has the Simpana path.

Return Code 6

Issue:

  1. The requested backup image or log file cannot be found.

Solution

  1. Verify the backup was taken.
  2. Verify the backup has not been pruned.

Return Code 11

Issue:

  1. The initialization failed.

    Example

    dbserve36:/home/db2/V9.7B64/db2inst5>db2 backup db testdb load /opt/simpana/Base/libDb2Sbt.so
    SQL2062N An error occurred while accessing media "/opt/simpana/Base/libDb2Sbt.so". Reason code: "11".

Solution

  1. Verify the vendor parameters are specified correctly.
  2. Verify the connectivity between the CommCell, CommServe computer and MediaAgent was not interrupted.
  3. Verify the client services are running.
  4. Verify the SnapProtect group is set to the db2admin group.

Return Code 18

Issue:

  1. An error occurred during the data transfer.

    Example

    dbserve36:/home/db2/V9.7B64/db2inst5>db2 backup db testdb load /opt/simpana/Base/libDb2Sbt.so

Solution

  1. Verify the connectivity between the CommCell, CommServe and MediaAgent was not interrupted.
  2. Verify resources are available on the MediaAgent.

Return Code 30

Issue:

  1. There was a critical error.

    11403590 304 05/12 18:03:18 1048366 Db2Check::SqlInfoPrint() - 0:
    ---- error report ----
    app. message = backing up the database
    line = 4804
    file = ClDb2AgentLib.cpp
    SQLCODE = -2079
    SQL2079N An error was reported by the shared library
    "/opt/simpana/Base64/libDb2Sbt.so". Return code: "30".

    --- end error report ---
    11403590 304 05/12 18:03:18 1048366 ClDb2Agent::DataBackup() - 1: SetPendingCauseAndEvent:DB2_SBTCRITICAL_ERROR

Solution

  1. Contact support with the following information ready to send.
    • The database configurations,
    • the db2dump directory,
    • all crash dumps

Effective Troubleshooting

When you encounter a DB2 iDataAgent issue with backup or restore, provide the following information to the Customer Support for troubleshooting the issues:
  • Failed backup/restore Job Type, backup or restore with JOB ID number – Initiated either from CommCell Console or DB2 Command line.
  • Complete set of logs from the CommServe, MediaAgents and the DB2 iDataAgent having an issue. Do not collect logs by Time Range or JOB ID options as this will not provide the information required to troubleshoot a DB2 Backup or Restore issue.
  • The exact OS platform for UNIX or Windows servers that the DB2 Application is running on with FixPack or Maintenance levels.
  • Log on to the DB2 server using the DB2 user account configured in the CommCell Console and perform the following commands:

    db2level
    (This command will provide the current DB2 application level)
    db2 list db directory
    (This command will provide all of the DB2 databases currently configured)
    db2 get db cfg for database_name
    (The database_name for each database listed in the above command. This will need to be run multiple times, once for each database_name listed)

In most cases, you may be required to send the DB2 applications db2diag.log. This log is not part of the NetApp Software components. It is the DB2 application database log file.