Troubleshooting - Informix iDataAgent

Table of Contents

Backup Failures

The following section provides information on troubleshooting backups.

INF0001: Multi Stream Backups Fail if Enable Client Side Disk Cache Option is Selected at Client Level

Issue:

The Multi Stream backup fails.

The Informix iDataAgent allows users to run multi-stream backup on the same subclient at same time.

If you selected to  Enable client side disk cache the at client level, then BAR_MAX_BACKUP value in the ONCONFIG file on the Informix server should be 1. Multi stream backups will fail.

Resolution:

Use multistreams by running a backup to a storage policy using MediaAgent side deduplication or regular client side deduplication without selectingthe  Enable client side disk cache option on the client.

INF0002: Command Line Backup Failure when a 64-bit Application is running on a 64-bit Solaris Client

Issue:

If a 64-bit application is running on a 64-bit Solaris client, the command line backup fails with the following error:

GetProcAddress failed, geterror=ld.so.1: onbar_d: fatal: invalid handle

Example:

bash-3.00$ onbar -b -L 0
onbar is starting ..........
Galaxy Instance: Instance001
Loading Library:/opt/simpana/Base64/libCvIbsad.so from dir /opt/simpana/Base64
LoadLibrary failed when loading libCvIbsad.so, error = ld.so.1: onbar_d: fatal: libCstd.so.1: open failed: No such file or directory
onbar is starting ..........
Galaxy Instance: Instance001
Loading Library:/opt/simpana/Base64/libCvIbsad.so from dir /opt/simpana/Base64
GetProcAddress failed, geterror=ld.so.1: onbar_d: fatal: invalid handle

Resolution:
Export LD_LIBRARY_PATH=<SimpanaInstall>/Base64 and run the command line jobs.

INF0003: Timeout Failures

Issue:

Backups fail due to timeouts.

Resolution:

The default time for resources to allocate streams during an Informix third-party command line backup job is 86400 seconds (i.e., 24 hours). If a backup fails due to a timeout (86400 seconds) being reached, you can configure the sALLOCATESTREAMSECS additional setting to increase the waiting time period.

INF0004: Backup Job Goes to Pending State

Issue:

Case 1:

Backup jobs go to pending with following error in bar_act.log.

Windows:

2013-12-23 09:33:00 5468 5468 E:\Informix\12.10\bin\onbar_d.exe -b -L 0
2013-12-23 09:33:00 5468 5468 Working with cvsm as generic storage manager.
2013-12-23 09:33:00 5468 5468 Successfully connected to Storage Manager.
2013-12-23 09:33:00 5468 5468 (-43207) Unable to open connection to database server: could not fork server connection, SQLCODE -25553.

Unix:

2014-02-24 11:10:10 10775 10769 onbar_d -b -L 0
2014-02-24 11:10:10 10775 10769 Working with cvsm as generic storage manager.
2014-02-24 11:10:10 10775 10769 Successfully connected to Storage Manager.
2014-02-24 11:10:10 10775 10769 (-43207) Unable to open connection to database server: could not fork server connection, SQLCODE -25553.

Resolution

Check if the name of file containing sqlhosts information for the server is ‘sqlhosts’. If the name is different, add the  InformixServername_SQLHOSTFile  Additional Setting specifying full path to the sqlhosts file.

Use the onstat -g dis command to display the location of sqlhosts file for a particular server.

onstat -g dis command output:

There are 2 servers found
Server : ol_informix1210_1
Server Number : 111
Server Type : IDS
Server Status : Up
Server Version: IBM Informix Dynamic Server Version 12.10.FC3
Shared Memory : 0xa000000
INFORMIXDIR : /INFORMIX/ol_informix1210_1
ONCONFIG : /INFORMIX/ol_informix1210_1/etc/onconfig.ol_informix1210_1
SQLHOSTS : /INFORMIX/ol_informix1210_1/etc/sqlhosts.ol_informix1210_1
Host : breezer

Server : ol_informix1210_2
Server Number : 222
Server Type : IDS
Server Status : Down
Server Version: IBM Informix Dynamic Server Version 12.10.FC1
Shared Memory : 0xa000000
INFORMIXDIR : /INFORMIX/ol_informix1210_2
ONCONFIG : /INFORMIX/ol_informix1210_2/etc/onconfig.ol_informix1210_2
SQLHOSTS : /INFORMIX/ol_informix1210_2/etc/sqlhosts.ol_informix1210_2
Host : breezer

Add the following additional settings to protect these two servers.

ol_informix1210_1_SQLHOSTFile /INFORMIX/ol_informix1210_1/etc/sqlhosts.ol_informix1210_1

ol_informix1210_2_SQLHOSTFile /INFORMIX/ol_informix1210_1/etc/sqlhosts.ol_informix1210_2

Case 2

Backup jobs go to pending with following error in bar_act.log.

2014-02-24 11:12:27 10974 10967 onbar_d -b -L 0
2014-02-24 11:12:27 10974 10967 Working with cvsm as generic storage manager.
2014-02-24 11:12:27 10974 10967 Successfully connected to Storage Manager
2014-02-24 11:12:27 10974 10967 (-43207) Unable to open connection to database server: could not fork server connection, SQLCODE -25596.

Resolution:

Check if the entries in sqlhosts file are correct.

INF0005: Command Line Backup Fails when a Log Backup Is Triggered by the Alarm Program Interferes with a Full Backup

Issue

A command line backup fails with the following error message:

2015-09-28 00:10:43 3863 3861 /informix/bin/onbar_d -b -l
2015-09-28 00:10:44 3863 3861 Working with cvsm as generic storage manager.
2015-09-28 00:10:44 3863 3861 (-43395) A log backup is already running. Can't start another.
2015-09-28 00:10:49 3863 3861 /informix/bin/onbar_d complete, returning 152 (0x98)

Resolution

This occurs when you have set the BA_MAX_BACKUP onconfig parameter is set to 1.

  1. Increase the number of streams by setting the BA_MAX_BACKUP onconfig parameter.
  2. If there is no log space restriction, use the log_full.sh script in place of the alarmprogram.sh script to trigger the backups for full logical logs.

This delays the start of each log backup and prevents it from being triggered for each individual log.

Best Practice: Copy and save the log_full.sh script file before you modify the file.

Add the following lines to the log_full.sh script file.

Set the parameters as:

Parameter Value
N The number of full logical logs that you want to retain
CvClientName The client that is configured in the CommCell Console
CvInstanceName The name of the Informix instance that is configured in the CommCell Console

################################
# Added the following to only backup logs
# when there are N or more logs to be backed up.
echo "$3" >> $INFORMIXDIR/Log_Cnt
COUNT=`cat $INFORMIXDIR/Log_Cnt |wc -l`
if [ $COUNT -ge N ]
then
    export CvClientName= <client_name_as_seen_in_commcell_console>
    export CvInstanceName=<Instance00x>
    $BACKUP_CMD 2>&1 > /dev/null
    #Clear Log_Cnt after backing up logs...
    cat /dev/null > $INFORMIXDIR/Log_Cnt
fi
################################

Example

To trigger the log backup after there are 5 or more full logical logs, set the parameters as:

Parameter Value
N 5
CvClientName informix_client
CvInstanceName Instance001

################################
# Added the following to only backup logs
# when there are 5 or more logs to be backed up.
echo "$3" >> $INFORMIXDIR/Log_Cnt
COUNT=`cat $INFORMIXDIR/Log_Cnt |wc -l`
if [ $COUNT -ge 5 ]
then
    export CvClientName= informix_client
    export CvInstanceName=Instance001
    $BACKUP_CMD 2>&1 > /dev/null
    #Clear Log_Cnt after backing up logs...
    cat /dev/null > $INFORMIXDIR/Log_Cnt
fi
################################

Completed with One or More Errors

Backup jobs from Informix iDataAgent will be displayed as "Completed w/ one or more errors" in the Job History in the following cases:

  • Job is killed after creating some valid data archive files. For example: after a database backup is run, if job is killed during the Logs phase.