Moving CommCell Operations to Standby CommServe Host: Advanced Failover

If your CommServe database and logs are regularly protected on a dedicated standby CommServe host, when the production CommServe host becomes unusable or inaccessible, you can manually fail over the CommServe functionality to the standby CommServe host.

In a non-disaster scenario, if you want to move your CommCell operations to standby host and use the standby as the production for a long period of time, you can perform a planned manual failover.

The failover process is initiated manually from the passive node, or from any existing witness server hosts. Make sure to provide a minimum of 1 minute time interval between subsequent failover operations.

Before You Begin

Latest software updates must be installed on the standby CommServe host. See Installing Updates on the Standby CommServe Host for instructions.

For a planned failover, complete the following steps:

Procedure

  1. Verify which CommServe is the active node, from any machine (client or server) by running the cvfailover.cmd command with no parameters.

    The cvfailover.cmd run with no parameters returns the node ID of the active CommServe. If the ID returned is 0, a failover is in progress. Wait for the failover to complete and run the command again.

    Sample output:

    Passed file name: C:\Program Files\##_DOC_OEM_INSTALL_DESTINATION_COMPANY_FOLDER_##\SnapProtect2\CvFailover\bin\..\conf\zoo.cfg
    installationDir: C:\Program Files\##_DOC_OEM_INSTALL_DESTINATION_COMPANY_FOLDER_##\SnapProtect2\CvFailover\bin\..\
    My ID: 22
    Current active node is '1'

  2. Set the active CommServe node to the standby CommServe host. See Setting the Active Node for CommServe Failover.
  3. If failover was performed to recover from a disaster, you must resynchronize the Deduplication Databases (DDBs).
  4. After restoring the CommServe database, any DDBs that are inconsistent with the CommServe database are flagged into maintenance mode automatically. Therefore, you must synchronize the inconsistent DDBs to reusable state. See Resynchronize the Deduplication Database.

    Skip this step for planned failover operations.

  5. Resume any pending backup and restore jobs on the CommCell Console.

Result

The standby CommServe will now become the active node and performs all operations of the production CommServe host.

Note: After a failover operation, you will notice a considerable increase in the job ID for new jobs. This increase is added to avoid conflicts with job IDs referenced in the deduplication databases (DDBs) when the DDBs are resynchronized.

What To Do Next

  • If the clients are not part of any DNS server, update the host file on the individual clients with the active CommServe host name.
  • On Windows 2012 hosts, if CommCell services do not start automatically after the failover operation, then start the services manually.

    At the command prompt on the standby CommServe host, go to Installation_Directory/cvfailover, and type the following command:

    cvfailover_start.cmd /service

  • Download latest software updates to the software cache directory on the standby CommServe host. For more information on downloading updates, see Getting Started - Download Updates.
  • Rebuild your original production CommServe host and stage the CommServe database using the following steps:

    Note: Make sure Services are not running on the current active (standby) CommServe host.

    1. Update the DNS entry for floating CommServe name with the IP address of the original production CommServe host.
    2. Install the CommServe software on the original production host. Review CommServe Installation: Considerations for Disaster Recovery Failover.
    3. Stop Services on the original production CommServe host. See Stopping a Service.
    4. Start Services on the current active (standby) CommServe host. See Starting a Service.
    5. Install the SQL Server Agent on a new instance on the original production CommServe host.  Review SQL Server Agent Installation: Considerations for Disaster Recovery Failover.
    6. Install the CommServe Failover Component on three independent windows hosts (for witness server roles) and the second instance of the original production CommServe host.

      You can use a new or existing MediaAgent computer on the production CommServe location to act as the Witness Server.

    7. Configure a SQL Server Agent Subclient  for the second instance of the CommServe client.
    8. Configure the CommServe Failover Component using advanced configuration type. Provide the original configuration values used for the production CommServe host.
    9. Backup and restore the CommServe databases from the standby CommServe host to the original production CommServe host. See Synchronize the CommServe Databases.

    Note: Once the CommServe databases on production and standby hosts are synchronized, you can fail over the CommServe functionality to the production host using the same procedure.