Loading...

Network: Data Pipe

Table of Contents

Overview

The Data Pipe is the logical mechanism through which data is transferred between a client and a MediaAgent. You configure and optimize the Data Pipe using the Data Pipe Buffers, Network Agents and Network Bandwidth Throttling, and Application Read Size parameters. All these parameter settings should be optimized, so that data transfer is not only fast and lossless, but also does not use more resources than necessary.

Data Pipe Buffers

The Data Pipe Buffers setting controls the amount of shared memory allocated for data pipes on each client or MediaAgent computer. By default, the system allocates 30 pipeline buffers of 64KB size on each client computer. (30 buffers times 64KB each allocates about 2 MB of shared memory.)

You can configure additional buffers for transferring data between the client and the MediaAgent. Having more pipeline buffers may translate into faster dedicated (non-multiplexed) backups to tape devices, and therefore improve data transfer performance. Of course, additional buffers consume more shared memory.

Configuring Data Pipe Buffers

This is an advanced feature. Please use caution when modifying this setting.

Important: You can set the number of data pipe buffers on the MediaAgent or the client. If the value is set both on the client and the MediaAgent, the setting on the client is used.

Change the default data pipe buffers value (30) as follows:

  • Setting the number of pipeline buffers and the pipeline buffer size from the subclient level from the Commcell Console. For more information, see subclient specific performance tab.
  • The number of pipeline buffers can also be set by using the nNumPipelineBuffers additional settings. For more information on configuring an additional settings see, Add or Modify an Additional Setting.

Network Agents

Network Agents establish the data pipes. By default, the system uses two concurrent network agents, which may be suitable for 100-megabit networks. Increasing this value may increase the data transfer throughput from the client. Other network features, such as NIC speed and network switch speed, also impact network throughput. You may need to experiment in your environment to determine whether you can realize faster data throughput by changing the number of concurrent network agents.

Setting the Network Bandwidth and Network Agents for a Data Protection Operation

Note: All subclients except those using NAS agents support network bandwidth throttling.

To set a limit for network bandwidth and the number of concurrent network agents for a subclient backup operation, perform these steps:

  1. In the CommCell Browser, expand Client Computers > client_name > agent > backup set or instance.

    Note: If limits are set at the instance level and the subclient level, the subclient settings override the instance settings.

  2. Right-click the subclient and click Properties.

    The Subclient Properties dialog box is displayed.

  3. For file system subclients, update the following:
    1. On the Storage Device tab, click the Data Transfer Option tab.
    2. Select the Throttle Network Bandwidth (MB/HR) check box and enter the throughput limit. The SnapProtect software throttles the transfer on a per-network-agent basis.
    3. Click Advanced.

      The Advanced Subclient Properties dialog box is displayed.

    4. On the Performance tab, set the maximum number of Network Agents to run concurrently when performing a subclient backup.
  4. For non-file system subclients, on the Storage Device tab, click the Data Transfer Option tab and update the following:
    • Set the maximum number of Network Agents to run concurrently when performing a subclient backup.
    • Select the Throttle Network Bandwidth (MB/HR) check box and enter the throughput limit. The SnapProtect software throttles the transfer on a per-network-agent basis.
  5. Click OK to save your changes.

Setting the Network Bandwidth and Network Agents for Multiple Subclients Using Subclient Policy

For agents that support subclient policies, network agents can be established for creating multiple subclients using an existing subclient policy.

Use these steps to set the Network Bandwidth and Network Agents while creating multiple subclients:

  1. From the CommCell Browser, expand Policies.
  2. Right-click Subclient Policies, then click New Subclient Policy.
  3. In the Name box, enter a name for your subclient policy.
  4. Click the iDataAgent list and select Windows File System.
  5. Click once in the Storage Policy column cell and select a storage policy for the default subclient.
  6. Click the Association tab, then select the backup set on the client that you want to associate with this subclient policy.
  7. Click the General tab, then select the default subclient row and click Edit.
  8. Click the Content tab, then enter the contents for the subclient by browsing to them or by entering their path. If browsing, select a file or folder as content, then click Add. When finished adding content, click Close.
  9. Click the Storage Device tab and select the storage policy.
  10. Click the Data Transfer Option tab.
    1. Enter the number of Network Agents that must be used to perform backup operations on the subclient/instance.
    2. Click the Throttle Network Bandwidth (MB/HR) option, then enter a throughput value. Note that throttling is done on a per-Network-Agent basis.
  11. Click OK to save the changes.
  12. Click OK to complete this operation.

The default subclient is automatically created in all the clients with which the subclient policy is associated.

Application Read Size

Application read size is the size of the application data read from the clients for data transfer during backup operations. Values for Application Read Size must be in the power of 2; minimum value is 64 KB, maximum value is 4096 KB (4 MB).

Note that each application internally allocates a buffer size that is suitable for handling the application data. When the size of the application data read during backup operations matches the source application's internal buffer allocation then the overhead is minimized. So to achieve optimal rate of data transfer during backups, configure this value based on the source application's buffer allocation.

The value of application read size may be increased to reduce the amount of data read from the given application, which in turn reduces the amount of I/O operations performed against it. As a result, overall backup performance may increase. However, backup memory usage may also increase, which may inadvertently consume additional resources from the application. Thus, it is recommended that this value be set at either the default value or match the block size directed by the application.

This configuration is supported for the following agents:

  • Microsoft Windows File System Agent
  • Microsoft SQL Server Agent

Modifying Application Read Size

Use these steps to set the application read size:

  1. In the CommCell Browser, expand Client Computers > client > agent > backup set or instance.
  2. Right-click the subclient, and then click Properties.

    The Subclient Properties dialog box is displayed.

  3. For Windows file system subclients, do the following:
    1. Click Advanced.

      The Advanced Subclient Properties dialog box is displayed.

    2. On the Performance tab, select Application Read Size and select the number of kilobytes.
  4. For Microsoft SQL Server subclients, do the following:
    1. On the Storage Device tab, click the Data Transfer Option tab.
    2. Select Application Read Size and select the number of kilobytes.
  5. Click OK.