Configuring IBM Aspera as a data transfer tool
IBM Aspera is a data transfer tool that makes efficient, policy-based use of network bandwidth in high latency networks.
About this task
Aspera can be used to transfer data between data sources and the staging area. Using Aspera to transfer data between the staging area and the execution host is not supported. In common configurations, the network that connects the data staging area and the execution hosts is fast enough that data transfer speed isn’t a performance concern. The data transfer nodes (I/O nodes) are Aspera clients, which initiate all file transfers. The external data repositories (the data source and data destination hosts) are Aspera servers. Aspera uses SSH public keys for non-interactive authentication. Refer to the Aspera documentation for information about how to generate and configure SSH keys.
LSF data manager can work with any data transfer tool that supports a non-interactive command-line interface. The data transfer tool is configured by the parameter FILE_TRANSFER_CMD in the lsf.datamanager file. The argument to this parameter must be a single executable command. Passing command arguments by configuring the arguments directly in the parameter isn’t supported. The transfer command is run with the same user account as the job submission user.
For more information, see Data transfer job script interface.
The following steps show how to set up a simple integration for data manager file transfer that uses IBM Aspera: