Release Notes: IBM Aspera Command-Line Interface 3.7.6
Release Notes: IBM Aspera Command-Line Interface 3.7.6

Release Notes: IBM Aspera Command-Line Interface 3.7.6

Product Release: October 16, 2017
Release Notes Updated: October 23, 2017

IBM Aspera Command-Line Interface (the Aspera CLI) is a collection of Aspera tools for performing high-speed, secure data transfers from the command line. The Aspera CLI is for users and organizations who want to automate their transfer workflows.

The Aspera CLI is comprised of three command-line programs:
  • aspera, a command-line client for performing transfers with Aspera Faspex, Aspera Files, and Aspera Shares transfer servers.
  • The ascp command-line FASP transfer program.
  • ascp4, or A4, a FASP transfer program similar to ascp that has been optimized for sending large sets of individual files and can support UDP multicast through Aspera FASPStream.
Note: This release of the Aspera CLI does not support Shares 2.0.x. Support for Shares 2.0.x will be added in a future release.
Note: This release of the Aspera CLI does not support the Shares Management API or the Faspex v4 API.

NEW FEATURES

This release greatly expands the options available for the Aspera CLI with Faspex, Files, and Shares:
  • The Aspera CLI now includes the following additional options for the faspex send subcommand:
    --fasp-proxy=proxy_hostOrIp The hostname or IP address of the proxy computer (forward proxy).
    --file-encrypt Encrypt files. To use this option, you must first set the encryption/decryption passphrase with the ASPERA_SCP_FILEPASS environment variable.
    --ignore-host-key Ignore the server's SSH host key fingerprint.
    -j

    --encode-jpeg

    Encode all HTTP transfers as JPEG files.
    -k resume_level

    --resume-level=resume_level

    Enable the resuming of partially transferred files at the specified resume level.
    resume_level can have the following values:
    • 0 (default) - Always retransfer the entire file.
    • 1 - Compare file attributes and resume if they match, and retransfer if they do not.
    • 2 - Compare file attributes and the sparse file checksums; resume if they match, and retransfer if they do not.
    • 3 - Compare file attributes and the full file checksums; resume if they match, and retransfer if they do not.
    -l new_rate Attempt to revise the target rate (if server settings allow) to a new throughput value, in kbps.
    -L path

    --log-path=path

    The path to the logfile.
    -m new_rate Attempt to revise the minimum rate (if server settings allow) to a new throughput value, in kbps.
    --move-after-transfer=path Specify the directory where the file will be moved to after it is transferred.
    -O port_num

    --udp-port=port_num

    Set the UDP port for FASP transfers.
    --preserve-dates Preserve the timestamps on transferred files.
    -P port_num

    --tcp-port=port_num

    Set the TCP port for FASP transfers.
    --partial-file-suffix=suffix Set a suffix to append to the filenames of partially transferred files.
    --remove-empty-source-directory When the transfer is complete, remove empty source directories.
    --remote-host=host_address The remote host address.
    --save-before-overwrite Save a copy of a file before it is overwritten by the transfer. A copy of filename.ext is saved as filename.yyyy.mm.dd.hh.mm.ss.index.ext in the same directory.
    --src-base=prefix Strip the specified prefix from each source path. The remaining portion of the source path is kept intact at the destination.
    -T

    --disable-encryption

    Disable encryption.
    --user=username The username of the Aspera transfer user.
    --username=username The Faspex username.
    -Z datagram_size

    --mtu=datagram_size

    Set the MTU manually, in bytes.
  • The Aspera CLI now includes the following additional options for the files send subcommand:
    --check-sshfp=fingerprint Compare fingerprint to the server SSH host key fingerprint that is set with ssh_host_key_fingerprint in aspera.conf.
    --cipher=cipher Attempt to set the encryption cipher (if server settings allow).
    cipher can have the following values:
    • aes-128
    • aes-192
    • aes-256
    • none
    -e

    --remove-empty-directories

    When the transfer is complete, remove empty directories.
    -E pattern

    --exclude=pattern

    Exclude files that match the given pattern. To specify multiple patterns, repeat the -E option.
    --fasp-proxy=proxy_hostOrIp The hostname or IP address of the proxy computer (forward proxy).
    --file-checksum=hash Enable checksum reporting for transferred files.
    hash is the type of checksum to calculate:
    • none (default)
    • md5
    • sha1
    • sha-256
    • sha-384
    • sha-512
    --file-encrypt Encrypt files. To use this option, you must first set the encryption/decryption passphrase with the ASPERA_SCP_FILEPASS environment variable.
    --ignore-host-key Ignore the server's SSH host key fingerprint.
    -j

    --encode-jpeg

    Encode all HTTP transfers as JPEG files.
    -k resume_level

    --resume-level=resume_level

    Enable the resuming of partially transferred files at the specified resume level.
    resume_level can have the following values:
    • 0 (default) - Always retransfer the entire file.
    • 1 - Compare file attributes and resume if they match, and retransfer if they do not.
    • 2 - Compare file attributes and the sparse file checksums; resume if they match, and retransfer if they do not.
    • 3 - Compare file attributes and the full file checksums; resume if they match, and retransfer if they do not.
    -l new_rate

    --target-rate=new_rate

    Attempt to revise the target rate (if server settings allow) to a new throughput value, in kbps.
    -L path

    --log-path=path

    The path to the logfile.
    -m new_rate

    --min-rate=new_rate

    Attempt to revise the minimum rate (if server settings allow) to a new throughput value, in kbps.
    --move-after-transfer=path Specify the directory where the file will be moved to after it is transferred.
    -n package_name

    --name=package_name

    A name for the package.
    --message=body_text [Optional] Text for the body of the email message.
    -O port_num

    --udp-port=port_num

    Set the UDP port for FASP transfers.
    --preserve-dates Preserve the timestamps on transferred files.
    -P port_num

    --tcp-port=port_num

    Set the TCP port for FASP transfers.
    --partial-file-suffix=suffix Set a suffix to append to the filenames of partially transferred files.
    --precalculate-job-size Precalculate the size of the transfer, for a progress bar.
    -R

    --remove-after-transfer

    When the transfer is complete, remove the transferred content from the source.
    --rate-policy=policy Attempt to revise the rate policy (if server settings allow).
    The options for policy are
    • fixed
    • high
    • fair
    • low
    --remote-host=host_address The address of the remote host.
    --remove-empty-source-directory When the transfer is complete, remove empty source directories.
    --source-prefix=prefix_path Set a path to be prepended to each source path.
    --src-base=prefix Strip the specified prefix from each source path. The remaining portion of the source path is kept intact at the destination.
    -T

    --disable-encryption

    Disable encryption.
    --user=username The username of the Aspera transfer user.
    --username=username The Files username (an email address).
    -X size

    --rexmsg-size=size

    Set the size of retransmit requests, in bytes.
    -Z datagram_size

    --mtu=datagram_size

    Set the MTU manually, in bytes.
  • The Aspera CLI now includes the following additional options for the shares upload subcommand:
    --fasp-proxy=proxy_hostOrIp The hostname or IP address of the proxy computer (forward proxy).
    --file-encrypt Encrypt files. To use this option, you must first set the encryption/decryption passphrase with the ASPERA_SCP_FILEPASS environment variable.
    --ignore-host-key Ignore the server's SSH host key fingerprint.
    -j

    --encode-jpeg

    Encode all HTTP transfers as JPEG files.
    -k resume_level

    --resume-level=resume_level

    Enable the resuming of partially transferred files at the specified resume level.
    resume_level can have the following values:
    • 0 (default) - Always retransfer the entire file.
    • 1 - Compare file attributes and resume if they match, and retransfer if they do not.
    • 2 - Compare file attributes and the sparse file checksums; resume if they match, and retransfer if they do not.
    • 3 - Compare file attributes and the full file checksums; resume if they match, and retransfer if they do not.
    -l new_rate

    --target-rate=new_rate

    Attempt to revise the target rate (if server settings allow) to a new throughput value, in kbps.
    -L path

    --log-path=path

    The path to the logfile.
    -m new_rate Attempt to revise the minimum rate (if server settings allow) to a new throughput value, in kbps.
    --move-after-transfer=path Specify the directory where the file will be moved to after it is transferred.
    -O port_num

    --udp-port=port_num

    Set the UDP port for FASP transfers.
    --preserve-dates Preserve the timestamps on transferred files.
    -P port_num

    --tcp-port=port_num

    Set the TCP port for FASP transfers.
    --partial-file-suffix=suffix Set a suffix to append to the filenames of partially transferred files.
    --remote-host=host_address The remote host address.
    --remove-empty-source-directory When the transfer is complete, remove empty source directories.
    --save-before-overwrite Save a copy of a file before it is overwritten by the transfer. A copy of filename.ext is saved as filename.yyyy.mm.dd.hh.mm.ss.index.ext in the same directory.
    --src-base=prefix Strip the specified prefix from each source path. The remaining portion of the source path is kept intact at the destination.
    -T

    --disable-encryption

    Disable encryption.
    --user=username The username of the Aspera transfer user.
    --username=username The Shares username.
    -Z datagram_size

    --mtu=datagram_size

    Set the MTU manually, in bytes.

BREAKING CHANGES

If you are upgrading from a previous release, the following changes in this release may require you to adjust your workflow, configuration, or usage.

In order to match ascp and ascp4 syntax, several previous options for the aspera component have changed. Examine any Aspera CLI sequences you may have saved or automated, and adjust them accordingly. Details of the changed options are below:
Changed Aspera CLI Command Option Definition in Prior Releases New Definition Migration Notes
aspera faspex send -m Send metadata (JSON object text) with the package. Attempt to revise the minimum rate (if server settings allow) to a new throughput value, in kbps. For metadata, use --metadata instead.
aspera faspex send --proxy The hostname or IP address of the proxy computer (forward proxy). Has been replaced by the --fasp-proxy option. Use the --fasp-proxy option.
aspera faspex send -T The listening port on the Faspex server. Disable encryption. For the the Faspex port number, use --port instead.
aspera faspex send --user The Faspex username. Has been replaced by the --username option. For the Faspex username, use --username instead.
aspera faspex send -x The hostname or IP address of the proxy computer (forward proxy). The hostname or IP address of the HTTP proxy computer. For a FASP proxy, use --fasp-proxy instead.
aspera files send -m Text for the body of the email message. Attempt to revise the minimum rate (if server settings allow) to a new throughput value, in kbps. For the email message, use --message instead.
aspera files send --user The Files username (an email address). The username of the Aspera transfer user. For the Files username, use --username instead.
aspera shares upload --proxy The hostname or IP address of the proxy computer (forward proxy). Has been replaced by the --fasp-proxy option. For a FASP proxy, use --fasp-proxy instead.
aspera shares upload -T The listening port on the Shares server. Disable encryption. For the Shares port, use --port instead.
aspera shares upload --user The Shares username. The username of the Aspera transfer user. For the Shares username, use --username instead.
aspera shares upload -x The hostname or IP address of the proxy computer (forward proxy). The hostname or IP address of the HTTP proxy computer. For a FASP proxy, use --fasp-proxy instead.

SYSTEM REQUIREMENTS

Linux 64-bit: RedHat 6-7, CentOS 6-7, Fedora 16-20, Ubuntu 12-14, Debian 6-7, SLES 11-12 Kernel 2.4 or higher; and libc version GLIB 2.3.4-2.40.

Mac: OS X 10.8, 10.9, 10.10, 10.11, macOS 10.12, 10.13.

Windows: Windows 7, 8, 10, or Windows Server 2008 R2, 2012 R2, 2016.

PACKAGE INFORMATION

Linux 64-bit (rpm): aspera-cli-3.7.6.636.4d51a47-linux-64-release.sh
md5: 12fee0d5965237ea8be056f331a7d9b1
sha1: 7090ca5869bf19bb8c60d33c2e9e08c0bab8c404
Mac OS X: aspera-cli-3.7.6.636.4d51a47-mac-10.7-64-release.sh
md5: b9ca7fc9fc7140577e2af8318b3fa487
sha1: 7ee5660d5f25f3df8bb214c819d0f797ccf0cea0
Windows: aspera-cli-3.7.6.636.4d51a47-win-v140_xp-32-release.zip
md5: 939f52c4a5966076420e94b61d0f4a92
sha1: 82f01df857dfe0ec11fbfef4dc0f99c89f0738a9

KNOWN ISSUES

Note: This version of the Aspera CLI was developed using two different issue-tracking systems; so the list below uses two different formats for issue numbers.

aspera

ACLI-141 - Transfers that use the aspera faspex send command fail if you use the -s (or --source_id) option and if the remote source argument for -f is written as just the filename (data.txt) or as the filename prepended with a backslash (\data.txt). Workaround: Enter the filename prepended with a forward slash (/data.txt).

ACLI-93 - [Windows only] For systems using PowerShell, you must use the following syntax to set the password as an environment variable:
> $env:ASPERA_PASS = "password"
When you set an environment variable for your password, your Aspera CLI command sequences will not require the -p option to specify the password.

ACLI-71 - [Mac OS X only] If you install the Aspera CLI to a location to which you lack permissions, the installation fails, but the installer nevertheless reports a successful installation.

ACLI-61 - [Windows only] When you specify a filepath using the %homepath% variable, the Aspera CLI action fails.

ACLI-48 - When you use the faspex list subcommand with the -x or --xml option for XML-formatted content, there is an issue with the formatting of the URLs that are returned. As a result, when you download the XML content with faspex get, the following error is returned:
Cannot parse an empty string
Workaround: Remove XML escape sequences (either manually or with an XML parser) from URLS before passing them as input to a faspex get subcommand.

ACLI-44 - When you specify the -b or --base-ca-path option with a filepath variable, the Aspera CLI fails to find the file or directory you specify.

ACLI-43 - When a Shares command is run on a share that the user does not have authorization for, the following error is returned:
JSON Exception: Error: Path not found
The error does not indicate that authorization is denied.

ACLI-18 - The Aspera CLI does not HTTP fallback. This support will be added in a future release.

#35287 - Transfers from the local computer to the Azure default inbox fail with the following error:
Setting up FASP parameters... Invalid access: Cannot convert empty value.

ascp

ATT-470 - When using HTTP fallback, downloads with in-line decryption sometimes fail after an upload to a server with EAR.

ATT-405 - When monitoring an ascp transfer, the FASP2 Management Protocol reports inconsistent WrittenBytes, FileBytes, and TransferredBytes.

ATT-395 - When running a persistent ascp session, a FaspManager FILEERROR message truncates a filename that is longer than 128 characters to only the first 128 characters.

ATT-361 - ascp transfers to S3 fail when the --symbolic-link=copy or --symbolic-link=copy+force option is used.

ATT-360 - Directory timestamps are not always preserved on the destination during an ascp transfer.

ATT-226 - If the docroot is a URL path, ascp reports incorrect bytes for the sessions that are involved in a multi-session transfer.

ATT-205 - ascp transfer fails with an internal memory error if <network_rc><module> is set to air in the <bandwidth> section of aspera.conf.

ATT-189 - In rare cases, ascp keeps running after it encounters a disk read error. (CIM-233)

ATT-185 - ascp does not reconnect to Redis database when asperanoded is restarted.

ATT-134 - [Windows, Linux]asdelete and ascp --delete-before-transfer commands that are initiated from Windows machines fail to delete files that are named with ASCII characters (< | ? * > : " \ ) on Linux machines.

ES-349 - When ascp is run with resume enabled (-k {1|2|3}), --preserve-file-owner-uid and --preserve-file-owner-gid are ignored.

ES-267 - Under rare conditions, ascp transfers to cloud object storage may be reported as successful even though Trapd reports an error and the content is not in the storage. (CIM-475)

ES-177 - The range_low value of a -@ argument is not respected.

ES-96 (#33091) - [Windows] Downloading files or folders that have names that end in a trailing space fail.

#35010 - If the source path in an ascp transfer is a file that is named \ (which is not supported by Aspera), the file is not transferred and an error is generated, but the folder then contains the file and all other files in that folder are transferred.

#34322 - [Linux CentOS 7.2] ascp fails to authenticate SSH with a large banner file size (approximately 2000 bytes).

#33094 - The ascp option delete-before-transfer is not supported for URI storage.

#32890 - During an ascp transfer that uses the --preserve-xattrs= metafile --remote-preserve-xattrs=metafile options, the metafile is not transferred.

#32680 - The option to create a directory (ascp -d) may create a directory at a destination before an expected session failure.

#32553 - When the FASP Session log source file list exceeds 500 bytes and contains multibyte UTF-8 characters, the output is truncated in a manner that creates an invalid UTF-8 sequence.

#31423 - It is possible for an ascp transfer of a file on a full disk to be reported as successful by both the sender and the receiver.

#30324 - During an ascp upload to cloud storage, if a mid-file read failure occurs on the sending computer (which is rare) it can cause the server-side ascp to crash and possibly fail to report transfer completion. This read failure can be caused when a source file is truncated during transfer, a drive or file system fails, or a transfer is canceled with Ctrl+C or other means.

#30231 - [Windows] If a pre-post script includes a path that contains ".." and the client is running on Windows, ascp fails with the following message:
Cannot open -e pre-post command: Unknown error

#29255 - Download from SoftLayer of a file larger than 62 GB is unsuccessful. Workaround: Do not use time-stamp preservation with SoftLayer.

#29043 - [Windows] Under serious WAN impairment, Hot Folder push transfers with a prepost script intermittently report an asssh error approximately 10-15 seconds after Transfer Statistics are reported to the log. This delay in process shutdown can hold the handle to the file list such that subsequent ascp processes fail.

#28939 - If command line ascp neglects to specify a destination host, then the failed transfer (error: "no remote host specified") gets recorded in SQLite with client_node_id NULL, instead of being populated with the uuid of the node. This database error causes an issue with Console.

#26281 - If you run approximately 100 (or a similarly high number) concurrent uploads to S3, intermittent transfer session failures can occur.

#26185 - During an upload to S3 storage, an error may result if ascp reports a successful file transfer before the transfer to S3 completes.

#25865 - Allowing symbolic links to be copied also allows access to locations outside the docroot.

#23503 - Uploads of zero-byte files to Akamai appear to be successful, but no file is present at the destination.

#22905 - When you copy a file in S3 storage with ascp, if a slash is appended to the destination -- for example, /path/ -- the file is renamed path/. Because of the trailing slash, it appears to be a directory, but is actually a file.

A4

ATT-438 - A4 downloads from object storage fail if the source filename contains special Unicode characters, such as Japanese font.

ATT-433 - [Windows] An ascp4 transfer that uses the --file-list option fails. (CIM-433) Workaround: Add an extra blank line to the end of the file list. For example,
c:\project\images
c:\project\docs

ATT-432 - [Linux Ubuntu] When downloading files from a server by using ascp4 in receive mode, if a docroot is configured for the transfer user and multiple source files are specified on the command line then only the first file is downloaded.

ATT-426 - To use SSH key authentication in A4, the full path to the private key must be specified as the argument for -i because A4 does not automatically look in the transfer user's home folder. This prevents Aspera Console from being able to use SSH keys if it is configured to use A4 instead of ascp because the full path cannot be specified in the Console UI.

ATT-366 - An ascp4 transfer to object storage does not fail immediately if chunk size is configured incorrectly. Workaround: On the client command line, set the chunk size equal to the size of the shared memory buffers used by Trapd with the --chunk-size option. By default, shared memory buffers are 1 MB, in which case --chunk-size=1048576 should be used.

ATT-338 - Parallel uploads of several large (>1 GB) files to object or HDFS storage may fail with the error "Peer aborted session" if the number of threads that are specified in the ascp4 command exceeds the number of jobs that are allowed to run by Trapd. Workaround: Open /opt/aspera/etc/trapd/trap.properties and set the value for aspera.session.upload.max-jobs to one larger than the number of ascp4 threads. For example,
# Number of jobs allowed to run in parallel for uploads.
            # Default is 15
            aspera.session.upload.max-jobs=50

ATT-321 - To use ascp4 to transfer with object storage, you must set the chunk size on the server to 64 kb for transfers that include primarily small files, and set it to 1 Mb for transfers that include primarily large files. If the chunk size is not set on the server, then the transfer fails.

ATT-298 - [Windows] When the destination has several million files, the source has few files, and <worker_threads> is set to 64 in aspera.conf, a download run with the --delete-before or --delete-after option can take several hours to delete files and might time out before finishing the deletion.

ATT-191 - [Linux] Symbolic links are not updated on the destination when the symbolic link option is follow (the default value when none is set) or copy.

ATT-44 - The timestamp of a parent directory that is transferred with ascp -p is not preserved, while the timestamps of the child files and directories are preserved.

ATT-39 - When ascp4 is run with the --no-write option, a 0-byte file and an .aspx (partial) file are created at the destination.

ATT-30 and ATT-46 - ascp4 transfer is slow when you upload many small files (for example, 1 million 4-byte files) to S3 storage.

ATT-29 - Files that are transferred to S3 storage with ascp4 retain a .partial extension when viewed in the GUI.

ATT-27 - Direct-to-cloud ascp4 transfers are skipped unless the full destination path is specified.

ATT-2 (#32295) - The default minimum transfer rate is not picked up from aspera.conf.

ES-247 - Console-initiated ascp4 transfers fail if the docroot on the source is a UNC path (for example, \\localhost\SHARE), returning the error ERR Source base/path is not a valid directory/file (doesn't match any source path). (CIM-397)

ES-151 - ascp4 does not recognize the UNC-path docroot of a Console transfer user. (CIM-197)

#29613 - [Windows] Timestamps are not preserved on top-level directories.

#26484 - A4 through Aspera Proxy is not supported.

PRODUCT SUPPORT

For on-line support resources for Aspera products, including raising new support tickets, please visit the Aspera Support Portal. Note that you may have an existing account if you contacted the Aspera support team in the past. Before creating a new account, first try setting a password for the email that you use to interact with us. You may also call one of our regional support centers.