Release Notes: IBM Aspera Command-Line Interface 3.9.0
Release Notes: IBM Aspera Command-Line Interface 3.9.0

Release Notes: IBM Aspera Command-Line Interface 3.9.0

Product Release: September 28, 2018
Release Notes Updated: October 10, 2018

This release of the IBM Aspera Command-Line Interface (the Aspera CLI) provides the new features, fixes, and other changes listed below. In particular, the Breaking Changes section provides important information about modifications to the product that may require you to adjust your workflow, configuration, or usage. Additional sections cover system requirements and known problems.

This release features substantial changes to the aspera program within the Aspera CLI, including the following:
  • Greatly expanded functionality for the Aspera CLI with IBM Aspera on Cloud (formerly called IBM Aspera Files).
  • Support for Aspera on Cloud transfer service (AoCts), and accompanying options.
  • Support for the node transfer server type, and accompanying options.

These features are described in detail below.

NEW FEATURES

  • New aoc Command

    The files command has been renamed aoc, to match the renaming of Aspera Files to Aspera on Cloud.

    In addition to the change to its name, the aoc command features the following new subcommands. For a complete list of the arguments for each of these commands, see the man page included with the product, or the Aspera Command-Line Interface Guide.
    Subcommand Definition
    browse Browse an IBM Aspera on Cloud directory.
    delete Delete a file in Aspera on Cloud.
    download Download a file or folder from Aspera on Cloud.
    get Download a package from Aspera on Cloud.
    mkdir Create a folder in Aspera on Cloud.
    node list Interact with an Aspera on Cloud node.
    rename Rename a file in Aspera on Cloud.
    upload Upload a file or folder to Aspera on Cloud.
    workspace list Interact with an Aspera on Cloud workspace.
  • New ats Command

    The ats command has been added with this release, to support transfers using the Aspera on Cloud transfer service.

    The ats command features the following new subcommands. For a complete list of the arguments for each of these commands, see the man page included with the product, or the Aspera Command-Line Interface Guide.
    Subcommand Definition
    browse Browse files and directories.
    delete Delete one or more files or directories.
    download Download files or directories.
    help View help for a specific command.
    rename Rename a file or directory.
    upload Upload files or directories.
  • New node Command

    The node command has been added with this release, to support transfers using node transfer servers. This enhancement eliminates the need to use aspera shares commands to interact with node transfer servers.

    The node command features the following new subcommands. For a complete list of the arguments for each of these commands, see the man page included with the product, or the Aspera Command-Line Interface Guide.
    Subcommand Definition
    browse Browse files and directories.
    delete Delete one or more files or directories.
    download Download files or directories.
    help View help for a specific command.
    rename Rename a file or directory.
    upload Upload files or directories.
  • Interactive Prompts

    If you enter a command sequence that lacks required arguments, the Aspera CLI now prompts you for the missing information.

  • Positional Parameters
    • For the aspera aoc command's upload and download subcommands, the source and destination of the transfer are positional parameters. That is, you can form your command sequence without explicitly including the --source and --destination arguments themselves, as long as your source and destination values follow these rules:
      • Source and destination must both be named.
      • Source must be named first, then destination.
      • Source and destination must be named after all other arguments.
      • Source and destination must be named before any ascp arguments (using the new -- syntax).
      If your command sequence omits either source or destination, or inverts their order, the Aspera CLI prompts you for the required information.
    • For the aspera aoc browse command, --path is a positional parameter. You can omit the argument name itself, but the path (to the Aspera on Cloud location that you want to browse) must appear after all other arguments.
  • New faspex Arguments
    The following arguments have been added for use with the aspera faspex commands:
    Argument Definition
    aspera faspex [get|list] --source New option to specify the source server ID or name, as given in the .aspera_cli_conf configuration file.
    aspera faspex send --cc-on-download Notify users when a package is downloaded. You can notify an email address, a Faspex username, or a dropbox.
    aspera faspex send --cc-on-upload Notify users when a package is uploaded. You can notify an email address, a Faspex username, or a dropbox.
  • New shares Arguments
    The following arguments have been added for use with the aspera shares upload command:
    Argument Definition
    -q, --filelist Specify a file that contains a list of files to transfer.
  • Customizing Log Location

    You can now customize the location of Aspera CLI log files. To set the log location, use the -L or --log-path argument with any aspera command. To send log files to stdout, use -L - or --log-path -.

BREAKING CHANGES

If you are upgrading from a previous release, the following changes in this release may require you to adjust your workflow, configuration, or usage.

  • Name of Installer File

    The installer program for the Aspera CLI now includes IBM in its filename. The installer is now named ibm-aspera-cli-productVersion-platformOs-platformVersion-release.fileExtension.

  • Location of Certificates

    The installer program now places a certificates bundle at installation-dir/etc/curl-ca-bundle.crt.

  • Configuration File: Aspera on Cloud Accounts
    In the configuration file .aspera_cli_conf, the section AsperaFiles has been renamed to AoCAccounts. Within that section, the keys now have updated names:
    Key in Prior Releases Key as of This Release
    client_id clientId
    client_secret clientSecret
    private_key_file_name privateKeyFilename
    files_organizations, subkey name organization
    files_organizations, subkey host hostname
  • Configuration File: Faspex Accounts
    • With the introduction of the node command, you no longer include node-related information under your Faspex configuration in the .aspera_cli_conf configuration file. Instead, use aspera node commands with arguments to specify the hostname and authentication (username and password). If your .aspera_cli_conf file includes node-related information, remove it and save the file.
    • The uid key for a Faspex server is now named id. Adjust your configuration file accordingly.
  • Getting Version Information
    The argument for viewing the version number of the Aspera CLI is now --version:
    $ aspera --version
  • Displaying Verbose Output
    The argument for displaying verbose output has changed to match standard Aspera debugging levels. To set the debug level, the options are as follows, for all aspera commands:
    Argument Debug Level
    -D INFO
    -DD DEBUG
    --DDD TRACE
  • Passing ascp Arguments
    The aoc command (formerly files) now uses a different syntax to pass arguments directly to ascp. In the new syntax, you enter the aspera command, subcommand, and arguments first; then the ascp arguments after --. For example:
    $ aspera aoc download --organization org-name --workspace workspace-name source_path destination_path -- [ascp-arguments]
    Note: This feature applies to the aoc command only. In a future release, the ats, faspex, node, and shares commands will be updated to use this syntax as well.
  • Changes to faspex Options

    In order to standardize syntax, several options for faspex commands have changed. Examine any Aspera CLI sequences you may have saved or automated, and adjust them accordingly. Details of the changed options are below:

    Definition in Prior Releases New Definition Migration Notes
    aspera faspex browse aspera node browse To browse a Faspex transfer server, use aspera node browse.
    aspera faspex send -s aspera faspex send --source-id The short-form option -s has been removed; use the long-form option --source-id instead.
    aspera faspex send args --save-before-overwrite [none] The --save-before-overwrite option has been removed in this release.
  • Changes to files Options
    In addition to the change from files to aoc and change in syntax for passing arguments to ascp, the following options have changed. Examine any Aspera CLI sequences you may have saved or automated, and adjust them accordingly.
    Definition in Prior Releases New Definition Migration Notes
    --lsworkspace aspera aoc workspace list Instead of aspera files send --lsworkspace, use the new aspera aoc workspace list subcommand.
    -d --dropbox The short-form equivalent has been removed to avoid a conflict with arguments to ascp.
    -n --name The short-form equivalent has been removed to avoid a conflict with arguments to ascp.
    -s --metadata-file The short-form equivalent has been removed to avoid a conflict with arguments to ascp.

    --check-sshfp

    --cipher

    --disable-encryption

    --encode-jpeg

    --exclude

    --fasp-proxy

    --file-checksum

    --file-encrypt

    --ignore-host-key

    --min-rate

    --move-after-transfer

    --mtu

    --partial-file-suffix

    --precalculate-job-size

    --preserve-dates

    --rate-policy

    --remote-host

    --remove-empty-directories

    --remove-empty-source-directory

    --resume-level

    --rexmsg-size

    -R

    --source-prefix

    --src-base

    --target-rate

    --tcp-port

    --udp-port

    --user

    In prior releases, these were arguments passed to the aspera files subcommands. With the new syntax for passing arguments to ascp, you now pass these arguments in their ascp form, after the -- separator. For ascp reference information, see the Aspera Command-Line Interface Guide.

ISSUES FIXED IN THIS RELEASE

ACLI-217 - The Aspera CLI did not process the -q or --filelist argument correctly.

ACLI-203 - The root certificate bundle has been updated, resolving an issue.

ACLI-202 - The aspera faspex send command includes the following new optional arguments:
Argument Definition
--cc-on-upload Notify the sender when the package has finished uploading.
--cc-on-download Notify the sender when the recipient has downloaded the package.

ACLI -190 - The aspera shares upload command did not use the SSH port that was set in the transfer specification. (CIM-966)

ACLI-141 - Transfers that used the aspera faspex send command failed if you used the -s (or --source_id) option and if the remote source argument for -f was written as just the filename (data.txt) or as the filename prepended with a backslash (\data.txt).

ACLI-44 - When you specified the -b or --base-ca-path option with a filepath variable, the Aspera CLI failed to find the file or directory you specified.

ACLI-17 - You can now set an HTTP proxy for transfers with all aspera commands, using the --http-proxy argument.

ACLI-9 - In ascp log files, --tags have been replaced by --tags64.

SYSTEM REQUIREMENTS

Limitations
  • This release of the Aspera CLI does not support the Shares Management API.
  • This release of the Aspera CLI does not support the Faspex v4 API.
Linux 64-bit:
  • OS versions (glibc 2.9 and higher): RHEL 6.7,7.3, 7.4, CentOS 6-7, Debian 7-9, Fedora 26-27, SLES 11-12, OpenSUSE 42.3, Ubuntu 14.04 LTS, 16.04 LTS, 17.10
  • Required Libraries: OpenSSL 1.0.2g or higher, glib2 2.28 or higher

Mac: OS X 10.10, 10.11, macOS 10.12, 10.13.

Windows: Windows 7, 8, 10, or Windows Server 2008 R2, 2012 R2, 2016.

PACKAGE INFORMATION

Linux 64-bit (rpm): ibm-aspera-cli-3.9.0.1326.6985b21-linux-64-release.sh
md5: 8073111b32c4a01acb3117081d25b501
sha1: 8eedf898d6db4b5c7ffa8738255c7753b348465f
Mac OS X: ibm-aspera-cli-3.9.0.1326.6985b21-mac-10.7-64-release.sh
md5: 8a3c2ae552f96b13bc561fab4bc7ef94
sha1: 9aa195656d4b352ea570e223a09ef336d39cbd8d
Windows: ibm-aspera-cli-3.9.0.1326.6985b21-win-v140_xp-32-release.zip
md5: a836d1016dffea088bbc1e098280a440
sha1: 923b5d9f52fb7e9dbfc4d7794dac254cebd8e65c

KNOWN ISSUES

Note: This version of the Aspera CLI was developed using two different issue-tracking systems; so the list below uses two different formats for issue numbers.

aspera

ACLI-71 - [Mac OS X only] If you install the Aspera CLI to a location to which you lack permissions, the installation fails, but the installer nevertheless reports a successful installation.

ACLI-61 - [Windows only] When you specify a filepath using the %homepath% variable, the Aspera CLI action fails.

ACLI-48 - When you use the faspex list subcommand with the -x or --xml option for XML-formatted content, there is an issue with the formatting of the URLs that are returned. As a result, when you download the XML content with faspex get, the following error is returned:
Cannot parse an empty string
Workaround: Remove XML escape sequences (either manually or with an XML parser) from URLS before passing them as input to a faspex get subcommand.
ACLI-43 - When a Shares command is run on a share that the user does not have authorization for, the following error is returned:
JSON Exception: Error: Path not found
The error does not indicate that authorization is denied.
#35287 - Transfers from the local computer to the Azure default inbox fail with the following error:
Setting up FASP parameters... Invalid access: Cannot convert empty value.

ascp

ATT-657 - Multi-session transfers that use -k1, or multi-session transfers from cloud storage to local storage, can result in files on the destination that are missing bytes. This occurs when a file is split between sessions; the first session creates matching attributes for the source and destination (partial) file, and the other sessions do not recognize that there are more bytes for transfer. Workaround: For multi-session downloads from cloud storage, use --overwrite=always (to prevent resuming file transfers) or --multi-session-threshold=0 (to prevent file-splitting across sessions).

ATT-613 - When uploading files to a HST Server on a Windows computer, if the file name contains a carriage return character ("\r"), then the transfer fails. (CIM-1205)

ATT-537 - Downloads that use ascp --overwrite=always fail when they are authenticated using ASPERA_LOCAL_TOKEN that specifies a local storage path.

ATT-435 - save-before-overwrite is not supported for URI

ATT-395 - When running a persistent ascp session, a FaspManager FILEERROR message truncates a filename that is longer than 128 characters to only the first 128 characters.

ATT-361 - ascp transfers to S3 fail when the --symbolic-link=copy or --symbolic-link=copy+force option is used.

ATT-360 and NODE-545 - Directory timestamps are not always preserved on the destination during an ascp transfer that uses -p.

ATT-226 - If the docroot is a URL path, ascp reports incorrect bytes for the sessions that are involved in a multi-session transfer.

ATT-185 - ascp does not reconnect to Redis database when asperanoded is restarted.

ES-645 - The ascp -@ option is not supported when the destination is stdio://.

ES-626 - As of 3.8.0, ascp truncates JSON tags if the tags exceed 4 Kb. With this fix, tag length is checked before the transfer is started and an error is returned if the tags exceed 4 Kb.

ES-624 - [Windows] No files are transferred if the argument for --src-base includes a semicolon (";") but the transfer is reported as successful.

ES-359 - ascp downloads from SoftLayer do not support --move-after-transfer.

ES-267 - Under rare conditions, ascp transfers to cloud object storage may be reported as successful even though Trapd reports an error and the content is not in the storage. (CIM-475)

ES-177 - The range_low value of a -@ argument is not respected.

ES-96 (#33091) - [Windows] Downloading files or folders that have names that end in a trailing space fail.

#35010 - If the source path in an ascp transfer is a file that is named \ (which is not supported by Aspera), the file is not transferred and an error is generated, but the folder then contains the file and all other files in that folder are transferred.

#34322 - [Linux CentOS 7.2] ascp fails to authenticate SSH with a large banner file size (approximately 2000 bytes).

#32890 - During an ascp transfer that uses the --preserve-xattrs= metafile --remote-preserve-xattrs=metafile options, the metafile is not transferred.

#32680 - The option to create a directory (ascp -d) may create a directory at a destination before an expected session failure.

#30324 - During an ascp upload to cloud storage, if a mid-file read failure occurs on the sending computer (which is rare) it can cause the server-side ascp to crash and possibly fail to report transfer completion. This read failure can be caused when a source file is truncated during transfer, a drive or file system fails, or a transfer is canceled with Ctrl+C or other means.

#30231 - [Windows] If a pre-post script includes a path that contains ".." and the client is running on Windows, ascp fails with the following message:
Cannot open -e pre-post command: Unknown error

#29043 - [Windows] Under serious WAN impairment, Hot Folder push transfers with a prepost script intermittently report an asssh error approximately 10-15 seconds after Transfer Statistics are reported to the log. This delay in process shutdown can hold the handle to the file list such that subsequent ascp processes fail.

#28939 - If command line ascp neglects to specify a destination host, then the failed transfer (error: "no remote host specified") gets recorded in SQLite with client_node_id NULL, instead of being populated with the uuid of the node. This database error causes an issue with Console.

#26281 - If you run approximately 100 (or a similarly high number) concurrent uploads to S3, intermittent transfer session failures can occur.

#26185 - During an upload to S3 storage, an error may result if ascp reports a successful file transfer before the transfer to S3 completes.

ascp4

ATT-717 - Persistent Ascp4 sessions do not report SSH errors (such as an invalid hostname, username, password, port, or file list) to management.

ATT-637 - As of 3.7.4, an empty folder is created on the destination even when the source folder is excluded from the transfer (by using -E /folder/pathname).

ATT-621 - As of 3.8.0, Ascp 4 is not backward compatible.

ATT-583 - Ascp 4 does not automatically create a destination folder when the source is a file list and the destination does not exist. Instead, it writes all the files in the file list into one file. (CIM-1198)

ATT-582 - Ascp 4 sessions run with -d and a file list do not report an error if the destination already exists and is not a folder. (CIM-1199)

ATT-545 - Ascp 4 downloads all content from an AWS S3 docroot, rather than the specified content, if the docroot contains ?storage-class=REDUCED_REDUNDANCY.

ATT-515 - When ascp4 is used by the GUI and transfers are encrypted with AES-128, the GUI incorrectly shows that encryption is "none". (CIM-953)

ATT-485 - Persistent session Ascp 4 downloads from object storage do not report a STOP message to management after the transfer completes.

ATT-477 - When files are transferred to a server with an S3 docroot and quickly retransferred with the --delete-before-transfer enabled, some files are deleted from the destination.

ATT-473 - Ascp 4 uploads to object storage that specify -k 1 (resume if file sizes match) are also sensitive to checksum, such that if a file transfer is resumed and the file has the same size but a different checksum then the entire file is retransferred, rather than resumed from the last successful chunk.

ATT-451 - Ascp 4 does not respect exclude filters if the file path is part of the command line.

ATT-438 - Ascp 4 downloads from object storage fail if the source filename contains special Unicode characters, such as Japanese font.

ATT-432 - [Linux Ubuntu] When downloading files from a server by using ascp4, if a docroot is configured for the transfer user and multiple source files are specified on the command line then only the first file is downloaded.

ATT-428 - During a persistent ascp4 session, when it a file transferred to a non-existant path and -d is used, the file transfers successfully and the destination path is created but the file is renamed to the last element of the destination path instead of being placed inside.

ATT-409 - If a file list contains an invalid path, no error is reported or logged.

ATT-338 - Parallel uploads of several large (>1 GB) files to object or HDFS storage may fail with the error "Peer aborted session" if the number of threads that are specified in the ascp4 command exceeds the number of jobs that are allowed to run by Trapd. Workaround: Open /opt/aspera/etc/trapd/trap.properties and set the value for aspera.session.upload.max-jobs to one larger than the number of ascp4 threads. For example,
# Number of jobs allowed to run in parallel for uploads.
            # Default is 15
            aspera.session.upload.max-jobs=50

ATT-298 - [Windows] When the destination has several million files, the source has few files, and <worker_threads> is set to 64 in aspera.conf, a download run with the --delete-before or --delete-after option can take several hours to delete files and might time out before finishing the deletion.

ATT-191 - [Linux] Symbolic links are not updated on the destination when the symbolic link option is follow (the default value when none is set) or copy.

ATT-186 - An ascp4 multicast session does not fail if the multicast IP address and port is already in use on the receiver.

ATT-29 - Files that are transferred to S3 storage with ascp4 retain a .partial extension when viewed in the GUI.

ATT-2 (#32295) - The default minimum transfer rate set in aspera.conf is not respected.

ES-247 - Console-initiated ascp4 transfers fail if the docroot on the source is a UNC path (for example, \\localhost\SHARE), returning the error ERR Source base/path is not a valid directory/file (doesn't match any source path). (CIM-397)

ES-151 - ascp4 does not recognize the UNC-path docroot of a Console transfer user. (CIM-197)

PRODUCT SUPPORT

For on-line support resources for Aspera products, including raising new support tickets, please visit the Aspera Support Portal. Note that you may have an existing account if you contacted the Aspera support team in the past. Before creating a new account, first try setting a password for the email that you use to interact with us. You may also call one of our regional support centers.