Release Notes: IBM Aspera Orchestrator 4.0.0
Release Notes: IBM Aspera Orchestrator 4.0.0

Release Notes: IBM Aspera Orchestrator 4.0.0

Product Release: June 30, 2020
Release Notes Updated: June 30, 2020

This release of IBM Aspera Orchestrator 4.0.0 provides the new features, fixes, and other changes listed below. In particular, the Breaking Changes section provides important information about modifications to the product that may require you to adjust your workflow, configuration, or usage. Additional sections cover system requirements and known problems.

NEW FEATURES

Improved Performance

  • Upgraded MySQL from 5.1 to 5.7.
  • Added indexing to improve performance during frequent queries from high-volume users.
  • High-performance polling for new steps in a work order is now enabled by default.

Security Improvements

  • When changing the password of another user, the value of the Current User's Password field is now the password of the admin user who is making the change.
  • After installation, the system user is no longer enabled by default. An admin must activate the system user manually.
  • There is improved logging for SAML issues.

Updates to the Orchestrator UI

  • Logo, naming, color schemes and other UI elements have been updated throughout the application to improve usability and conform with IBM Carbon standards.
  • The work order and work step details remain collapsed or expanded.
  • The left navigation menu remains collapsed or expanded during while the UI is open.

General

  • A new user permission, Aspera:Orchestrator:JournalEntries:Edit allows admins to control which users and groups have access to the Delete button for journal entries. For more information on how to add permissions, see "Viewing and Granting Permissions" in the Admin Guide.
  • There are two new properties in the Orchestrator configuration:
    • recent_workflow_count - returns the specified number of workflows. The default value is 10.
    • minimal_xml_escaping - specifies how characters in XML responses for the Orchestrator endpoints are escaped. If false, all characters that are conventionally escaped in XML—according to best practices—are escaped. If true, only "<'" and "&" are escaped.
  • The WorkStep Activity portlet now allows the user to specify which attribute to use for entry ordering. By default the ordering is done using the updated_at field.
  • The license page now displays the license information entered by an admin user. (CIM-3024)
  • Maintenance processes are now able to get updated values from the Orchestrator configuration file (/opt/aspera/var/config/orchestrator/orchestrator.yml) without the user having to run asctl orchestrator:restart.

Plugins

  • New plugins in Orchestrator include:
    • Amazon MediaConvert - Transcodes video content. The plugin can be used with a custom payload.
    • Amazon Transcribe - Sends audio and video to the AWS Amazon Transcribe service and receives transcribed data in JSON format.
    • Aspera OTFV (Out of Transfer File Validation) - Detects validation requests for both transfer sessions and files and allows you to implement a validation workflow in Orchestrator.
    • Custom Python - Users can now write and run their own Python code in Orchestrator.
    • Facebook Upload - Uploads videos to Facebook.
    • Google Cloud Pub/Sub - Uses the Google Cloud Pub/Sub messaging service.
    • Google Cloud Storage - Uses the Google Cloud Storage service.
    • Hybrik Transcoder - Transcodes your digital content.
    • Jira Issue Creation - Creates Jira tickets.
    • Kafka Stream - Writes workflow execution data to an external monitoring system.
    • Softlayer API - Accesses the SoftLayer Network Storage service, which adds control over IBM cloud storage.
    • Watson Language Translator - Accesses the Watson Language Translator service.
    • Watson Speech to Text - Gets speech transcriptions with the Watson Speech to Text service.
  • For a step using the Email Notification plugin, it is now possible to disable the mailer configuration-if, for example, you don't want to send out emails while testing the workflow-and still have the step complete successfully.
  • The Amazon S3 plugin now allows users to list all objects within a given bucket. (CIM-2398)
  • The Node API plugin has a new option: Verbose logging. If selected, all requests and responses to and from Node (polling, as well as job submission) print to the log.
    Note: Using this option can potentially crowd your log file. IBM Aspera strongly recommends that you only enable this option for debugging.
  • The Media Mate Plugin now uses the model path in the Run operation .

Work Orders

  • On the Work Orders page, when Cancel All is clicked, the user now gets a dialog asking for confirmation that all work orders should be canceled. (CIM-3173)

API

New endpoints in Orchestrator are:
  • Publish workflows: /publish_workflow lets you publish a workflow so it can launch running instances (work orders).
  • Enable plugins: /enable_plugin lets you enable plugins automatically.
  • Import workflows: /find_constraints and /import_with_constraints work together to let you import workflows with their associated dependencies (such as sub-workflows and remote nodes).
  • Get workflow steps: /retrieve-steps gets all workflow steps—across all workflows—that use the same plugin, along with their associated work orders.
  • Delete queued elements - /delete_queued deletes an element from a managed queue.
  • The /workstep_reset endpoint now allows you to restart a work order from a specific step, or restart multiple work orders from a specific step in each work order. (CIM-2373)

Workflows

  • You can now configure your Orchestrator instance to work with Orchestrator's GitHub integration, so that when you update a workflow, you can push your changes to Github from your development instance or pull updates from Github to your production instance.

Breaking Changes

If you are upgrading from a previous release, the following changes in this release may require you to adjust your workflow, configuration, or usage.
  • There are new default configuration values in this release for cleanup_cutoff (60), journal_cleanup_cutoff (10), journal_purge_cutoff (30), and purge_cutoff (30). IBM Aspera strongly recommends that you enter the smallest number that is compatible with business requirements instead of relying on the default values. If you have a large volume of work orders and limitations on the capacity of your database, the default value is probably too high for your system and could lead to system failure. Additionally, you might experience slow performance immediately after an upgrade because-potentially-Orchestrator could start purging a high volume of data.

    You should watch your database and adjust this value as needed for optimal functioning of the system.

  • In part because of the upgrade to MySQL 5.7, the Orchestrator upgrade procedure requires a number of additional steps and verifications. For detailed information, see "Upgrading a Single Orchestrator Node from v. 2.3.5+ to Orchestrator 4.0" in the Admin Guide.
  • The feature that lets you replace the default logo for IBM Aspera Orchestrator with a custom logo is now implemented differently. Before you use the feature, see the updated procedure "Customizing the Orchestrator Logo" in the Admin Guide.

Documentation Updates

The upgrade procedure has important updates related to the upgrade of the MySQL version used by Orchestrator.

There is a new "Troubleshooting" section, which includes the articles, "Troubleshooting Orchestrator Startup Operations", "Managing Your Orchestrator Logs", and "Troubleshooting for High Availability Setups".

There is a new "Database Configuration and Maintenance" section, which includes the articles, "Reclaiming Disk Space Used by Orchestrator" and "Migrating Data from a Current Orchestrator Instance to a New One".

There are two updated procedures for single-node installation, "Installing Orchestrator with MySQL on the Same Server" and "Installing Orchestrator with MySQL on a Remote Server".

There is a new article to help users navigate through the API, "Available Endpoints in Orchestrator".

The procedure "Customizing the Orchestrator Logo" is updated to reflect the new implementation of the feature.

ISSUES FIXED IN THIS RELEASE

ORC-2781 - The command asctl mysql:set_root_password fails with the message, database.yml.sample not found.

ORC-2718 - The admin user can't delete a user that is a member of the Administrator group.

ORC-2677 - Restarting a fan-out step that uses the Vantage plugin (in a work order) now starts new Vantage jobs. The expected behavior is that the existing Vantage jobs from the initial run of the Vantage steps should be restarted.

ORC-2666 - For the Diva Plugin, after assigning the Object Category parameter a value in ERB (embedded Ruby) format, the value does not appear on the Inputs tab.

ORC-2665 - The Work Orders page is slow to load.

ORC-2656 - In the UI, the user changes a resource from Online to Offline, or vice versa, but the /resource_online or /resource_offline endpoint returns an error.

ORC-2644 - Trying to import a workflow on a high availability system in the cloud-with no shared storage-can result in an error.

ORC-2611 - A user can't publish a workflow with a runtime parameter of type other than String or Number.

ORC-2575 - When a workflow is saved, the workflow designer does not confirm whether or not the user has made changes to the workflow.

ORC-2570 - The Spreadsheet Parser plugin fails with some Excel spreadsheets.

ORC-2563 - On Queues > Manage Queues, after editing an existing queue, the queue ID changes. For example, the queue ID is Status for queue 'Transcode' before editing, and Status for queue 'Transcode_9' after editing.

ORC-2558 - Operations performed with the Media Mate Plugin return the error, Execute completed without returning a valid status (In Progress).

ORC-2552 - On the Work Order page, when the user selects the Bookmarked filter and no workflows are bookmarked, the UI may display an error.

ORC-2533 - Installation of Orchestrator fails on Red Hat when the /usr/lib/systemd/system/ directory does not exist on the system.

ORC-2528 - An attempt to query work orders fails with an error.

ORC-2491 - Parameter (string) values for steps can be corrupted during work order execution and when the user reopens a workflow step in the workflow designer.

ORC-2467 - When a user renames a workflow before it is imported to another Orchestrator instance, the import fails with an error.

ORC-2434 - Every time a user edits the Amazon S3 plugin configuration, the key and secret for AWS are automatically refilled by the browser (CIM-2397).

ORC-2410 - After a new user is created, when that user enters two incorrect passwords in a row, they get the error, You are not authorized to access that page. Access Unauthorized.

ORC-2366 - When the user clicks View by Work Order, the Bulk Restart button is not available.

ORC-2273 - When the Console Smart Transfer plugin fails with an SSL error, the transfer still completes. This issue only occurs when the Monitor Status option is enabled in the plugin and the certificate is missing.

ORC-2088 - There are multiple issues in the UI for journals.

ORC-1593 - The Soap Request Listener plugin doesn't work with Ruby 2.3.0.

ORC-1458 - In the Dalet Media plugin, when the user opens the Saved inputs section and enters text into the Password field, the Password field can't be cleared.

ORC-1177 - With the scripts/generate plugin execute method, outputs such as @outputs[OUT_XML_RESULT] = data are not working as expected.

ORC-1051 - After an upgrade, the AsperaOrchestrator Linux service is turned on. This presents an issue in a high availability setup with ACM for which this service (plus Apache and MySQL) needs to be turned off. Otherwise ACM will not be able to run properly.

ORC-721 - The Vantage plugin overrides parameters set in the workflow when the workflow name from a runtime input is used.

ORC-682 - After the user creates a file on an FTP server, a step using the FTP Trigger plugin triggers on the file. If the file is deleted and then recreated, the FTP Trigger step does not trigger on the recreated file.

SYSTEM REQUIREMENTS

Aspera Common Components 1.2.29

Linux: Linux 64-bit: RHEL 7-8; CentOS 7-8, SLES 12, SUSE 12

Browsers: Firefox 57-77, Google Chrome 64-85

KNOWN ISSUES

General

  • ORC-2876 - If a purging activity starts, and Orchestrator is restarted before the purge completes, the purge does not continue or complete.
  • ORC-2288 - It is possible to have "orphaned" work orders -- work orders linked to workflows that no longer exist in the system--because when you revert the Orchestrator server to a snapshot, Orchestrator intentionally keeps all existing work orders.
  • ORC-389 - Folder monitor does not always report an error when a directory is not found.

Work Orders

  • ORC-2492 - When the outputs of a step are very large, the user gets an error when the outputs are saved to the database: 'MySQL client is not connected'.
  • ORC-2081 - During the relaunch of work order, a runtime parameter isn't preserved if contains JSON format.

Workflows

  • ORC-2518 - In a workflow with two parameters that both have the same name, after the user saves or publishes the workflow, the parameters overlap so that only one parameter is visible.

PRODUCT SUPPORT

For online support, go to the IBM Aspera Support site at https://www.ibm.com/mysupport/. To open a support case, log in with your IBMid or set up a new IBMid account.