Dalet Flex 2021.5.0 (LTS) Release Notes

Dalet is excited to present the May 2021 release of Dalet Flex 2021.5.0.

This release of Dalet Flex provides the new features, quality and security enhancements, bug fixes, and other changes listed below.

Flex 2021.5 is an LTS (Long Term Supported) release track and will receive both critical bug fix updates and security updates until it reaches end-of-life.

Dalet Flex Platform

New Features

BlackPearl integration

Dalet Flex is now a certified Rio Broker client. Dalet will continue to support and make improvements to the plugin in future releases.

Save draft workflow definitions

Flex now provides the ability to save a workflow definition as a draft. Draft workflow definitions will not be used when launching new workflows, rather Flex will refer to the latest deployed version of the workflow definition.

When editing a workflow definition, users can choose between “Save” and “Save and Deploy” to either save a draft or save a new version to be used in subsequent workflow executions.

User and group role inheritance

Flex now supports inheriting permissions assigned at the user group level. If a user is a member of multiple user groups, they will inherit the aggregated list of permissions from all their groups plus the permissions directly assigned to their user.

Please note that the “Upload” permission assigned at the user group level is not currently respected in the FlexMAM. For now, please assign this permission directly to the user’s role rather than their group. This is a known issue which will be addressed in a future release.

Resource tagging

We’ve added the ability to add tags to resources to improve ease of maintenance and readability of configuration within Flex. This is especially useful when defining workflow logic to be reused across accounts. For example, an import workflow can be defined in the root account, but the actual storage resource that gets used can be different for each account that launches the workflow. Resources can have one or more tags and support the use of spring expressions for the tags to be evaluated at runtime.

Flex Satellite Nodes for distributed deployments

Flex services can now be deployed across multiple datacentres or cloud regions. A central Flex environment can send instructions to distributed Flex services to run remotely, starting with the File Processor (FFP) and Stream Processor (FSP) Services. Enabling these services for distributed deployment allows for file operations (e.g. hashing, tar-ing) and media processing (e.g. proxy creation, technical metadata extraction) without first transferring large assets back to the central Flex environment.

New Move and Copy workflow actions

New move and copy actions have been created for this release. The new actions are now based on the Flex Job Execution Framework (JEF) allowing for improved performance and compatibility with Flex Satellite Nodes. These should be used in place of the existing, legacy, move and copy actions.

Flex Premiere Panel deprecation

The Flex Premiere Panel has been deprecated in favour of the new FlexXTEND Adobe Panel. This will be the last release of Flex Premiere Panel; it will be removed in the next release.

Limits on Quota of Concurrent Workflows (new for 2021.5.1)

To provide better control over resource usage, Flex now allows you to limit the number of concurrently running workflows. You can define these quotas for the number of concurrent workflows per account type:

Support for STL Subtitles (new for 2021.5.1)

Support has been added in FMP to import STL subtitles and convert to them SRT, WebVTT, and TTML formats while dropping the styling info. Subtitle files can be imported, associated to a parent asset, converted to a compatible format, and then played back in FMP. Future releases will contain more functionality.

Improvements in Advanced Search in Flex Core UI and APIs (new for 2021.5.1)

Advanced searches performed in Flex Core UI has been improved. The Enterprise search is now powered by Flex Query Language (FQL) as used in Flex MAM.

Due to the evolution of our architecture (particularly, moving metadata from MariaDB to ArangoDB, for scalability), the traditional searching APIs on Flex Core (as used in both the Flex Core UI and, for example, the Groovy script SDK), had to evolve away from just searching MariaDB, if metadata criteria and/or free text searching was requested by the query.

• Our initial implementation of this was relatively simple, generating and performing an FQL search just for the metadata or free text components of the search, and then postprocessing the result set with any other filter criteria in a MariaDB query. As a result, certain combinations of search criteria could erroneously return zero results (e.g. if the first page of results from the FQL search all end up being post-processed away).
• The new implementation translates the majority of criteria - not just the metadata and free text components - into an FQL query. As a result minimal postprocessing is performed and the result sets are more reliable.

Mandatory Account Storage (new for 2021.5.1)

From Release 2021.5.1, Account Storage is mandatory for account files/images (User/UDO/Report template/Theme images) for all accounts including master account. The Consul KV accountFileVfsEnabled has also been removed.

For an existing installation, if the migration script was not run before, it must be run now as part of this release.

Vantage plugin upgrade (new for 2021.5.2)

The Flex Vantage plugin has been upgraded to support the latest version of Vantage and make use of the Flex Job Execution Framework (JEF) including a new Vantage Workflow Resource, and new Launch Vantage Workflow action plugin. The Flex integration supports intelligent retry capability, ability to cancel, restart, and monitor progress of Vantage workflows, workflow variables, multiple input assets, and import of media produced by Vantage.

Note: the integration currently supports Flex FILE assets, with additional VFS protocols coming in a future release.

Drop Frame Support for Fractional Frame Rates 29.72|59.94 fps (new for 2021.5.5)

Dalet Flex now supports fractional frame rates (29.97, 59.94f) with drop-frame (DF) support. Before this change all 29.97|59.94 were assumed to be non-drop-frame (NDF). This has been updated across the whole platform: from Extract Technical Metadata mapping to API and FMP UI. VideoStream dropFrame boolean property is used to differentiate DF/NDF options.

Storage DR: Proxy (New for 2021.5.5)

Hot swapping of proxy source Storage Resource (Cloud Storage or File) can now be configured using Folder Resource and a new Selector Policy: Custom Resource Selector Policy. It is possible to define multiple Storage Resources to host proxy file/segmented assets. Managed asset proxies will resolve to the Storage Resource defined in the Folder Resource Selector Policy. Proxy Managed Asset now also supports subpath. Cloning Proxy assets across Storage Resources is not yet performed automatically. This should be achieved using S3 replication or configuring copy jobs at workflow level.

Search by Name in Flex Core (New for 2021.5.5)

To improve search performance, from this release, Search by Name in Flex Core retrieves results using “starts with” instead of “contains”. This means that for any object with multiple values in the Name field, you should only use the first value in your search query. For example, for an Asset called “Dalet Flex”, search for Dalet and for an Action called ”Add Asset to UDO”, search for “Add”.
NOTE: Search using the global search text box is unchanged. Type any text in the Search for text box and global search will list all objects containing that text.

FlexMAM

New Features

Search Experience

The ability to set operators and conditions in search within FlexMAM has been improved.

Search chips can now be refined with the following conditions (where appropriate): contains, is, is not, starts with, ends with, is empty, is not empty.

Groups of search chips are now colour coded to be more explicit. Users can also update operators between search filter chips that are already on the search bar.

Chips belonging to the same key (for example “Type is Video” and “Type is Image”), and taxons from the same structure will be grouped together automatically. Operators can then be changed between each group to further configure search.

Saved searches can now be shared using ACL permissions. Shared saved searches can now be found under the “Shared Searches” menu item allowing users a clearer view of their saved searches.

Thesaurus (new for 2021.5.1)

Thesaurus support enables users to map related terms within Metadata Designer. Within FlexMAM, the user can then use the Thesaurus when searching assets and when filling in metadata forms.

For search, when the user starts typing in the typeahead, they are presented with a list of matching terms that they can then choose from. This works for both searching with filter chips in advanced search, as well as table column headers. Use the Thesaurus icon to open the Thesaurus suggestion panel to find alternative related terms to use.

For metadata forms, when the user makes a selection for a field, a suggestion panel of related terms will appear, allowing them to select from the list.

Caps, spaces, and special characters can all be used within thesaurus terms.

Multiple Audio Tracks Selection and UX Improvements (new for 2021.5.1)

From Flex release 2021.5.1, you can individually select which track is enabled and previewed when you open an asset, (with HLS format), in the Manage Asset page. You can select single or multiple tracks to preview to create a mix of audio tracks directly in FlexMAM. For backward compatibility, you can disable multiple track selection whenever required. This functionality is supported for up to 16 audio tracks.

To simplify managing audio tracks, you can rename audio track names easily and you can expand/reduce the width of the audio tracks label area by simply dragging the separator between the label area and the track. Drag the playhead through an asset to change its position and the progress bar on the player jumps to the same position.

Navigate easily between waveform and clips view using the Timeline and ClipBin menu tabs above the audio tracks.

Bulk Update of Markers (new for 2021.5.1)

Improvements have been made to the way you bulk update metadata in FlexMAM. FlexMAM now supports bulk update of metadata on markers. You can use Ctrl + click to select multiple markers and then use the Edit Metadata Quick Action to perform a bulk update of marker metadata.

For all other asset types, the Quick Action menu is available whenever you select multiple assets in any listing, not just in the Bulk Assets screen. The Edit Metadata quick action can be used to bulk update metadata for all asset types.

Duration in FlexMAM is displayed as HH:mm:ss.SSS (new for 2021.5.8)

Duration in FlexMAM is now represented as Time instead of as TIMECODE, so it is represented in this format HH:mm:ss.SSS (hours:minutes:seconds.milliseconds). The Duration field is displayed in this way in all relevant locations in FlexMAM.

NOTE: The change in the Duration format was also applied to markers. Markers should be displayed as Timecode and this issue will be addressed in a future release.

Changelog

2021.5.10

Enhancements in Dalet Flex

• JEF: Local filtering of message with no progress (don't backport nextStepDelay req for archive/restore plugins, part of 62399) for 2021.5.9 version.
• Testing task for satellite deployment testing for reviewer.

Fixed Issues in Dalet Flex

• Fixed an issue where the Sort by function in FlexREVIEW does not work for any of the selected options.
• Fixed an issue where asset visibility via ACLs was ignored when navigating the UDO structure.
• Fixed an issue where job priority is not respected if a quota is applied to a resource and a job is set to pending because the quota is full.
• Fixed an issue where this error was thrown by DB pruner when pruning a batch size of 1000 Error 1406: Data too long for column 'jobsDeleted' at row XXXXX. The error occurs when the total length of the stored procedure output, enumerating all the jobs that have just been pruned, exceeds 65535 characters (the length of a TEXT value). To overcome this limitation the column type for DB pruning temporary column was updated to MEDIUMTEXT. 
• Fixed an issue in the legacy SDK where it was not possible to set a drill down metadata taxon field to a taxon when it contains an ampersand.
• Fixed an issue FSP 9.7.6 to support backwards compatible naming for single video stream HLS output.
• Unable to set a taxon value containing '&' character.

Fixed Issues in FlexMAM

• Fixed an issue where the reset password button was visible for external SAML users. Issue fixed by adding a condition to only display the reset button for Flex users that do not have external authorization.

List of Services upgraded between 2021.5.9 and 2021.5.10

2021.5.9

Enhancements in Dalet Flex

• Allow stream based chunked transcode of package based proxies. This improves efficiency of transcode of sources with many audio streams by outputting separate full streams.
• Support for channelSplit on FSP output audio profile for stitch, sequence and transcode jobs added. Improved support for handling unknown audio channel layouts across stitch, sequence and transcode jobs added.
• Image Sequence: support start-time in image_sequence Plugin Schema (Metadata Definition)
• Jobs in workflows were completing but were not triggering the next job. Workflows now always trigger the next job after a job is completed, if configured, solving any deadlock issues.
• Whilst parsing technical metadata results for MXF files with the Extract Tech Metadata action this exception was returned:Exception while parsing technical metadata results. The source of the exception was the dropFrame property. To fix the problem, the dropFrame property is derived from FSP.FormatContext.timecode if it exists. If it doesn’t exist, it is derived from videoStreamContext in JobExecution library.
• JAE to filter RUNNING messages so that if there is no progress in a job then no job status message is sent.
• As part of the improvement to workflow scheduling, added a new index, MIO_WORKFLOW.
• To prevent Keyframedownloadcontroller accessing anonymously, keyframedownload controller removed from exclude patterns of auth filters.
• Asset and UDO metadata instance conflict prevention added to SDK metadata manipulation performed in FlexCORE or using scripts.
• To prevent unreliable metadata updates where the job and asset history show that metadata update had been successful when in fact changes were not saved, metadata revision support has been added to asset and UDO metadata REST API. The metadata instance revision is returned as an HTTP header when running a GET command and is passed back using a PUT enabling Flex to check for version conflicts. If the version is not what Flex expects, an error is returned and the update is not performed.
• Flex Core application now detects and notifies the users if an asset or UDO metadata is being updated by two different users at the same time. The later attempt fails, and an error message is displayed.
• Revised the way VFSLocation paths are fetched from the FlexResourceApiFacade cache to ensure that the correct path is received.

Fixed Issues in Dalet Flex

• Fixed an issue where the default chunk processing was not applied due to “image sequence duration cannot be calculated” when the framerate input is forced.
• Fixed an issue where a video output stream sequence job was failing due to an ordering issue when stitching multiple package format source streams so that a transcode and extract operation performs correctly.
• Default value of Consul KVflex/flex-streamprocessor-service/disableDefaultShardingOnFrameExtractOutputchanged fromfalsetotrueso by default, when sharding is enabled, FSP extracts keyframes to sharded locations based on file name.
• New external user creation retrieves all existing workspace members and then adds the new user to the workspace by sending it together with the list of existing members. This prevents existing members being removed whilst the new member is added to the workspace.
• Fixed an issue where although GXF is listed as a supported format, clip jobs using any profile/codec other than COPY codec were failing and all stitch actions on GXF assets were failing.

Fixed Issues in FlexMAM

• Review timeline comments are not displayed correctly in FlexMAM when the Start of Material is non-zero.
• Fixed an issue where a radio button in a metadata form was clickable even if the cursor was not on it.

2021.5.8

Enhancements in Dalet

• Improve code which checks for failed workflow for any Asset
• Fixed an issue where the concurrent limit for workflows and actions was not observed when retrying a failed job
• Fixed endless loop issue when requesting asset information for images
• Improved how timecode information is captured, stored and delivered.
• Output profiles updated to enable specific configuration for explicit or template modes.
• Improvements made to job progress reporting: Flex Media Processing Engine (FMPE) reports realtime progress for all jobs including stages and the concept of job history
• To improve the mechanism for watching jobs in case of a restart, different KVs implemented for running/queued/timed out job thresholds. In addition, a configurable initial delay KV for starting the watcher has been added together with a configurable KV for the periodic interval the watcher runs at.

Fixed Issues in Dalet Flex

• Event accountId is used for filtering visible Event Handlers instead of ownerId. This prevents errors if a particular user is deleted from the system.
• A bug in Core REST API did not return the correct values for fileSize and compressionLevel. To solve this, ImageContext fileSize and compressionLevel changes have been backported to 2021.12.5
• Fixed an issue where the Search API returned the wrong results when the offset is used and greater than the totalCount
• Fixed an issue where there was an error on move action from S3 to S3
• Fixed an issue where FSP jobs as CreateProxy, Composition or Transcode got stuck in an indefinite loop after failing to execute the Extract Technical Metadata operation. The handler enableExtractTechMetadataAfterTranscode (default = true) was added to disable this behavior if needed.
• Fixed issue where DPX image sequences converted from HDR to SDR result in black frames.
• Resolved a bug in ffmpeg that was causing frames proxy creation jobs to fail. FSP updated.
• Fixed an issue where a user that does not have permissions for a workspace but does have ACL access to assets that are children in a UDO could not see the assets. The issue was fixed by the system checking ACL permissions when a user does not have workspace permissions.
• Improve code which checks for failed workflow for any Asset. This prevents a general overload in Flex.
• Fixed an issue where retried workflows did not respect concurrency limits and cause other workflows to get stuck in a pending state.
• Fixed an issue where FSP was creating an incorrect signed URL when running an extract keyframe job with a specific job template using a VFS file type. FSP updated.
• Fixed an issue where jobs are running without failing or doing anything for a long time
• Fixed issue where proxies creation fails when during times of high load
• Fixed issue where proxies for QuickTime AVC files failed to create
• Added asynchronous logic to workflow state changes to reduce the opportunity for deadlocks to happen when transitioning from one job to the next.
• Fixed an issue where DPX images were wrongly identified as a YUV format
• Fixed issue where video fails to ingest where timecode does not start at zero.

Enhancements in FlexMAM

• Increase maximum file upload sizes: 2MB for user profile image, app logo, favicon, keyframe. 10 MB for login background. Unlimited for metadata attachments.
• Media duration now displayed as ‘time’ instead of ‘timecode’ in Technical Metadata panel, i.e. HH:MM:SS.MILLISEC

Fixed Issues in FlexMAM

• Fixed an issue where the timestamp was displayed incorrectly on comments created in REVIEW app
• Fixed an issue where the My Task section was impacted by actions by other users. The fix made sure that updates made by users other than the current user are isolated from the current users My Task section.

List of Services upgraded between 2021.5.7 and 2021.5.8

2021.5.7

Enhancements in Dalet

• Use rational number based fractional frame rates in Java Timecode library
• Add new "version" property to AssetContext.version. Value = 1 for new assets Value = 0 for existing assets

Fixed Issues in Dalet Flex

• Fixed a query building error that prevented search asset by asset type in CORE
• Fixed an issue where API in Staging was not returning the complete list of items
• Fixed an issue where "codec" parameter appearing as "Unknown" under videoStreamContexts in Publish V2 (JEF) API call for asset
• Fixed an issue where keyframe extract was taking a long time

Fixed Issues in FlexMAM

• Fixed an issue where selecting a collection task redirects to the collection metadata but metadata on summary panel does not open up

2021.5.6

Enhancements in Dalet Flex

• Improvement correctness of summary counts in Data Aggregation
• Fixed an issue with incorrect mapping of enum fields in FastObject Asset API
• Added support for using the formatcontext timecode if the the videostream timecode is not defined

Fixed Issues in Dalet Flex

• Fixed an issue related to parsing the updated date on Vantage jobs that was causing jobs to fail
• Fixed an issue where If an Enterprise API query is delegated to SearchElastic, and sets the parameter Deleted to “true” or “null”, the SearchElastic would fail to return deleted records
• Fixed a job where FSP jobs are failing if interlace format in associated profile is anything other than "Progressive"
• Fixed a job execution issue in the new eventHandler plugin

Fixed Issues in FlexMAM

• Fixed an issue where the download icon is not clickable on the Manage page

2021.5.5

 Enhancements in Dalet Flex

• Search Issue - Taking longer time to have results than usual
• FastObject service does not return parent information for assets with a UDO / generic object parent
• Align chunk times on video frame boundaries to prevent discrepancies between the duration of an image sequence and the generated HLS proxy
• Improve locking while getting/setting workflow variables and executing job by introducing new KV and using feature toggle to keep existing behavior
• Add consul KV flex/flex-streamprocessor-service/forceProgressiveOutput support for FSP. Use to solve an issue where proxy files had green frames.
• Fixed and issue where image overlay for stitch or sequence job fails if target is local filesystem
• Use the external authentication account metadata to set default owner, timezone and country code for the external users to be created
• CreateUser API has been extended to allow you to set the default owner of the user being created
• Improvements to Job Scheduler to process pending jobs in batches where each batch is processed in separate transactions.
• Update to latest stable ffmpeg to prevent errors when extracting keyframes from HLS proxy versions
• Added support for drop frame support for fractional framerate (29.97f/59.94f) NDF to support timecode + capturing richer info
• Fixed a number of inconsistencies in API Asset Technical Metadata fields which were throwing exception when null values were used/not used.
• Storage DR: Proxy

Fixed Issues in Dalet Flex

• Fixed an issue where it was not possible to delete custom markers (from the UI) when they are being deleted too quickly
• Fixed an issue where image sequence import was failing because the format was being flagged by BDAV anti virus software
• Resolved an issue where there was the wrong file duration and end timecode in technical metadata
• New presets added to FSP
• Fixed an issue where previously ingested image sequence assets are not reachable anymore in API
• Fixed an issue where AS02 proxies appear green in FMP
• SAML SSO Created By field doesn't follow the user that was set as Default Owner
• Fixed an issue where frames proxy creation was failing with error code 255
• Fixed an issue where MAM UI displays purged annotations
• Fixed an issue with frame accuracy problem in player for 29.97 and 59.94 FPS files having no start time code
• Fixed an issue where using “Fixed interval” calculation for frame extract can cause repeated frame extracts in a loop
• Fixed an issue where concurrent task execution is not constrained in FSP
• Fixed an issue where FSP was not recognising timecode stream
• Fixed an issue where a new keyframe does not override the previous master keyframe
• Fixed an issue where report templates and documentation was missing
• Fixed an issue where Enterprise API did not map AssetAPIQuery.frameRate field to FQL so that the filter for asset API queries was not applied
• Fixed an issue where it was possible to set several master keyframes.
• Fixed an issue where Enterprise API does not map AssetAPIQuery.parentAssetId field to FQL
• Fixed an indexing issue for image sequence assets
• Fixed an issue where Fast Object service is generating errors when try to index file or add in collection
• Update flex-elasticsearch-library to check version conflicts when using deleteByQuery function. A retryable annotation has been added to the deleteByQuery logic so that the function is rerun if a version conflict is discovered.
• Fixed an issue where retrying jobs that were stuck before upgrade were failing with InternalServerError: status 500 reading AssetApiFeign#getAsset(long)
• Fixed an issue where demux action is not working

Enhancements in FlexMAM

• Implemented support in Flex player for correct NDF clock display of 29.97 and 59.94 frame

Fixed Issues in FlexMAM

• Fixed an issue where the Download icon is not available on the summary panel and manage page even though they there are download permissions
• Marker edit icon from manage page not clickable
• Next frame icon does not work
• Start time code is not consistent between asset TCR, player and technical metadata
• Fixed an issue where it was not possible to access Admin app and upload after assigning permission via group role
• Fixed an issue where start time code is not consistence between asset TCR, player and technical metadata
• Fixed an issue where account data was not being saved
• Fixed an issues where the reset password email was not sent so the password could not be reset
• Fixed an issue where does not have access to Admin app and upload after assigning permissions via group role
• Support special characters asset name and special characters and spaces in workspace name

2021.5.4

Enhancements in Dalet Flex

• Add further retry logic and improve error detail logging
• JEF Hotfolder Plugin should not scan sub-directory files when Max Depth is 0. It only scans the current folder.
• Fixed an issue where assets in a different workspace that were found in Asset search were not listed in the asset table when performing a purge action
• Allow /mnt/operation in legacy script operations and support Consul overrides
• When parsing extracted timecode string, extracted timecode hours are wrapped to 24 hour period if hours in hh:mm:ss:ff is greater than 99
• When performing a Bulk Action in Enterprise, the workspace check is skipped so that assets are loaded without a permission check
• Support for pass-through source DAR output profile
• Force BT709 colour profile when transcoding default presets
  - New presets created to perform audio channel 5.1 or 7.1 mix-down to stereo to support atmos audio
• DB Pruner configuration guards
• Enterprise UI improved to extend panel window size when assigning groups/visibility
• Changes to Flex Authorisation Service API - user permission (user+ groups)
• Added an option to synch IDP groups with Flex using groups external ID rather than the Flexuser group name.
• Implement a watchdog timer to discover stuck active jobs caused by FSP instance termination and to re-queue them
• Support image overlay in output profile for stitch and sequence jobs
• Added asterisk to GUI to indicate that if a variant of type group exists, variant is a mandatory field when creating a new group
• Modified code to update all (asset/udo) having older definition with latest definition after completing merge job. This will just update Asset Metadata Def id to refer latest version, it will not add document in ArangoDB.
• Add width/height to Image Sequence Technical Metadata
• Vantage Resource to support Folder Mapping for unmanaged assets
• Support for thesaurus manipulation via the JEF SDK added

Fixed Issues in Dalet Flex

• Fixed an issue where AS02 proxies appear green in FMP
• Fixed an issue where audio files are imported as miscellaneous files when using Adobe Flex Panel
• Fixed an issue where metadata merge was failing
• Copy file from AWS bucket to Alicloud bucket works but job fails because it looks at the wrong path
• Fixed an issue where create proxy job is failing
• Fixed an issue where Power Bar buttons are not functioning.
• GET calls in REST API documentation have matrixParam marked as Required even though in practice they are not mandatory
• Fixed an issue where Publish to Facebook with a Draft setting was always published as "Published"
• Enhance script security around filesystem read/write
• FMP/Enterprise Timecode issues. Wrong Timecode calculation: microseconds to frameCount (hence timecode) issue
• FMP/Enterprise-UI drop-frame inconsistency with Enterprise Timecode class used in scripting (and REST API)
• Fixed an issue where user permissions are not applied and data is saved even though the user does not have edit permissions
• Fixed an issue when a tag/taxon is renamed or enabled/disabled, the tag/taxon is not updated for annotations which are indexed as part of Assets.
• Fixed an issue where the Finish option of Reviewer app was not available when selecting a custom status that does not have a metadata definition associated to it
• Wrong Timecode calculation: microseconds to frameCount (hence timecode) issue
• Fixed issue where create composition proxy is not working on package asset
• Fixed an issue where assets in a different workspace that were found in Asset search were not listed in the asset table when performing a purge action
• Fixed an issue when a tag/taxon is renamed or enabled/disabled, tag/taxon is not updated for annotations.
• StartTime in enterprise persist via NDF calculation(in plugin side) though the frameRate (29.97|59.94) is of DF one
• Fixed an issue where there were memory leaks when transcoding/stitching files

Enhancements in FlexMAM

• Time display uses the user timezone set in user profile. If there is no timezone value, UTC timezone is used.
• Date / time primitive is displayed in the Search Results table with both the date and time and is not truncated
• Update search request with the timezone defined in the user profile

Fixed issues in FlexMAM

• Fixed an issue where modifications made to user details in FlexMAM are not saved and not reflected in Enterprise
• MAM/Reviewer : startTimecode does not have to convert from NDF to DF frames in case of dropFrame timecode
• Fixed an issue where Keyframe uploads fail in MAM for files close to 1MB in size

2021.5.3

Enhancements in Dalet Flex

• To remove unrelated search results, a free text search makes non-wildcard free text queries always use the AND operator and not the OR operator
• Support configuring S3 resource with a direct AWS Cloud Connect URL
• Get metadata response call returns the enabled/disabled tag for tag and taxon
• Flex Web Uploader supports uploading XML files (brought back lost functionality)
• "Keep technical metadata" option in "Purge file only action" updated to Keep Technical Detail and Markers
• Skip workspace check set to true when performing a Bulk Action to cancel multiple workflows
• Searchelastic: All date and date/time values retrieved in a search are in ISO8601 format
• Support (optionally) specifying user timezone in search requests. All results are returned in UTC.
• Events generated by Taxonomy Service for taxon rename/enable/ disable have been fixed to correctly indicate the change. When a taxon is enabled/disabled, all of the taxons are enabled/disabled underneath it (at the lower levels) and an event is generated for each taxon.
• When a taxon is renamed or enabled/disabled, all documents in ES which reference that taxon are found and updated accordingly
• Add expected vs actual output format and stream duration checks to sequence and stitch jobs
• Taxonomy pagination implementation reverted

Fixed Issues in Dalet Flex

• Fixed issue of JEF Copy/Move failing on media asset (2021.5.3) by setting Consul KV - flex/flex-fileproccessor-service/enableUsageMeteringStats = false
• Fixed issue where wizard trigger object was not supplied as a workflow variable when asset has ACL set
• Fixed an issue that it was impossible to delete a Metadata Timeline that was associated to an asset that had been deleted
• Fixed an issue where Publish to Facebook failed with Java out of memory error
• Fixed issue where Start time code is not available for AS02 package on player by adding startTime/startTimecode in format, video and audio stream context for get asset API call on AS02 assets
• Fixed an issue where import jobs were failing because of mis-identification of DPX files by Mediainfo
• Fixed an issue where getMioObjectListVariable throws an NPE if variable does not exist
• Fixed issue where the Timezone set in Enterprise for a user was not returned in an API call to get user details
• Fixed issue that Video Proxy Service was not able to access proxy files in 2021.5.2 because of an incorrect parameter name in the service-support initialisation object
• Fixed the NPE thrown when (re)indexing variants without metadata definitions (dummy metadata)
• Backported fix that Republish Player icon does not appear on the asset powerbar after running Publish Player action
• Set the behavior for sorting search results that return missing values. When sorting in ascending order, missing values appear first and when sorting in descending order, missing values appear at the end.
• Fixed an issue where dummy metadata fields are not transformed with var-???- prefixes prior to indexing by applying the same transformations to dummy metadata that are applied to real metadata
• To search for exact matching of metadata field in a specific metadata definition you must use var-xxx- prefix in the FQL query
• Fixed issue that JEF Copy action is failing
• Fixed issue that searching with any value for ;originalFileName =, Flex Enterprise API returnsthis error: "SQLGrammarException: could not extract ResultSet"
• Fixed an issue that quotas cannot not be disabled
• Fixed an issue where sorting in FQL using DESC or ASC returns an error
• Fixed an issue where player in Reviewer is not playing media asset in a satellite deployment test environment
• Fixed an issue where subtitles are not available when playing assets in Reviewer player

Enhancements in FlexMAM

• Timeline comments sorting options updated to Newest/Oldest and Timestamp In in ascending/descending order. Sorting by Timestamp Out has been removed.
• Support of multiple sorting on tables in admin app and sorting on a Bulk Asset table column backported to 2021.5.3
• Tags and taxons are displayed with a strike-through on Search Listing and on Metadata Form

Fixed issues in FlexMAM

• Fixed an issue that Clipbin functions do not work - 2021.5.3 env
• Fixed a regression that the Download icon in the Manage page right panel could not be clicked
• Fixed a regression that taking and starting tasks in FMP was not working and gave an error
• Fixed an issue so that the Marker tab in summary panel is displayed for assets which had 'Purge File Only' action executed on them
• Fixed an issue that the Date picker remains blank when a date format is not defined for a user account profile in admin app
• Fixed an issue where selecting Search with typeahead variable in the Metadata Summary tab does not add a search chip
• Fixed a regression issue where the Save button in Edit Template is not enabled if Share this template checkbox value is changed

2021.5.2

Enhancements in Dalet Flex

• Add support for tif extension on TIFF files
• Support NBSP UTF-8 character in VFS
• Support FFV1 codec and fix de-interlacing and interlace passthrough 
• Support mix-down of unknown source channel formats and provide new channel split profile feature
• Add reorder function to Collections in the SDK
• Image sequence TAR file verification
• Add workflow definition’s concurrent workflow limit in exported xml
• When not directly searching annotations, return up to a maximum of "n" annotations, instead of all of them
• Update events generated by Tag Service for tag rename / enabled / disabled.
• Support for STL Subtitles
• Ability to Update Start Time Code using the JEF SDK

Fixed Issues in Dalet Flex

• TTF and .OFT files should be categorised as Text Asset insted of Media Asset
• Collections type is not available in Create variant
• Large, gzipped file based result was not being created before source was deleted
• Searching for a tag is not working when backed by a thesaurus
• Flex Workflows API: Error while fetching workflow variables when an object variable is present in workflow
• workflowsInProgress count at account level is incorrect in quotas/workflow api
• Job gets failed when try to update image seq data using groovy for updateAsset method
• Incorrect records returned for =NULL condition on nested field
• Disabling a Tag removes it from the asset's metadata form in FMP
• SAML Authentication: Default workspace and Member Workspace are not assigned to external user
• Fix asset context format, width, and height on published image assets
• Add missing AnnotationAPIQuery parameters
• Tech metadata extraction failure - Error with job: [500 Internal Server Error] during [PUT]
• flex-jobasyncexecutor-service throwing up error and jobs timing out
• Object variables are set to be String variables when using JEF script actions
• NullPointerException when setting tags in metadata
• ttf & .oft file-type support in Web Transfer
• Error on Import action with HLS asset
• assetContext is not populated in API
• Import ISO Broadcast] Failing on job node, activated auto retry not happening
• Sort function on the ASSET column on My Tasks not properly sorting the tasks
• SQLGrammarException: could not extract ResultSet

Enhancements in FlexMAM

• Change bulk marker metadata edit columns
• Display disabled tag and taxon with a strike-through on Metadata Form
• Add a default sort column to the custom table templates
• Sync filter chips with new search criteria

Fixed issues in FlexMAM

• Markers are not getting created in FMP
• Keyframe extraction, pooling inefficient
• Asset selection slow after 2021.5.1 upgrade
• Audio Meters stop working after upgrade
• UI issue on manage page right panel
• Year can't be selected for date filter
• Name display discrepancy between app switcher icon and browser tab for Operational Dashboard

 2021.5.1

Enhancements Dalet Flex

• Improved handling of search queries in Flex Core GUI
• Increase taxon maximum length to 100 characters
• Add support for auto-scaling use case for Stream Processor Service
• A call to retrieve children under a UDO sorts the children when returning the results
• Sub fields of complex objects (tag/taxonomy/thesaurus) are filtered out when creating a template or selecting a field from a chip filter in a search.
• Enabling/Disabling a tag generates an Enable/Disable event
• Extract aspect ratio and scan information from frame decoding if it is not available in the MP4 wrapper information. If no information is extracted use SAR 1:1 and Progressive by default.
• Fixed an issue where the incorrect objectUUID for collections was passed in the Event Data
• FFP controller nodes in Kubernetes can execute subsets of these job types: settags, archive, restore, restore-poll (internal job type)
• Ability to set quotas to limit concurrency at the workflow level added
• Support for STL Subtitles
• Support searching for tag metadata fields that have backing store of type Thesaurus
• Method added to get/set the data of a resource item. Maximum file size to retrieve is 30MB
• Support for Partial File Retrieval with Spectra Logic Black Pearl added
• Support for job persistency if flex-BlackPearl-service restarts/ there is a network outage while performing archive/restore with Spectra Logic Black Pearl
• Alias Mapping Resource added to Import Action to resolve conflict if multiple alias paths match the Source VFS Location.
• Identical aliasName/aliasVFSLocation on the same account is not supported. To prevent clashes, configurations are validated on saving a Resource.
• Support of datacenter and tags for FSP action and resource plugins : Extract Technical Metadata, Extract Keyframe, Extract Tiled Keyframe, Audio Waveform, Image Transcode (FSP), Segmented Proxy and Create Composition Proxy, Demux, Transcode and File Processor Resource and FileProcessor plugins (tar, untar, copy, move, cloudarchive, cloudrestore)
• Comments on Assets and UDOs in MySQL database are migrated to ArangoDb and removed from the MIO_VARIABLE_COMMENT table in MySQL
• FPS indicates it is DOWN if Redis disconnects or dependent services (secrets-service) are unavailable
• Remove consul KV: accountFileVfsEnabled
• Compiler warnings removed from Hot Folder service logs
• Added the ability to specify job types to execute in job-to-completion mode
• Workflow graphs can be saved without making any changes in the workflow design.
• Improvements made to prevent polling failures in hot folders
• FastObject returns the format name when searching the assetContext.formatContext.format field
• Added Collection Deleted to event handler filters and removed legacy Collections reference
• Fixed error in Flex Stream Processor Error: Truncated input file detected as output duration.
• Improved Business Intelligence plugin. Optimized Elastic Search index rollover size to 1 GB (configurable via KV), added minimum retention period configurable via the plugin (default of 90 days), multiple URLs and ports supported for the Elastic Search cluster.

Fixed Issues Dalet Flex

• Fixed an issue where hotfolders on different storage resources needed to be regularly restarted otherwise they got stuck
• Inconsistent format and Video Stream Information between core and FlexMAM for AS02 Assets 
• Fixed an issue so that the duration of an asset in CoreUI is consistent with duration in FMP
• A user can only add assets visible in a workspace to a collection in that workspace
• Fixed an issue where Mpeg Dash proxy added to a sequence does not work.
• Fixed an issue using an extract action for Alibaba Cloud Storage
• Missing authorization logic, in particular regarding visibility, has been implemented in flex-authorisation-service
• Fixed an issue where Business Intelligence creates 3 primary and 3 replica shards per index
• Fixed an issue where Publish job for assets in GCP fails with google-api-client IllegalStateException
• Fixed an issue where Flex Core returned a bitrate for data streams
• Fixed issue where Metadata definitions and variant tab were not accessible after assigning permissions via Group Role. User permissions are checked with authorization api call.
• Improved FSP code handling of alpha channels in RGB formats to prevent issues with transcode
• Fixed an issue where the incorrect Timecode was taken from last decoded frame in asset technical metadata and used for the SOM
• Start timecode was showing N/A, now defaults to 00:00:00:00
• Mediainfo parsing was not handling stream seeking
• Improved support for expression language in JEF event handlers
• MOV file with EIA 708 captions were not handled properly by ffmpeg 
• Removed unsupported webvtt text stream templating in MP4 and HLS profiles 
• Upgrade migration script failing for collections with more than 1000 items
• Fixed issue that FFP did not pick the correct datacenter when a tag and a datacenter name were set for FFP resource
• Fixed the issue where items in a collection are not visible in Search results when a collection has been shared with “modify” permissions 
• Support for naming of HLS output streams has been reinstated in transcode profiles
• Fixed an issue where some image sequences wre not available in FMP after upgrade because of an error in indexing
• FSP in job mode can sometimes consume more than one job and not exit
• Fixed an issue where searching with Starts with and Ends with returned incorrect results
• Fixed an issue where wait jobs were not being completed even if all the signals have been received. Wait job is completed if signal is given via Groovy script.
• Transaction timedout error when job run for longer time - i,e, import from s3 to s3 also for 40 GB file. Fixed an issue where long running jobs were failing because of an acknowledgement timeout limit on RabbitMQ
• Proxies were not playing in FlexMAM when the filename contained whitespaces 
• Fixed an issue where Wizard metadata added to a UDO was duplicated in FlexMAM
• Fixed error while saving event handlers with expressions
• Fixed issue of .error spamming when open MioObject pages in UI
• Fixed issue where the Mime Type was not being displayed correctly when performing an import action on a pdf file

Enhancements FlexMAM

• When a template is updated from being shared or inactivated, the out of the box default template is reset as the default Template in FlexMAM and in the FMP Admin app.
• Upload icon in FMP is only available for users who have Upload permissions
• Is not empty search filter condition is display completely
• Fixed an issue so that the duration of an asset in FMP is consistent with duration in CoreUI
• Added the ability to sort the notifications task list by Oldest/Newest Created Date and Ascending/Descending Asset Name
• A tooltip is displayed in the notifications task list when an asset name is too long
• Type ahead table header filter to add additional search chips is displayed properly when the chips bar is multiline.
• Multiple column sorting added (maximum 2 columns)

Fixed Issues FlexMAM

• Clips created form a collection item are only added to the Bin/active Working Collection
• Fixed an issue where blank options were displayed in a drill down option field
• Fixed an issue where selecting the All drop down in the searchbar refreshed the search results
• Fixed an issue where asset selection was not updating properly when switching between assets after opening assets in the manage assets page
• Added space in the GUI between the All drop down menu and the chip bar in advanced search
• Fixed an issue where the Edit filter box for the Created in the Marker listing page at column was not working properly
• Fixed an issue where the wrong tooltip was displayed when hovering over workflow notifications
• Fixed an issue where the Mark with metadata icon was not displayed below the player
• Fixed an issue where scrubbing on the progress bar on the player was not working properly
• Fixed an issue where the search queries using the Is Not dateand duration filter were not working
• Fixed an issue where the Manage page was not loading well for placeholder media assets
• FlexMAM search performance for single-select fields improved.

2021.5.0

Enhancements Dalet Flex

• Account Storage: Metadata Storage is released and mandatory
• Account Storage: MasterAccount: available for theme images, UDO, user icons, report templates
• Account Storage: MasterAccount migration script available for theme images, UDO, User icons Reports
• Improved the stability of the Flex job queue.
• Improved Flex Service health checks.
• Added database constraints against null parent.
• Improved logging for extract Waveform actions.
• The Flex filestore migration scripts have been improved to better handle files with no extension at the end.
• Improved support for expression language in JEF event handlers.
• Cleaned up B2B API response for group assets.
• Added support for Groovy’s XML MarkupBuilder in the new Flex SDK.
• Added support for Groovy’s JsonSlurper in the new Flex SDK.
• JEF: added support for Spring Expressions on Date fields (dd-MMM-yyyy).
• JEF: Added support for Spring Expressions on transcode plugin clipping configuration (“00:00:10:00@23.976” or “00:00:10:00@29.97(DF)”).
• Improved processing speed of extracting audio waveforms for assets with many audio streams.
• Increased taxon maximum length to 100 characters.
• Improved stability during full reindexes.
• Added support for uploading PAC subtitle files in FlexMOVE.
• Legacy collections have now been removed (previously deprecated).
• Added JEF examples
• Added ability to extract Technical Metadata as a file (tgz).
• Extracting technical metadata on image sequences now follows standard workflow conventions.
• Improved support for clipping assets with b-frames
• Improved progress reporting in the Flex Stream Processor
• Index asset membership information fields "membershipName" and "index" in ElasticSearch
• Added option to Purge File Only action to allow preservation of technical metadata
• Support AS02, IMF, CPL, and Image Sequence assets in FlexREVIEW
• Updated default landing page after set password, reset password, etc. to be FlexMAM instead of Enterprise
• Added support for Publishing CPL Assets
• Added timeout for Redis locks during reindexing
• Improved support for audio channel mixing for assets with many channels and non-standard audio streams
• Improved health checks for Java based microservices
• Added metadata designer to support thesaurus-backed tag fields
• Improved support for storage mappings in asset API call /api/assets
• Resource Storage Mapping: reconciliation using alias (Import, AssetUpdate...)
• Improved support for importing IMF Packages from Hot Folders
• Improved stability of Tar Import jobs while retrying
• Added Flex ProjectAsset as a new asset type (more support coming soon)
• Storage Resources can now support for more than one parent (e.g. both HotFolder and Folder Resource)
• Improved handling of inconsistent Timecode and Start Time values extracted from media assets
• Improved data validations during metadata merge operations

Fixed Issues Dalet Flex

• Fixed an issue where the Flex search services caused an error in FlexMAM.
• Fixed an issue where a new user email directed to Flex core returns a service error message due to permission level.
• Fixed an issue where creating TAR jobs was returning 404 and asset not found errors.
• Fixed an issue where sorting DESC was not working properly for tags (String Array) in fql request.
• Fixed an issue where indexing errors were causing assets to be not found in FMP.
• Fixed an issue where asset Syndications API response says ‘no records’ despite records being returned.
• Fixed an issue where the waveforms of audio assets are not displaying correctly.
• Fixed an issue where the attachment migration tool changed the file name after migration.
• Fixed an issue where proxies stored on S3 were not playing in the FlexREVIEW.
• Fixed an issue where Flex was not displaying any assets.
• Fixed an issue where DASH asset durations were only accurate to the second (rounded down).
• Fixed an issue where files were not deleted from their package asset when a purge action was run.
• Fixed an issue where no results are returned when a free text search contains a comma.
• Fixed an issue where published metadata-images disappear from publish API response of UDO after republishing UDO with "Republish metadata-images = false".
• Fixed an issue where an extract waveform job fails when an s3 folder is set in Account Metadata.
• Improved stability when indexing technical metadata for certain assets.
• Fixed an issue where using the tasks API endpoint was returning an error during a reindex.
• Fixed an issue where the Admin app was ignoring user permissions.
• Fixed an issue where keyframe size was inflated when importing from JPGs

Enhancements FlexMAM

• The FlexMAM now supports multiplicity (e.g. 0..1, 0..*, etc) on drill-down fields.
• User Defined Object memberships can now be viewed on the Group asset membership tab.
• Timecodes are now displayed as time@framerate instead of time,framerate.
• Attachments are now listed in the relationship tab for Group and Package assets, and user defined objects.
• The Technical metadata and Markers tabs will no longer be displayed for Placeholder assets.
• Enhanced search filter chip experience.
• App switcher will now only display the Admin app icon if user has permission to access it.
• Added the ability to make drill-down metadata fields read-only

Fixed Issues FlexMAM

• Fixed an issue where video assets inside a Group asset shows a duration of 00:00:00.
• Fixed an issue where the Marker edit button from the search view does not work.
• Fixed an issue where timeline markers were not available on package assets.
• Fixed an issue where image asset thumbnails are not displayed for image assets.
• Fixed an issue where clicking ‘x’ icon in the search bar changes search text to 'undefined' instead of clearing it.
• Fixed an issue where saving metadata was removing Key-Value and Multiplicity 0..* fields.
• Fixed an issue where the download icon is not available for assets, child assets and attachments after assigning permission via group role permission.