Dalet Flex 2021.9.0 Release Notes

Dalet is excited to present the release of Dalet Flex 2021.9.0.
This release of Dalet Flex provides the new features, quality and security enhancements, bug fixes, and other changes listed below.

Dalet Flex

New Features

Improving pruning performance and preventing db timeout

Configuration guards have been added to flex-db-pruner to prevent pruning jobs failing when execution is run too often with a batch size that is too large.
A hard maximum of 10,000 for the Job/WF (MySQL) pruning batch size and of 100,000 for the Event (Mongo) pruning batch size has been set.
In Consul, you can configure the batch size for a pruning process.
The system records the last run start time, end time, job/wf pruning duration, and event pruning duration. At the start of a new run, the metrics of the last run are assessed, and if:

• Total job/wf pruning duration > 4 mins, flex-db-pruner autoscales the job/wf pruning batch size down accordingly
• Total event pruning duration > 4 mins, flex-db-pruner autoscales the event pruning batch size down accordingly
• The last run finished less than 2 minutes ago, flex-db-pruner aborts the pruning run logging an error to point out that the configured pruning interval is too short
• The last run start time is more recent than the last run end time, (indicating that a failure occurred or that the concurrency lock was broken somehow), flex-db-pruner aborts the pruning run and prevents future runs for the next 2 hours with the error: Last recorded run start (2021-10-08T11:00:01Z) is AFTER than last recorded run end (2021-10-08T08:32:02Z) - the last pruning run must have failed. Aborting this and future pruning runs until at least 2 hrs have passed since last run start

Column layout for complex metadata fields

This release introduces support of a column layout for complex metadata fields in FMP. A column layout improves the way connected metadata is displayed and makes better use of the space in metadata forms. To configure a column layout for complex metadata fields, a new option Column Layout has been added to a Complex metadata field in Metadata Designer. Select this check box to display the fields inside the Complex in columns. Note these properties when configuring a column layout complex metadata field:

• Supported field types: string, password, text, integer, long, float, double, single-option.
• When a non-supported field is added to a column layout complex, the complex and the invalid field are highlighted in red and you cannot save the metadata schema
• The maximum recommended number of fields to add to a column layout complex is four
• To avoid issues, dynamic warnings are displayed as you configure a column layout complex

Use Storage Resources as a target destination for an import action

You can now configure a Storage Resource as well as a Folder Resource as a target destination for media when you create an import action.

Dalet FlexMAM

New Features

Column layout for complex metadata fields

See above

Launch wizards from timecode range and markers

It is now possible to configure a wizard in Flex Enterprise specifically for timecode ranges in FlexMAM. Once the Launch From: “Timecode” checkbox in the wizard configuration has been checked, the wizard will show up in FlexMAM after marking an in and out range under the preview player.

**NOTE:** New and existing wizards **MUST** have the “Timecode” box checked in order for them to show in the FlexMAM timecode range wizard dropdown menu.

Changelog

2021.9.0

Enhancements DaletFlex

• To remove unrelated search results, a free text search makes non-wildcard free text queries always use the AND operator and not the OR operator
• When extracting technical metadata, tags beginning with < are only restricted if they end with > or space so technical metadata extraction will no longer fail
• Support configuring S3 resource with a direct AWS Cloud Connect URL
• DB Pruner configuration guards. Configuration guards have been added to flex-db-pruner to prevent pruning jobs failing when execution is run too often with a batch size that is too large.
• Fix discrepancy in get metadata response call between Enterprise and FastObject
• Flex Web Uploader supports uploading XML files (brought back lost functionality)
• JEF Cloud archive restore action event augmentation to include the asset size and the restore class
• When performing a Bulk Action in Enterprise, the workspace check is skipped so that assets are loaded without a permission check
• Skip workspace check set to true when performing a Bulk Action to cancel multiple workflows
• Allow single-option fields in complex column layout in metadata designer
• "Keep technical metadata" option in "Purge file only action" updated to Keep Technical Detail and Markers
• Implement IRSA to allow services running in EKS to acquire S3 bucket credentials in addition to Instance Metadata credentials
• Added an option to synch IDP groups with Flex using groups external ID rather than the Flexuser group name.
• Timecode check box option has been added to the "Launch From" options when creating Wizards.
• Admin app saves a user setting for default search on FMP login
  - In Enterprise API, less than (<) metadata field searches, you now add the less than symbol (<) before the value in the search query as opposed to after as was required previously
  - Specifying description and introduced fields of FeatureToggles is mandatory
• AAF parse and generate support to support importing referenced source assets
• Import action works with both Folder and Storage Resources as target destination
• SpringBoot: Include request correlator in logs
• Update from legacy index templates to composable index templates
• Remove collection related data from database as part of exposing collections via Enterprise API/SDK
• Ensure clip outpoints/duration reflect documented functionality in ffmpeg, mtrim and fmpe
• Add expected vs actual output format and stream duration checks to sequence and stitch jobs
• Provide output files in job response and ability to also extract output file info for package, stitch, sequence
• Column layout for complex metadata fields introduced
• Membership capabilities added to Flex Edit
• Taxonomy pagination implementation reverted
• Align video context enum values between Flex Enterprise and Flex FastObject Service
• Add missing hash fields to the asset end point
• Mechanism implemented to retrieve required minimum playable metadata required by FMP from Image Sequence TIF, DPX and other formats
• JEF Import Edit: AAF Parse ExecutionStep to call TR added to FSP
• Flex Project/IMP/CPL (package), Flex Edit/A02/CPL (CompositionAssetTrack), Flex Sequence and Assets(Media, Subtitle…) ;dependencies upstream (genealogy)
• Flex Project/IMP/CPL (package), Flex Edit/A02/CPL (CompositionAssetTrack), Flex Sequence and Assets(Media, Subtitle…) ;dependencies downtream (genealogy)
• collection lookup by item details - matched item keys field is returned for get item collection request
• Support for MVP Column layout for complex metadata fields added
• Automatic migration so that New Collection2 object type is used for all collections instead of old Collection object type
• JEF Export Edit: Export Edit action created to export existing Flex Edit (aaf) without referenced sources
• JEF Import Edit: Import Edit plugin modified to import referenced source assets: AAF from Avid Media Composer + referenced MediaAssets
• [Authorisation support] Improvements to MetadataService to introduce user permission checks. User can only set Metadata Definition Entity on Collection/Asset/UDO/Annotation if they have REFERENCE permission on Metadata Definition Entity
• Set Asset using JEF Groovy Script Context plugin

Fixed Issues DaletFlex

• Fixed an issue where execute TAR was not working due to wrong memory requests and limits
• Fixed an issue in Flex Image Proxy Service where there was an error while fetching image asset with request param "loadParentAsset=true". Fixed by adding a new validator for proxy request.
• Fixed an issue in Flex Taxonomy Service that requesting a single taxonId value did not return the value, even if it exists
• Fixed an issue that the system allows a user to get set metadata with metadata service even if the user does not have reference permission for the metadata definition
• Fixed an issue that AWS instance credentials were not being set when running a Cloud archive job
• Fixed an issue that newly added collection items were not being returned in FMP evethough they were being returned using an API query. The issue was resolved by clearing the flex-authorisation-service.rolesmap cache on the platform.
• unable to create placeholder asset and default ingest , it keeps loading
• Allow/Skip multiple flex object (Asset) in entity mapping migration
• Fixed an issue where Publish to Facebook failed with Java out of memory error
• Fixed an issue where proxy creation is failing because transcode clipping does not work for image sequences containing slugs with different frame sizes
• Fixed issue where Start time code is not available for AS02 package on player by adding startTime/startTimecode in format, video and audio stream context for get asset API call on AS02 assets
• Rio Broker restore action is not supported for unmanaged assets
• Fixed an issue where import jobs were failing because of mis-identification of DPX files by Mediainfo
• Fixed an issue where getMioObjectListVariable throws an NPE if variable does not exist
• Fixed issue where the Timezone set in Enterprise for a user was not returned in an API call to get user details
• Fixed an issue so that the API call to users/5/metadata give a meaningful message if the user does not exist instead of throwing an NPE
• Fixed an issue that a system error message was displayed when clicking on any asset in advanced search or in Asset View in Enterprise
• Set the behavior for sorting search results that return missing values. When sorting in ascending order, missing values appear first and when sorting in descending order, missing values appear at the end.
• Fixed an issue where a GET asset returns 500 error after PATCH ImageSequence frameRate update. After the fix, framerate and PixelAspectRatio are updated as num/den in ArangoDB and only fields in the image-sequence-metadata-definition can be updated.
• Fixed an issue where a DateListVariable passed to a workflow variable shows a parsing error. DateListDeserializer has been added to parse the list of dates.
• Fixed an issue so that “managed” matrix parameter on the List Assets API handles the value all as expected and it is equivalent to no parameter being specified
• Fixed an issue where the system did not allow you to create a new UDO instance with the same name as a deleted UDO instance
• Fixed issue that searching with any value for ;originalFileName =, Flex Enterprise API returns this error: "SQLGrammarException: could not extract ResultSet"
• Fixed an issue in Flex Enterprise and in Flex REST whereaAsset type for image sequence was not being set while searching for image sequence and hence ES was searching from all the objects and was showing wrong results
• Fixed an issue in REST API where the DeleteAnnotation class was used for both input payload (where the originatorCorrelationId and originatorContext are required) and for the response payload (where the itemsDeleted field is populated. The fix also fixed the incorrect display in the REST API documentation.
• Fixed an issue where dummy metadata fields are not transformed with var-???- prefixes prior to indexing by applying the same transformations to dummy metadata that are applied to real metadata
• Fixed an issue where Flex Event Handler Service was throwing an NPE on saving the details page of Master Account. Issue solved by adding master account to account resource type in the objects type library
• Code updated to support setting metadata with value attribute for date and time variables
• Added a new parameter defaultRoleName to the pull command to pull the defaultRole by name
• Behavior changed so it is possible to save a metadata definition if a variable name has the same name as any variable's child name
• To solve an issue where user details are not being updated after edit in Flex Enterprise, it is important to refresh cache to clear objects in memory cache
• Fixed an issue where player in Reviewer is not playing media asset in a satellite deployment test environment
• Fixed an issue when metadata merge was not working for Annotations. To solve the issue the metadata definition is loaded by the metadata definition ID stored on Arango DB and the metadata merge is successful.
• Fixed an issue where the username of the user who has modified tags for an Account type tagCollection is displayed incorrectly in the events log
• Fixed an issue where subtitles are not available when playing assets in Reviewer player
• Fixed an issue where Metadata Merge fails when adding/moving variables inside complex variables
• Proxy creation on TIFF sequence stuck on running because the TIFF files do not contain frame rate information
  - Fixed issue that information in video stream context was missing when importing an image sequence causing issues in viewing the media in FMP

Enhancements FlexMAM

• Timeline comments sorting options updated to Newest/Oldest and Timestamp In in ascending/descending order. Sorting by Timestamp Out has been removed.
• Improvements in green line indicator behavior when selecting tabs in FMP
• Suppress the 10,000 search result toast notification
• New icon added for Timecode range and marker wizards triggered using the new "Timecode" Launch From option
• Tooltips have been added to the App Switcher for those apps whose name is truncated in the GUI
• Services versions tab in FMP has been renamed Versions and updated to Flex MAM and Flex REVIEW.
• Tags and taxons are displayed with a strike-through on Search Listing and on Metadata Form
• The history tab in the Summary panel only displays a designated list of event types and messages and a tooltip on the profile icon displays the first and last name of the user who generated the activity.
• Remove old collection dependency code/data from FMP
• Support single-option fields in complex column layout
• Metadata column layout available in metadata forms in FMP

Fixed Issues FlexMAM

• Fixed an issue selecting a collection task redirects to collection metadata but metadata on summary panel does not open up
• Regression-Fixed an issue where annotation timeline icons are not displayed on player area and marker listing page
• Fixed a regression that the Download icon in the Manage page right panel could not be clicked
• Fixed an issue so that the Marker tab in summary panel is displayed for assets which had 'Purge File Only' action executed on them
• Fixed an issue with the alignment of the sequence player when changing the size of the player. Player should always be centrally aligned.
• Fixed an issue of a discrepancy of one to two frames between the timecode displayed on the player video and the timecode displayed below the player.
• Fixed an issue where dragging and dropping a collection item was not dropping in the selected position
• Fixed a UI issue in the chipbar on Marker listing page
• FMP options for searching for formats updated in the assetContext->formatContext code
• Fixed an issue where selecting Search with typeahead variable in the Metadata Summary tab does not add a search chip
• Fixed a regression issue where the Save button in Edit Template is not enabled if Share this template checkbox value is changed