Dalet Flex 2024.3.0 Release Notes

Nisan Kowalsky
  • Updated

 

Dalet is excited to present the release of Dalet Flex 2024.3. 

This release of Dalet Flex provides the new features, quality and security enhancements, bug fixes, and other changes listed below. 

 

Release Highlights

  • Drill-down field picker in the chip search
  • Backend: denormalise Collection items to enable more complex searching, sorting etc. of items in collections;
  • Collections ACL model: remove account-level sharing;
  • P. Flex MAM: Membership of media assets in Pyramid objects;
  • OpAtom MXF (Avid integration): MVP new type, import, transcode and export;
    Permissions to access each UI;
  • ...and a whole bunch of other bug fixes!

 

Important Notes

Upgrade to Spring Boot 3

Most microservices have been upgraded to Spring Boot 3

 

Dalet Flex

New Features

Apply granular ACL rights for assets and UDOs

Dalet Flex now supports different ACL rights for assets and UDOs. It supports allowing specific users or groups to have read only access or read and write access. For backward compatibility, a DB migration script sets all exiting ACL to read/write access.

 

Permissions to access each UI component

You can now set permissions to control access to the Dalet Flex user interfaces

Role > Apps section

  • Flex Core
  • Flex Admin
  • FlexMAM
  • FlexMOVE
  • FlexREVIEW
  • FlexTRACK
  • Metadata Designer

You can now update the apps to allow access only when the selected user has the relevant permission from above.

In the DB, all apps now have a common prefix app.flex- to all name identifiers of entries in the `apps` Permissions section, and added FlexXTEND. E.g app.flex-core.

 

Backend: increased performance for Collection items to enable more complex searching, sorting etc.

 

Implemented FexMOVE with Aspera Integration

FlexMOVE has received a brand-new user interface to align with other Flex modules. Existing functionality will continue to be available including the selection of inboxes and bulk metadata entry.

FlexMOVE integrates with Aspera Connect for accelerated file uploads. Aspera configuration can be entered into the Flex Inbox Resource, and the user will be instructed to install the Connect client when they’ve selected the Inbox in FlexMOVE. The Connect integration supports the latest version of Aspera’s High-Speed Transfer Server (HSTS) 4.4 and the Node API.

 

OpAtom MXF is now supported in Dalet Flex for import, transcode, editing and export for the Avid integration, including Avid Log Exchange for exporting Edits from Dalet Cut to OpAtom. 

 

Dalet FlexMAM

New Features

Drill-down field picker in the chip search

For drill-down metadata fields, it is now possible to search within a given branch of that tree.

  • You have a lookup field
  • the lookup field show placeholder depending if you selected one already to contextualize the search or not
  • you can remove at any level of the hierarchy and the lookup will start to look at that point
  • this new picker is similar to the timecode or date picker.
  • For table or chip on table click on the column header and open the filter picker, for chip, edit an existing one, or select a new chip
  • if you click on refine, a new chip will be added alike the timecode or date picker
  • if you click out, nothing will happen
  • if you click on opening the picker with having one already defined, you're not editing the existing, but you are adding a new filter, alike date and timecode

 

Changelog

2024.3.0

Enhancements DaletFlex

  • Flex File Processor Resource allows Archive of source media assets to Google Cloud Storage (GCS) and Azure Blob (AZBS).
  • Added awareness and manageability of Flex assets in Pyramid and other NRCS systems, mainly the location of an asset in a rundown, and the stories and scripts in which that asset is embedded. 
  • External apps like Pyramid can now subscribe to message queue notifications of Flex to interact with the system.
  • Query GraphQL POST endpoint using JSON or application/graphql type can be passed as the request body as well as raw text.
  • Upgraded JAE service to build and run with Java 17.
  • Added validations to ensure that assets ingested incorrectly to the wrong storage location do not to cause all assets in the folder to be removed on Purge.
  • Introduced CORS validation to Spring-Boot applications. Validated for Metadata Designer App, Workflow Designer App, Web Transfer App.
  • Expanded JEF Aspera Transfer JEF Aspera for Export Action.
  • Improvements to indexing of taxons in Elastics earch: Bulk-update descendant taxon ancestry, when a taxon is renamed, deleted, enabled, disabled or otherwise changed.
  • IndexElastic: Improve the handling of taxon deletion events to ensure taxons and related taxonomy ids are not visible in Elasticsearch.
  • Deprecate individual filter parameters in ValueSuggestionRequest in favour of a single `fqlFilter` parameter. Currently support fqlFilter and existing query params.
  • Conform (other plugins): Add a new optional config param to select AssetOrigin. Parameter behaves as follows: If AssetOrigin field value is not null, use the value otherwise set AssetOrigin to Transcode. If Create Proxy is set to Yes in the Transcode action, ignore the configured AssetOrigin value and set AssetOrigin to Proxy.
  • Enterprise DB migrations can run with latest MariaDB 10.6.x.
  • As part of the Aspera integration, added the ability to provide a token for the Aspera inbox via API for the front-end application to connect to the aspera server for file uploading.
  • Image Transcode: AssetUpdate resilience on retry (failing with 500 because proxy reference name exists) Add resilience to image transcode jobs by improving the retrieval and reuse of existing placeholders on job retry. This will also limit the creation of ghost children during this process.
  • Added support for subpaths and file names in the Create Proxy actions for MPEG-DASH and HLS files. NOTE the temporary limitation: do not include file extension in the file name configuration.
  • Added CIP CAP, SCC subtitle formats and support for importing of these formats as Subtitle Assets as opposed to default Media Assets.
  • Allow metadata instances to be saved referencing deleted objects, taxons, tags etc from scripts. New parameter ‘allowDeletedReferences’ for API ‘PUT /api/{assetId}/metadata’ in Enterprise.
  • Added to Standard DNxHD 1080p 720p, DNxHR and verified others.
  • Improvements to indexing of UDOs in Elasticsearch: Bulk-update descendant UDO ancestry, when a UDO is renamed or otherwise changed.
  • For specified search criteria, and a specified individual result, return the page of results containing that result.
  • Event handler service: Catch all exceptions when applying an event handler's filter.

  • Fast Object service GET /api/objects/{objectId}/acl now returns the mode value as “READ_ONLY” and “READ_WRITE” for both Media assets and UDO.

  • JEF SDK using getAcl now returns the mode value as “READ_ONLY” and “READ_WRITE” for both Media assets and UDOs.

  • Provided a new tool, flex-technicalmetadata-migration, to unsure persistent data in case of media assets with drop frame, fractional frame rate (e.g.: 29.97, 59.94 and videoStream startTimecode not 0) with extracted technical metadata.

  • Updated syndications API to take new syndication properties.

  • Deleted syndication API endpoint when deleting embedded asset.

  • Due to a limitational change to read account/write account, customers who found the readAccount and writeAccount settings useful, can use a migration script for each Flex account containing collections, where they wish to retain the readAccount and writeAccount setting.

  • Amberfin: Add a control polling cadence for Amberfin launch actions in JEF configuration. To improve performance, default polling value is 10s.

  • The video/webm mime type now supports the WEBM format.

  • Syndication Events on Create, Update/Delete operations.

  • Fixed a problem where a Waveform types was extracted from the wrong object.

  • Fixed a bug related to *.r3d files that called FSP multiple times to get partial duration.

  • Added support for Cloud object Archive and Restore for GCS & AZBS.

  • Added the ability to set object tags for cloud storage providers on copy/move jobs.

  • Upgraded flex-registry-service, flex-replicationgateway-service, flex-sitesync to Spring Boot 3.

  • Increased thesaurus term max length to 100 characters in the Thesaurus Service.

  • Increased thesaurus term max length to 100 characters in the Metadata Designer.

  • Job Scheduler: added logic for Job loading and enqueue sequence in Job Scheduler logic updated.

  • Introduced a metric, dalet_service_info, capturing the service state (Boolean) and other metadata (version, runtime, runtime_name, runtime_version, runtime_framework).

  • Additional improvements to Job Scheduler to exclude jobs for which job quota for action configuration has been exceeded. Excluded jobs remain in a Pending state.

  • Improved VFS and VPS proxy efficiency by tweaking internal caching logic and adding a new Grafana dashboard to better monitor performance.

  • Added logic to support Avid AAF package assets as inputs to waveform, frame extract, stitch, sequence and transcode jobs.

  • Implemented pagination for enterprise API /api/users/<ID>/accessibleWorkspaces endpoint, to get accessible workspaces.

  • Removed the limit=1000 parameter when calling enterprise API to get accessible workspace and will now fetch accessible workspace batches of 100 records.

  • Added support to signal workflow manually if it stuck and not moving to next node using new API endpoint.

  • Added the “pluralName” property in the /assets/id/variables/id/values endpoint so UDO will return with their UDO type.

  • Fixed a problem that prevented the FMP app to update to latest service-support.

  • Updated FMP app to apply helmet/CORS options through service support init callbacks.

  • FSP now supports the the Keda (Kubernetes Event-driven Autoscaling) external push-scaler service to trigger job scaling directly.

  • The Flex UI now shows user in which ways an asset is used in Dalet Pyramid.

  • When using the feature toggle `flex/shared/flex-enterprise/featureToggles/queueJobOnCreation`, set execution token when job is directly set to queued at the time of creation.

  • Added support to ignore the timed out state for workflow when consul KV is set.

  • Pperational Keys Redisson typo has been fixed. Dalet will not support the typo versions after Flex 2024.3.0.

  • Support skipping the Global sync job running limit using a Consul KV. Validated the Consul KV flex/enterprise/workflow/enableSyncJobLimitCheck for Enterprise jobs that have flex-enterprise/featureToggles/queueJobOnCreation = true and flex/enterprise/scheduler/job/maxSyncJobsInProgress >1

  • Fixed a problem where delete action will not fail if it tries to move any folder location which is referenced by any of the storage resource in the Flex.

  • Purge plugin will fail if path is equal to the storage resource root.

  • Fixed severe FindBugs warnings caused by use of BeanUtils in MD Designer.

  • Added logic to try stream based copy on local filesystem if NodeJS fileCopy fails with EBADF.

  • Updated Authorization provider library to not clear cookies when security context has nullified authentication.In case of SAML logout, Flex need cookies to retrieve existing session from Redis.

  • Added ACL mode data in events.

  • Added the ability, for a specified search criteria and a specified individual result, to return to the page of results containing that result.

Fixed Issues DaletFlex

  • Fixed and issue in the Collection service where users could create collection requests and add collections as parents regardless of access
  • Fixed an issue where HotFolder path containing // or lowercase/usercase “S” in S3 naming would cause workflows to fail to launch
  • Fixed an issue where MimeType AIFF in the mediacore objects library was missing
  • Fixed an issue where the Create Proxy failed if the existing Reference Name contained the word 'proxy'.
  • Fixed an issue where multiple job scheduler threads were running on same master where only one was required.
  • Fixed an issue in the Flex Mobile app where only a single Thesaurus term was returned instead of a list of terms. The getTerm() query has been renamed getTerms().
  • Fixed an issue where transcoded MPEG-DASH files where created but not attached as child files to an asset. Issue solved by updating response handler to accept both format code and extension.
  • Fixed an issue where Workflows could get stuck in Pending status due to an issue with checking the workflow limit when a workflow was launched from a Hot Folder.
  • Fixed an issue where the annotation metadata end-point is not returning metadata definition reference object.
  • Collection service: Fixed an issue where there was overeager removal of parentUuids when processing subcollections by ensuring that subcollection deletion stops for branches that encounter subcollections not owned by the deleting user.
  • Fixed an issue for workflows contain many jobs. Performance improved by ensuring the batch sizes are not exceeded in MongoDB pruning and causing timeouts.
  • Fixed an issue where extract subtitle technical metadata action was throwing an NPE. Issue caused by WebVTT Cue identifier throws an NPE when text is followed by an empty line. Issue fixed by safeguarding against blank lines when parsing VTT files.
  • Image Transcode: Reset filename when job is retried using a forced FSP new execution.
  • Fixed an issue where Set Metadata API failed
  • Fixed an issue with the Login app SAML configuration. SAML configuration (response, assertation) is now overwritten based on the IdP returned response signing.
  • Fixed an issue where Workflows with timed-out jobs which were eventually completed did not change their status from Timed out back to Running.
  • Metadata fields with specified permissions are not returned when the query is made with masteruser credentials. To return results edit the metadata definition on the master account to give access rights for the filed to master user.
  • Fixed an issue where CORS rules do not allow for hyphens in hostnames in regex.
  • Fixed an issue where hotfolder import failed when the Storage Resource as used by the hotfolder and the file.
  • Fixed an issue where drill-down field were configured to reference a non-root UDOT.
  • Fixed a problem that rendered Flex Mobile unusable due to Enterprise rejecting the 'sub' claim as invalid for being an integer instead of a String
  • Improved the logging of error responses of the NodeJS Service Support library Enterprise Client
  • Fixed an issue so Get ExternalIds for an asset that does not have any externalIds returns an empty array and status code 200 instead of returning 404.
  • Fixed an issue where changes in asset metadata do not trigger the update metadata workflow. Issue caused because metadata modified events sent from the metadata service did not include workspace ID, so the event handler used the default workspace ID of the account so events for assets in workspaces could not be used to start jobs. Issue fixed by adding workspace ID to events generated by metadata service service.
  • Fixed a problem where the Flex Xtend panel did not successfully log on, even though all details were correct.
  • Fixed an issue where Flex Enterprise could not resolve name for UDO publish event data when set as backing store to drill down field.
  • Fixed a problem where requests for vol, mpd or m3u8 files or those with noCache query param did skip in the memory stat cache
  • VFS layer improvement in terms of file locking errors that caused image sequences in FSP to be considered growing.
  • Corrected both Dash manifest parsing logic to select correct sub-folder to scan and VFS layer file listing readable stream logic to ensure package format asset info does not recursively list unrelated sibling folders
  • Added a midway fix so that Flex does not throw an exception if multiple UDOs are retrieved.
  • Fixed a problem that prevented the restore poll logic to be invoked when restoring content from deep archive
  • Fixed a problem where incorrect error messages were logged in Kibana
  • Fixed an issue that caused existing custom markers to be wiped out when changing the metadata definition for that asset
  • Fixed an issue where searching for UDOs with SearchElastic, using the `approved` field would throws an error as the field is not yet indexed
  • On mobile devices, when playing video, Dalet Flex will not interrupt video when changing between portrait and landscape mode
  • On iOS devices, in reviewer and FMP pages, Flex MAM will display three dots if a Asset name had to be truncated due to its length.
  • Fixed an issue where it was not possible to edit thesaurus terms with more than 40 characters (thesaurus service).
  • Fixed an issue in Flex Core where a concurrent limit set for the account level is not accepting the given value when the workflow definition limit set for specific workflow and a Limit per Workflow Type (System) also have been configured
  • Fixed an issue where Start Timecode values are showing different in Enterprise and FlexMAM for 29.97/59.94 from Enterprise.
  • Fixed a metadata issue where the password field has been erroneously updated whenever modifications are made to other fields in FMP.
  • Fixed an issue where it is not possible to clear metadata password fields when Enterprise uses the metadata service.
  • Fixed a Publish action failing for asset with HLS or MPD proxy to complete without any error
  • Fixed an issue where Enterprise copy and move action is not working for S3 to S3.
  • When Tags, Thesaurus, UDO and Taxonomy are configured under the Metadata definition with Backing Store option then User is able to retrieve the information related to the backing store id.
  • Fixed an issue with Elastic search missing UDO hierarchy data from index
  • Fixed an issue that caused a system error on launching any extract job on an image asset. Now such jobs complete without error.
  • Fixed an issue where a user can able to see more than specified limit of workflows showing as running.
  • Fixed an issue in the Authorisation service where an exception was thrown when retrieving subaccounts
  • Fixed an CORS issue in Web transfer where including Drilldown and Tag Fields triggered errors.
  • Fixed an issue with the integration with NetApp S3 compatible Storage Resource where the storage path could not be resolved.
  • Fixed an issue where Running jobs in Executor (e.g. long running script) could create a bottleneck on cancelling jobs. Other jobs were not cancelled (not executed either, just waiting for slot to execute) until these long running jobs cancelled, leading to Job execution aborted events sent long after the jobs get cancelled.
  • Implemented a fix to show custom message when disabling workspace from UI and Workspace that either has Assets or Workspace supported UDO! However, after confirming, it will still allow to disable workspace.
  • You can now allow metadata instances to be saved referencing deleted objects, taxons, tags etc from scripts
  • Fixed a specific issue for specific client storage.
  • Tool to fix manifest provided for an issue where audio waveforms are not matching their audio channel.
  • Fixed an issue where an API of flex-tableau-publisher-service is throwing 500 internal server error.
  • Fixed an issue where SAML SSO Azure AD login would fail, by overriding SAML configuration based on IdP returned response signing.
  • Fixed an issue where an Adobe Premiere error was thrown when importing asset that has been conformed.
  • Fixed a problem where the extraction of technical metadata fails on files that have stopped growing
  • Fixed a regression that prevented the media restore plugin to trigger a restore process.
  • Fixed an issue in where an error appeared while building an authorization query

Enhancements FlexMAM

  • Added the ability to associate variantId without metadataDefinitionEntityId to a collection and create and update a collection with variant which has no metadata definition.
  • [Uploader]: Upload files with no metadata definition configured
  • Truncate long thesaurus terms (100 characters): when content in fields exceed the available space, the display will show a truncated amount but will show the full content in a balloon tip.
  • Time Code fields in Metadata forms now feature a button that allows you to fill in its value from the current position of the playhead, instead of having to type it in. This is only available for video assets.
  • Only display Delete Action in the More Actions list if the user has permission to delete.
  • Dalet FlexMAM, a user can now open a UDO drilldown menu on a different tab.
  • Updated names in the app switcher to reflect our product brand names: Admin > Flex Admin, Upload > FlexMOVE, Metadata designer > Metadata Designer, Core > Core, Operational Dashboard > FlexTRACK, Reviewer > FlexREVIEW, Cut > Dalet CUT, MAM > FlexMAM
  • Connected the new drill down picker to the UDO and taxonomy backstores as preparation for feature completion.
  • The drilldown typeahead functionality is now available for a table header. When refine is clicked, a new chip filter is displayed at the top. For each filter added from table column, a new chip representing filter is shown.
  • The applications in the global header of FlexMAM interface are now converted to stand-alone and the code has moved to a single repository (mono repo).
  • Now the audio track displays the label name in the player timeline, thanks to code refactoring.
  • Uploading local VO files, in AFF format, into a placeholder is now available. It generates an URL string configured in the account metadata and associates the URL with the edited asset while saving.
  • The VO asset name is now the combination of edit asset name and the file name “<<Edit Asset name>> <<VO asset name>>”.

Fixed Issues FlexMAM

  • Fixed an issue where scroll on search listing keep scrolling.
  • Fixed an issue where the pagination ‘item per page’ in the keyframe tab was missing. As well as navigating between tabs, the pagination value was lost.
  • Fixed an issue when downloading a keyframe from the keyframe tab showed a file name with invalid characters.
  • Fixed an issue where the FMP apps were unable to build multi-architecture images.
  • Fixed an issue where users were unable to complete or pause a task when trying to upload images in the metadata form.
  • Fixed an issue in the search result page where the date time was not following the time zone setting of the user.
  • Fixed an issue where the ‘Go to’ button near the URL field in the metadata form was missing.
  • Fixed an issue where it was not possible to update a market and the message `Invalid timecode` was displayed even when updating a marker with a valid timecode.
  • [Flex Cut] Fixed an issue where the notification message “Project saved successfully” was not displayed when clicking on the ‘Save’ button.
  • Fixed an issue when preloading an asset in the Asset Management page for mpd and m3u8 files did not work, the loading was delayed because the proxyURL value was set before preloadginEnabled flag value got set.
  • Fixed an issue when reverting from a foreign language to English, the Boolean parameters (search operators) in the filter options were missing.
  • Fixed an issue where the subtitle dropdown menu displayed the same subtitle replicated multiple times. This behavior happened occasionally after creating a media asset with one associated subtitle.
  • [Flex Cut] Fixed an issue where a pop-up window showing an error message was not display when unsupported image file formats (BMP, TIFF, AVIF and GIF) were uploaded to the timeline.
  • Flex Cut Fixed an issue where the Save and Cancel buttons in the interface had different names.
  • Fixed an issue where the link to an UDO from the membership section, inside the Relationships tab, from the Summary panel, was not redirected to the UDO. – relevant when 10K UDOs exist in the account.
  • Fixed an issue where in the operational dashboard page, for jobs and workflows tabs, assets names appeared as undefined
  • Fixed an issue where search text is overlapping with the placeholder text and arrow icon is not aligned with tree in Flex UI
  • Fixed an issue in FMP Uploader which did not trigger the Aspera installation when clicking the install link. The installation works fine now.
  • Fixed an issue with Flex Uploader that prevented the XSRF cookie to be set on first request to /fmp/transfer path.
  • Fixed a Hazelcast issue that would keep the FMP Uploader in a loop when loading. The Uploader now works as expected.
  • Fixed an issue that prevented any users to be shown and therefore, selected, in the form “Share with people and groups”.
  • Fixed an FMP issue that would prevent dropdown menu values from being listed for object fields in asset metadata page
  • Improvement in the Single marker navigation and selection for search results: when clicking the summary panel, it shows the single marker that is selected in the search result marker page, along with a toggle icon to switch to full marker list.
  • Fixed an issue when clicking on a UDO redirected to another UDO
  • In wizard, Object metadata fields with no udo type selected and set to typeahead, the browser crashes when there are a lot of UDOs. Issue fixed by adding a limit to the number of UDOs to load while using a typeahead field.

 

Related articles

Comments

0 comments

Please sign in to leave a comment.

Powered by Zendesk