Dalet Flex 2023.12.X (LTS) Release Notes

Dalet is excited to present the release of Dalet Flex 2023.12 (LTS)

This release of Dalet Flex provides the new features, quality and security enhancements, bug fixes, and other changes listed below.

Flex 2023.12.0 is an LTS (Long Term Supported) release track and will receive both critical bug fix updates and security updates until it reaches end-of-life.

Important Note about FlexREVIEW

This release, 2023.12.0, is the last supported LTS using the legacy FlexREVIEW.

From the next LTS version, customers should upgrade to the new FlexREVIEW app (service name: flex_fmpreviewer_app).

New Features

Flex Mobile App

The Flex Mobile app now supports launching workflow wizards.

The Flex Mobile app is now compatible with FlexREVIEW. Using the dedicated switcher at the top of the screen, users can switch to Reviewer mode. This mode presents a list of assets to be reviewed, alongside sorting and search options.

Baton Integration

JEF Baton API (from version 8.3, 8.4 supported) has replaced the older Enterprise Baton QC Resource and Action. Main highlights of this integration:
- JEF
- Mounted File and S3 stored source media
- Existing functionality: attach QC results, partial verification, test plan parameter overriding

Dalet CUT

Flex enables fast-paced production workflows with Dalet CUT, an intuitive web-based video editor with advanced capabilities: a web-based video editor with a panel to access Flex library; multiple video and audio tracks; transitions and effects support; voice over; while scenarios support; central rendering of editing projects. 

Dalet CUT now supports saving the project to FlexMAM as a Flex Edit asset, based on an AAF file, retrieving that asset for later work, and sending that project to be rendered by FSP.

Dalet CUT can now export its AAF file to a FCPxml file to support the Flex Sequence asset as well as the AAF Edit asset.

Following previous advancements with growing files, Dalet CUT users can now use growing media in the Trimmer and in the Timeline, continuously preview them as they grow, use clips from growing media in sequences and projects, and render said clips. FlexMAM users can also trigger rendering jobs on projects in which clips come from growing media.

Dalet Brio Scheduler Integration

The Dalet Brio Scheduler is now integrated with Dalet Flex, allowing recorded events in Brio to be analyzed and previewed in Flex. When an event is scheduled in the Brio Scheduler, a placeholder is created in FlexMAM. When the recording begins, both high res and proxy formats are created and sent to FlexMAM. Technical metadata is extracted at the start (currently, updated technical metadata during the stream is not supported). FlexMAM users can then preview the growing media as it being recorded by Brio.

Out of scope for this feature: Dalet CUT and FlexXTEND; Formats other than MXF OP1a (high res) and Mpeg-DASH (proxy); Waveform generation; growing technical metadata.

Dalet Instream Integration

The Dalet InStream ingest integration with Dalet Flex now supports dedicated sets of metadata fields related to recordings. These values are inserted by InStream and can be updated via the API. This metadata is read-only for FlexMAM users.

As part of the Dalet InStream integration with Dalet Flex, when a recording is scheduled, a new placeholder is created. When the recording starts, the placeholder will become a Master asset, in case of high-res video only; or a Master/Child set of assets, in case of high-res and proxy files.

On-Prem Storage Support

Added support for inboxes/http resources/web transfer backend app on satellite infrastructure so that uploaded files go directly to their on-prem location.

Added support for hot folders configured on satellite on prem infrastructure. Files copied to these locations are then pushed to cloud storage.

Growing files on FlexXTEND

FlexXTEND now supports editing growing high resolution files in Adobe Premiere. This includes triggering a render job on a local or remote AME on clips from growing high resolution media. Current supported format: AVCi100 MXF.

New FlexMOVE interface and integration with Aspera Connect (from 2023.12.2)

FlexMOVE has received a brand-new user interface to align with other Flex modules. Existing functionality will continue to be available including the selection of inboxes and bulk metadata entry.

FlexMOVE also receives an integration with Aspera Connect for accelerated file uploads. Aspera configuration can be entered into the Flex Inbox Resource, and the user will be instructed to install the Connect client when they have selected the Inbox in FlexMOVE. The Connect integration supports the latest version of Aspera’s High-Speed Transfer Server (HSTS) 4.4 and the Node API.

Notifications for New Tasks (new from 2023.12.2)

Tasks have been connected to the notification framework so that users are notified when new tasks are available in either of the My Tasks or Available Tasks lists. When new tasks are added to the list a count appears in the Tasks sub-tab. A Refresh button is displayed in the Available Tasks to update any new tasks that have not yet been displayed. When the Tasks sub-tab contains new items, a dot notification is displayed on the notification bell in the Global Header, in the same way it is displayed for new Workflows.

Mitigate Risk on Segmented Asset purge/move/copy/delete-with-move operations, e.g. MPEG-DASH not in individual and unique folders (new from 2023.12.3)

When Flex Stream Processor creates a Flex Segmented Asset, it creates a unique folder that contains the manifest file and media files related to the Flex Segmented Asset. When performing copy/move/purge/delete with move the Flex Segmented Asset, Flex was assuming that all relevant objects are auto-contained in the “parent” folder, so it went to the parent container (mpd|m3u8) and performed the actions on this folder.

MPEG-DASH (.mpd) files created in Flex by a 3rd party, such as Brio, are not created in unique folders but are created in a flat folder that contains multiple manifest files and multiple media files. This means that when Flex tries to copy/move/purge/delete with move files, there is no unique parent file to perform the action on and unconnected Segmented Assets are also copied/moved/purged. This risks impacting other files in this folder not belonging to the Segmented Asset.

This release includes multiple enhancements and bug fixes that address functionality issues discovered with respect of Flex Segmented Assets. The improvements mean that copy/move/purge/delete with move actions support flat folder structure for Segmented Assets. The implemented checks and fixes ensure that in any case where the folder targeted by the action might contain files unrelated to the Segmented Asset, the operation fails. Instructions on how to identify Segmented Assets that might have an issue with Asset Import and Purge Action plugin scan be found here.

Improve optimization of volume of jobs consumed by Job Executor 

This release includes multiple enhancements and bug fixes that optimize the volume of jobs consumed by Job Executor with long and blocking jobs, for example scripts that take a long time to complete. The enhancements include improving warning messages, job distribution, restart performance and more. 

Changes to semantics in VFS affect Copy/Move API with API behavior (from 2023.12.3)

JEF Copy/Move: Changes to semantics in VFS mean that any scripted actions making use of FFP API to perform copy/move may be affected if the source location is a folder. In this case the FFP job will now move/copy the source folder (including the folder itself and not just its contents) to the target. To ensure just the contents of the folder are moved/copied, the source VFS location should include the property includesRegex: ".*" . Further details on this change to the semantics for copy/move are outlined here.

Known Issue

Due to implementation of CORS validation to Enterprise endpoints called by Flex Core UI, Flex Enterprise does not accept "null" (null as a string) and "" (blank string) as a valid origin headers.

And more enhancements and fixes below

Changelog

2023.12.3

Enhancements DaletFlex

• Added new endpoints to the jobexecutionmanager service to help support JEF-related diagnostics.
• FFP Plugins: Copy/Move (might be others in FFP or FSP plugins) protect moving/copying segmented asset with manifest folder with the same root folder as the Storage Resource base path.
• Implement server side validation for editable attribute. Validation is applicable for FlexMAM, FlexMOBILE and Xtend.
• Add KV flex/flex-enterprise/defaultWaveformTypeIfNull to set value of waveform type when asset.waveformType is null. Default value uses WAVEFORM_IN_BASE_FOLDER.
• Add event data for medadata definition modified event to indicate what changed, for example which fields are added, removed, reordered etc.
• Add support for quota service to JEF SDK.
• Add support for MessageTemplate and TimedAction services to JEF SDK.
• Support output profile with interlacing e.g. convert progressive to interlace or change field order to support mixed source conversion for stitch and sequence jobs.
• Support skipping the Global sync job running limit using a Consul KV. Validate the Consul KV flex/enterprise/workflow/enableSyncJobLimitCheck for Enterprise jobs that have flex-enterprise/featureToggles/queueJobOnCreation = true and flex/enterprise/scheduler/job/maxSyncJobsInProgress >1.
• Support .tcd timecode files for proxies.
• Invert the workflow state check in job scheduler to continue queueing jobs if it is NOT in pending state.
• FEATURE: Add support for `packetCount` property returned in assetInfoExtract jobs. Specify the property by setting enablePacketCountExtraction=true in the job spec or by default with the Consul KV set flex/flex-streamprocessor-service/enableDefaultExtractPacketCounts=true.
• Improve the performance of job summary by applying an index on last_modified column of mio_job table.
• Updated legacy Flex Enterprise Copy and Move action to fail the job if manifestFileName == mediaFolderName with a recommendation to use JEF copy/move action plugins when media folder is at the same level as manifest file for segmented asset. Also, added new checks when the job runs i.e. "Multiple Manifest Check" and "Other Files in Same Package".
• Auto-retry for Purge can fail due to file extension check
• Delete and Purge Action should complete when physical file is already removed.
• Collection service: allow KV to enable recursive collection deletion, regardless of sub collection ownership. Set flex/flex-collection-service/ignoreSubCollectionOwnershipWhenDeleting to force recursive collection deletion, regardless of the ownership of any descendant collections. This is not the default behavior.
• Add index for metadataDefinitionId field in metadata collections in Arango db.
• Add heuristic calculation of H264 bitrate (for AVC) in MXF bitrate AVCSubDescriptor is not available.
• Add meta tag <meta name="robots" content="noindex, nofollow" /> to login HTML to prevent search engines from indexing the Flex login page.

Fixed Issues DaletFlex

• Fixed an issue where Master url and others get indexed by most popular search engines and are displayed in search engines.
• Fixed an issue where a Decision script trying to cancel continuously from ExecutionStep and hammering Enterprise with events. Issue solved by maintaining a list of running jobs to avoid bottleneck on cancelled job cleanup.
• Fixed an issue where RioBroker Restore Plugin is stripping Off Top Level '/' path when setting Asset Current Path causing restore to fail due to an incorrect path.
• Fix Mapstruct mapper for legacy script getAsset.
• To prevent corruption of Arrango DB and its being irrecoverable after one host is down. For Spring Boot Apps using Arango: Replication factor of `arangolock` / `dbchangelog` collections should be at least 2. For existing environment collections this should be done manually.
• Fixed an issue where JEF copy/move Failed when moving Segmented asset. Issue fixed by including the package name in the destination vfs location.
• Add checkbox for optional configuration parameter to Copy/Move protect moving/copying whole SR if Segmented Asset is in root: support copying/moving if filename manifest==media folder
• Improve performance and messaging for API addKeyframe rest call. In case of ConflictException (keyframe already exist) during addKeyframe Enterprise rest call, add warning with proper message in job history and mark job complete. In case of OptimisticLockException/StaleStateException (Asset object updated simultaneously) during addKeyframe Enterprise rest call, job fails with proper message and retry solves the issue.
• Fixed an issue to ensure backwards compatibility for FFP restore jobs after restoreTier property renamed to restorePriority.
• Create new JBPM context for service info metrics and then close it after serving the request to retrieve database connectivity information.
• Fixed an issue where FO objectsid endpoint returns inconsistent results.
• JEF SDK: Fixed an issue where it was not possible to read account metadata using JEF SDK. Issue solved by retrieving account metadata as a JSON  document instead of as a metadata instance and setting account metadata using a JSON document instead of a metadata instance.
• Fixed an issue where creating a composition proxy on Flex CPL assets which have the same trackId for audio/video/subtitle sequences was creating proxies with incorrect durations. Issue fixed by creating a single sequence for the same trackId by consolidating all the clips with the same trackId.
• Fixed an issue where Extract Keyframe set the wrong master keyframe.
• Fixed an issue where Task dashboard shows incorrect number of tasks for users that have more then one role (via user group membership).
• If an asset is already associated with UDO, do not throw an error when trying to associate the asset again to the same UDO.
• Hot folder service: Fixed an issue where errors when retrieving the outcome of async file processing leads to reimporting of all files in resource. Issue was reported for hotfolders configured with max depth ==0.
• Fixed an issue where it was not possible to retrieve taxons with apostrophes using API and SDK.
• Fixed a system error that was thrown when creating Quota config for with processes plugin type.
• Fixed an issue where it is not possible to save metadata changes if the metadata schema contains non editable fields, even though these fields are not being edited.
• Fix an issue where an NPE was thrown for multi-asset import action because of an issue with auto-wiring.
• Job Scheduler updated to ignore maximum job limit keys with value zero in Redis.
• Fixed an issue where it was not possible for master user to fetch metadata fields that had permissions configured against them for other users. Master Account user can now access all fields regardless of which permissions are set for the field.
• Fixed an issue where Timed action failed and didn't restart. Issue fixed by, a new job being created when Timed Action jobs fails/cancelled and this message is displayed in the job history ` Retry the job again, new job will be triggered.
• Resolved an issue where JAE/Redis outage leads to QUEUE timeout and job active in 2 nodes. After Timeout, extract waveform cannot be retried because the json already exists.
• Resolved issue of jobs timing out after being in a Queued state.
• Fixed a UI error when creating UDO with the same name as already exists.
• Fixed an issue where deleting a single external ID via the legacy SDK with Groovy script deletes all of the external IDs.
• Fixed an issue where JEF Wait for Script gets timed-out instead of running for a long time. Fixed as part of optimization of volume of jobs consumed by Job Executor.
• MSE Proxy folder deletion remediation: Update purge and delete actions to support flat folder structure for Segmented Assets.
• Fixed an issue with slow pruning of standalone jobs. Issue solved by adding a new index and forcing the stored procedure used to prune standalone jobs to use the new index.
• Fixed an issue where incorrect frame rate was being retrieved during a JEF Extract technical metadata job.
• Fixed an issue where the Start Session Action was not starting in FlexMAM.
• AdobePanel: Fixed an issue where multiple audio name labels are not displayed in FlexXTEND.
• Sporadic errors in FSP Fixed an issue where an error polling the growing state of an asset triggered this error Growing state of job is undefined after 4 minutes on other jobs starting at the same time.
• Fixed an issue where it was not possible to create Oauth2 message action.
• Fixed an issue where Purge action is failing for all Package assets. Issue fixed by updating resource root validation check to consider filename when comparing asset location with storage resource root location.
• Fixed an issue where it was possible to edit non-editable variables keyvalue, vfsLocation and timecode when using the asset metadata API with fromUi parameter.

Enhancements FlexMAM

• [New FlexMOVE]: Support uploading files before metadata is filled in. Import workflow is triggered once the metadata has been added.
• FlexMAM: Display Format Variant in Technical Metadata UI. Displayed in technical Medatata tab, Chip filters, Listing of table and tile templates.
• Not possible to update non editable fields of an Asset/UDO if "fromUi" header is sent when updating metadata.
• Shift the logic of checking duplicate UDO names to Enterprise and remove from FlexMAM.
• Enhance the scrubbing behavior of Flex Player  

Fixed Issues FlexMAM

• Fixed an issue where subtitle switcher is not available in the task panel player for the asset associated with the subtitle. Subtitle switcher is now displayed.
• Fixed an issue where an incorrect tooltip (object object) is displayed in Workflow Notifications panel.
• Fixed an issue where the search value in the chip bar was not displayed when using searchable icon from the metadata tab. The username displayed in the typehead user field differed from the asset listing and user field selected type. The values for both scenarios were undefined.
• Fixed an issue where manifest extraction failed to identify audio url so that the MPD file could not be previewed.
• Bulk Panel: Fixed an issue where the "Add to Collection" and "Edit Metadata" option under Quick Actions for bulk assets are not working correctly.
• Fixed an issue where it was not possible to filter on a metadata field in FLexMAM.
• Fixed an issue where the configuration for 'Duplicate Name Supported' field for root UDOs and child UDOs was not properly respected.
• Fixed an issue with the date filter modals.
• Return ability to use a shortcut to create clips and to create markers via hotkey on the summary panel.
• Fixed an issue in the Left Collection menu that was sending multiple network calls from the browser so collections are loading information for a long time after page refresh. Issue fixed by adding a cache of 6 seconds when loading sub-collections.
• Fixed an issue where the Upload button was not enabled. Issue caused because the CSRF token was not cleared on logout and an invalid token was sent on re-login.
• Fixed an issue where the audio track waveform is not displayed in timeline.
• Fullscreen: Fixed an issue where it was not possible to minimize the screen after selecting the mark with metadata icon and moving to Full Screen.
• Shaka Player – Fixed an issue to perform pre manifest processing to support a manifest file delivered with no audio details. After fix, all audio tracks are displayed, audio switcher works and audio waveforms are displayed correctly.
• Support HEIC Format Upload via Flex Web Transfer.
• Support HEVC extension Upload via Flex Web Transfer.

List of Services upgraded between 2023.12.2 and 2023.12.3

2023.12.2

Enhancements DaletFlex

• Added validations to ensure that assets ingested incorrectly to the wrong storage location do not cause all assets in the folder to be removed on Purge.
• Added PixelFormat and hence bit per sample as listed enum field strings.
• Conform (other plugins): Added a new optional config param to select AssetOrigin. Parameter behaves as follows: If AssetOrigin field value is not null, use the value otherwise set AssetOrigin to Transcode. If Create Proxy is set to Yes in the Transcode action, ignore the configured AssetOrigin value and set AssetOrigin to Proxy.
• Added resilience to image transcode jobs by improving the retrieval and reuse of existing placeholders on job retry. This will also limit the creation of ghost children during this process.
• Gave a common prefix to all name identifiers of entries in the `apps` section of Permissions, add FlexXTEND.
• Added missing CIP CAP, SCC subtitle formats. Support importing these formats as Subtitle Assests as opposed to default Media Assets.
• Validated Min.io storage configuration for S3 protocol.
• Improved formatting for the Aspera Connect integration with Webtransfer (capitalization, file size units)
• Event handler service now catches all exceptions when applying an event handler's filter.
• Amberfin: Added a control polling cadence for Amberfin launch actions in JEF configuration. To improve performance, default polling value is 10s.
• MATROSKA mimetype added to support WEBM format.
• Fixed a problem where a Waveform types was extracted from the wrong object.
• Set the event correlation_id to the job Id for Event type PUBLISHED to fully enable usage tracking.
• Added support of Hazelcast locks for JEF scripting in addition to Redisson locks.
• When using the feature toggle `flex/shared/flex-enterprise/featureToggles/queueJobOnCreation`, set execution token when job is directly set to queued at the time of creation.
• Added Consul KV `flex/enterprise/workflow/ignoreTimedOutState`. When set to true the status of a workflow is displayed as Running and not move to Timed Out even if jobs associated to the workflow are moved to Timed Out. Default value is true.
• Added validation in the Purge plugin to fail a purge job if it tries to delete any directory which is configured as a storage resource root location.

Fixed Issues DaletFlex

• Added support for AIFF mapping mime-type in mediacore objects library.
• Fixed an issue where transcoded MPEG-DASH files where created but not attached as child files to an asset. Issue solved by updating response handler to accept both format code and extension.
• Fixed an issue where Workflows could get stuck in Pending status due to an issue with checking the workflow limit when a workflow was launched from a Hot Folder.
• Collection service: Fixed an issue where there was overeager removal of parentUuids when processing subcollections by ensuring that subcollection deletion stops for branches that encounter subcollections not owned by the deleting user.
• Fixed an issue where extract subtitle technical metadata action was throwing an NPE. Issue caused by WebVTT Cue identifier throws an NPE when text is followed by an empty line. Issue fixed by safeguarding against blank lines when parsing VTT files.
• Image Transcode: Reset filename when job is retried using a forced FSP new execution.
• Fixed an issue so Dozer is initialized once when a JEF script is executed and is then reused.
• Fixed an issue with when trying to perform setAssetMetadata and was throwing this error `FlexObjectReference; class invalid for deserialization`. Issue fixed by adding default no args constructir to FlexObjectReference.
• Fixed an issue so Get ExternalIds for an asset that does not have any externalIds returns an empty array and status code 200 instead of returning 404.
• Fixed an issue where changes in asset metadata do not trigger the update metadata workflow. Issue caused because metadata modified events sent from the metadata service did not include workspace ID, so the event handler used the default workspace ID of the account so events for assets in workspaces could not be used to start jobs. Issue fixed by adding workspace ID to events generated by metadata service service.
• Fixed a problem where requests for vol, mpd or m3u8 files or those with noCache query param did skip in the memory stat cache.
• Corrected both Dash manifest parsing logic to select correct sub-folder to scan and VFS layer file listing readable stream logic to ensure package format asset info does not recursively list unrelated sibling folders
• Fixed an issue with Events Handlers requests hitting the Events API before the corresponding event has been fully created, resulting in a false error. Fixed by adding retried to the Get Events API /api/events/<eventId>, by default at 0, 1.5, 4.5 seconds, but can include also attempts at 10.5, 22.5, 46.5 seconds. The configuration KV is flex/flex-events-service/getEvent/maxAttempts can be set to any integer 1-6, default 3.
• Fixed an issue where with S3 calls to get acceleration configuration for none supported region.
• Updated Spring-Boot apps to fix an issue where a trailing slash was added and HTTPS was downgraded to HTTP.
• Fixed an issue in the Xtend Adobe Panel extension failing. Fixed by allowing file:// as a CORS origin for the Login App.
• Fixed an issue in the History tab where the contents of the table did not clear on auto refresh.
• Fixed an issue to support NetApp S3 implementation so it is possible to start/enable the netapp storage resource.
• Fixed an issue where it was not possible to edit thesaurus terms with more than 40 characters (thesaurus service).
• Metadata Issue: Fixed an issue where the Password field is erroneously updated whenever modifications are made to other fields in FlexMAM.
• Fixed an issue where it is not possible to clear metadata password fields when Enterprise uses the metadata service.
• Fixed an issue where Enterprise copy and move action is not working for S3 to S3.
• Fixed an issue in FlexMOVE where the workflow is not kicked off and although the asset is uploaded it is not visible in FlexMAM or Flex Core.
• FlexMAM Uploader: Workflow fails on asset upload if timecode field is available in metadata but no value is passed.
• Image asset: Fixed an issue where a system error is displayed when running an extract job on an image asset.
• Fixed an issue in the Authorization service where an exception was thrown when retrieving subaccounts
• Fixed an issue where UdoService and assetservices fails with NoSuchFieldError:USER_DEFINED_OBJECT_TYPE.
• Fixed an issue where hotfolder repeatedly fails and restarts automatically and fails again.
• Fixed an issue where Flex player is not playing audio from mpeg-dash proxy.

Enhancements FlexMAM

• New FlexMOVE interface and integration with Aspera Connect
• Delete Action in the More Actions list is not displayed only if the user has permission to delete.

Fixed Issues FlexMAM

• Fixed an issue where it was not possible to download keyframes in FlexMAM for assets with special characters in the file name.
• Fixed an issue where a user received an error when Completing/Pausing a task for an asset with an image variable in the metadata definition.
• Fixed an issue so that DateTime Fields (Start Date, End Date, Start Time, End Time and Created Date in metadata and marker display template use the timezone setting for the user.
• Fixed an issue where it was not possible to save a sequence which has non-null metadata values.
• Fixed an issue with UDO creation by adding an option to create skip setting the default metadata instance when creating a child UDO with API script.
• Fixed an issue with the Operational Dashboard where the asset names were displayed as Undefined.
• Fixed an issue with Uploader XSRF set cookie request not being recognized.
• Fixed an issue where the workflow did not trigger for the FlexMAM ASPERA Uploader.

List of Services upgraded between 2023.12.1 and 2023.12.2

2023.12.1

Enhancements DaletFlex

• Validated the editing of growing high resolution assets using Flex XTEND in Adobe Premiere. Validated with Op1a AVC-Intra 100 MXF videos.
• In the Flex-Brio integration, optimized the update of duration in growing files by Extract Tech Metadata.
• Disabled ETM update for asset version lower then 4 and has asset having annotations by default. To force ETM update set {{flex-enterprise.featureToggles.allowTechnicalMetadataChanges}} to true.
• Improved annotation retrieval response times.
• As part of the Aspera integration, added the ability to provide a token for the Aspera inbox via API for the front-end application to connect to the aspera server for file uploading.
• Added support for subpaths and file names in the Create Proxy actions for MPEG-DASH and HLS files. NOTE the temporary limitation: do not include file extension in the file name configuration.
• Fixed subtitle timecodes in the media cortex plugin.
• Allowed metadata instances to be saved referencing deleted objects, taxons, tags etc from scripts. New parameter ‘allowDeletedReferences’ for API ‘PUT /api/{assetId}/metadata’ in Enterprise.
• Added support for exporting to AAF for transcode jobs by specifying new property in media profile: `formatVariant = "OPATOM"` with `format = "MXF"`.
• To improve performance for indexing metadata annotations, added caching for taxons, tags and objects in indexelastic.
• Set the event correlation_id to the job Id for Event type TRANSCODED to fully enable usage tracking.
• Set the event correlation_id to the job Id for Event type PROXY_CREATED to fully enable usage tracking.
• Set the event correlation_id to the job Id for Event type COPIED to fully enable usage tracking.
• Set the event correlation_id to the job Id for Event type Event type MOVED to fully enable usage tracking.
• Set the event correlation_id to the job Id for Event type Event type RESTORED to fully enable usage tracking.
• Job Scheduler: Logic for Job loading and enqueue sequence in Job Scheduler logic updated.
• Renamed Hazelcast cache keys used by Enterprise to align with the Enterprise convention.
• For Enterprise introduced a metric, dalet_service_info, capturing the service state (Boolean) and other metadata (version, runtime, runtime_name, runtime_version, runtime_framework).
• For Enterprise introduced a metric, dalet_service_info, capturing the service state (Boolean) and other metadata (version, runtime, runtime_name, runtime_version, runtime_framework).
• Additional improvements to Job Scheduler to exclude jobs for which job quota for action configuration has been exceeded. Excluded jobs main in a Pending state.
• To improve performance when fetching metadata instances, add metadata caching support to metadata, metadata merge, replicate gateway and fastobject services.
• To improve performance when indexing UDOs, now caching ACLs for parent UDOs in indexelastic.
• FFmpeg buffer size is now configured via consul and the env var. Use `flex/flex-streamprocessor-service/ffmpegNetworkReadBufferSizeMegabytes` to set the size of read buffers for network sources (non-image sequence sources) and use `flex/flex-streamprocessor-service/ffmpegNetworkShortSeekSizeMegabytes` to set the size of seek up to which the current network response will be streamed rather than creating a new byte range offset request in FFmpeg and FMPE.
• Added support for a new non-scaled WORKER instance deployment type for use with CONTROLLER instances.
• Exposed taxonomyId for a drill-down from the SDK.
• Ensured script locks are unlocked if a script errors.

Fixed Issues DaletFlex

• Fixed an issue where the metadata instance type ahead UI is broken for fields with multiplicity supported variables.

• Further fixes for FlexFileProcessor restore job for small file processing.

• Node JS Service support: Enforce that the origin regex must be a full-string natch with the domain regex to allow access.

• Fixed an issue where the annotation metadata end-point is not returning metadata definition reference object.

• Fixed an issue for workflows contain many jobs. Performance improved by ensuring the batch sizes are not exceeded in MongoDB pruning and causing timeouts.

• Authorisation service: Include subaccounts when adding master account to cache.

• Fixed an issue with the Login app SAML configuration. SAML configuration (response, assertation) is now overwritten based on the IdP returned response signing.

• Fixed an issue where Workflows with timed-out jobs which were eventually completed did not change their status from Timed out back to Running.

• Fixed an issue so Dozer is initialized once when a JEF script is executed and is then reused.

• Fixed an issue where CORS rules do not allow for hyphens in hostnames in regex.

• Fixed an issue with the integration with NetApp S3 compatible Storage Resource where the storage path could not be resolved.

• Validated template for JAE permission for all customer environments.

• Fixed an issue where metadata in Cast / Director / Writer fields is not loading / is nor displayed in Flex Core for new Season UDO.

• Fixed an issue with Custom Markers where createAnnotations triggers this exception ‘SocketTimeoutException: Read timed out’. Issue fixed by updating flex-master service to use metadata cache provider.

• Fixed an issue in Dalet Flex Cut where video effects do not render.

• Fixed an issue in Dalet Flex Cut where transitions do not render.

• Fixed an issue in Flex Dalet CUT where growing files did not grow in the trimmer due to a manifest caching issue in the Video Proxy Service.

• Fixed an issue where OPAtom transcoding did not work as the input file name contained an .mxf extension before the actual *.aaf extension. Fixed by ignoring files with .mxf in the file names but not as the final extension.

Enhancements FlexMAM

• To improve precision, Date and Time values in the date filter are sent as a readable string in the request instead of in millisecond values.
• In FlexMAM apps introduces a metric, dalet_service_info, capturing the service state (Boolean) and other metadata (service name, version, type (nodes, java…) and IP.

Fixed Issues FlexMAM

• An issue with an invalid tooltip was fixed where when you hover over an audio asset in Dalet FlexMAM, you see ‘Audio’ as the tool tip.
• An issue has been fixed where if you search for a marker of a video with timecode, which is set in the player, you will now see the timecode rather than real time in the marker search results.
• Fixed an issue in Dalet Flex Cut where no confirmation message appeared after quick saving.
• Fixed an issue in Dalet Flex Cut where no warning message appeared when importing unsupported media to the timeline.

List of Services upgraded between 2023.12.0 and 2023.12.1

2023.12.0

Enhancements DaletFlex

• Added optional formatted and localized query params to the Collection Service
• Added a new datacenter configuration param to HotFolder on a satellite node (HttpTransfer, FSP, FFP) and updated Enterprise to use this configuration to communicate with a service instance in each datacenter.
• Added support for growing/live assets to the Extract Technical Metadata plugin
• Added growing file support to the Conform Action. The action will create a child Asset as soon as media is detected and updating Technical Metadata. The Job will run until the file starts growing.
•  In the Flex Mobile app, as part of the efforts to implement editable metadata, added support for retrieving metadata Drill Down-options based on taxonomies. 
• In the Dalet Flex and Dalet Brio Integration, updated the import workflow to use Extract Tech Metadata from the *.mpd and *.mxf files to better support dynamic/growing files.
• Fallback PixelFormat to to String in FlexMAM mapping instead of UKNOWN
• The FormatVariant variable is now presented in the Flex Enterprise UI as well as the API.
• In the JEF Baton Quality Control integration, added support for S3 Input as IAM 
• Began the integration of the AAF converter too for AAF export to Adobe Premiere Pro
• In the JEF FSP Plugins, added support for Retry operation in cases of growing files or live assets.
• Implemented a new webTransferAutoDiscovery KV, set by default to True.
• Added support for exporting audio sequence only (no video) to Adobe Premiere Pro  
• Added a new KV to support a legacy behavior, legacyMp4FormatAndMimeTypeEnabled, set by default to false. When set to true, formatContext and mimeType will be based on the formatVariant.
• Added support to search the new denormalized collections-with-items index.
• Extract keyframes or bulk extract will no longer fail the request if a keyframe already exists at a target TC. A warning message will be sent in API regarding the conflict.
• As part of the Dalet CUT development, enhanced REST API support for media assets with extensions *.wav, *.aiff, *.mp3, *.wma, *.aac.
• Added CORS configuration to Enterprise Core UI.
• Converted the list of items to a single item in the denormalisedCollectionsAndItems index for consistency with the other denormalized indexes.
• Introduced a new KV for satellite domain name, flex/shared/consul/satelliteDatacenter/masterUrlEquivalent in WebTransfer servicesupport library. The value my include the protocol and optionally, the port.
• for the JEF Baton integration, updated plugin field labels. 
• Increased flexibility of DB pruning by workflow/job state

• Date variables in job context and workflow context variables now use direct mapping.

• Implemented the archive command for Google Cloud Services archives.

• Implemented support for detecting growing files for conform FSP jobs.
• Flex-dmt and FSP now the new DMT build.
• Implemented the Dalet Media Cortex plugin which allows automatic transcription and annotation.
• Implemented support in the FSP extract asset info jobs to detect growing/live sources, not completing the job until the file stops growing. 
• Implemented support in the FPS job framework for growing/live sources. Added growing and outputTimestamp to job result. Added consul KV to revert to legacy behavior: flex/flex-streamprocessor-service/disableGrowingSupport. Also added Consul KV flex/flex-streamprocessor-service/enableLegacyTimestampsAsStrings to revert to legacy behavior of API returning timestamps as string instead of integer values.
• Removed unmanaged storage resource type resource from enterprise. Kept Unmanaged type in ResourceImplType as this is required to see the list of deleted or existing unmanaged storage resources. 
• Removed the deprecated RSS-publish and unpublish plugin configurations from Enterprise.
• Created new mapping for POST/Users API call (overwrite existing mapping).
• Created a new Get/Users list of mappings for a given Flex user ID.
• Increased thesaurus term max length to 100 characters in the Thesaurus Service.
• Implemented support for mapping AAF hires audio stream layouts to proxy layouts for conform jobs.
• Enterprise API error responses now show the "flex.request.id" which can be used to look up relevant Kibana logs.
• Ensured all input/output streams are closed to optimize performance.
• Removed the workflow age from scheduling computation of the Job Scheduler to achieve a steady number of jobs on the job queue when there are many PENDING jobs.
• Allowed editlist in MP4 outputs to support AAC pre-roll packets and prevent extra audio frame duration in outputs
• Re-ordered DASH and HLS manifest files streams or correct order (DASH: video, audio, text; HLS: audio, text, video.
• Introduced caching for taxons, tags and objects in indexelastic to improve performance. 
• In the Spring-Boot and NodeJS services, introduces a metric, dalet_service_info, capturing the service state (Boolean) and other metadata (service name, version, type (nodes, java…) and IP.
• To prevent a case of a “perpetually growing” file state, added a configurable timeout to detect files/folders which have not been modified or changed size in a long time, even if the file is locked or a *.wrec file exists.

Fixed Issues DaletFlex

• Fixed an issue in the Workflow definition limit and the Running workflow counter in Redis not being updated correctly. 
• Fixed an issue in the iOS Flex mobile app version where users were unable to view the metadata and details tab. 
• Fixed an issue where users the output variable name was missing after an Amberfin workflow when the field "Succeed job in case of Amberfin job failure" is set to Yes. Fixed by setting the job state to RUNNING in the LaunchAmberfinJobExecutionStep step so workflow variables are added in the next step.
• Disabled timeout for the WaitForNamedSignal action, as it was timing out while waiting for signal.
• Fixed an issue where the Edit import failed when the file name contained a dot (.) and the Image import failed when the file name contained a space.
• Fixed an issue where when changing the metadata definition of an asset via the Core UI or the SDK did not update the metadata mapping collection, and the metadata was unsearchable.
• Fixed an issue where the Timed and Wait Actions did not persist in stepData (jobContext/WorkflowContext).
• Added an optional limit to variables values calls (limit=x) to avoid loading unnecessary amounts of variables.
• Updated the docker-compose configuration to allow storing of more flex-master logs in the restapi-automation phase.
• Fixed an issue where Copy and Move JEF actions failed when the target was a sharded Storage Resource.
• Fixed an issue with metadata field permissions where even if users had Write permission, they could not edit the field.
• Fixed an issue where jobs remained in PENDING state even if a workflow had a defined time limit.
• Fixed an issue with an unhandled rejection in VFS layer for local filesystem, where FPS jobs would fail.
• Standardized the checking of existing output locations for all job types.
• Fixed an issue where creation or update of metadata while creating assets using a legacy script (pre-2023.08) failed.
• Fixed an issue where the S3 Input Stream read method did not read all full number of bytes.
• Fixed an issue where a system error would appear when a new event handler was enabled. Fixed by removing an unnecessary cache.
• Fixed an issue where users could not delete an S3 folder when it had sub folders.
• Fixed an issue where while importing a live asset (growing), the AssetImporter failed as the *.mpd file had no duration.
• Fixed an issue where the Sync Workflow Limit task did not remove the limits from the Redis when the limit was removed from the Workflow Definition.
• Fixed and issue with the Launch and Wait workflow action failing due to a bean auto-wiring issue.
• Fixed an issue where the PDF preview button did not appear for attachments generated from the Baton Quality Control (JEF) action.
• NTSC and PAL will now be specified in DDM format string for additional frame rates.
• Removed the trailing slash from the panel URL in the Flex Xtend Panel App.
• Fixed and issue with the Collection service where deleting a collection that had a parent collection with specified write ACL access resulted in error.

• Node JS Service support: Enforce that the origin regex must be a full-string natch with the domain regex to allow access.

• Fixed an issue where Event Handler service does not paginate through the Enterprise /api/users/<userId>/accessibleWorkspaces API, so if the user is a member of >100 workspaces, then permission may denied incorrectly. Issue fixed by increasing the default limit of available workspaces to 1000.

• Fixed an issue with the WorkflowScheduler where moving workflows from pending to running resulted in error. Updated job scheduler NOT to check workflow quota but move eligible jobs to RUNNING if the respective workflow is in running state. Updated workflow scheduler to set the workflow back to PENDING state if there is an error to help avoid workflows with an inconsistent state, as a running workflow with no job.
• Fixed an issue with JEF Scripts failing.
• Fixed and issue where a Timed Action execution based on an ‘Execute Action On Metadata Condition’ plugin check resulted in an ‘item does not exist’ error.
• Fixed an issue where a wrong path or name could have led to the removal of all  ImageSequence Assets in a folder during a Purge action. Fixed by adding a condition to check file extensions. 
• Fixed an issue where a wrong path or name could have led to the removal of all Segmented Assets in a folder during a Purge action. Fixed by adding a condition to check file extensions and by checking uf the file exists in the vfs-location.
• Fixed and internal server error when /assets/{asset}/raw api.
• Added missing fields in the format and video stream information for R3D file: Format: Bitrate; Video Stream: Codec, Width, Height, Display Aspect Ratio, Pixel Aspect Ratio.
• Fixed an issue where the social media Publish action failed for an asset with a segmented proxy.
• Fixed an issue where the Twitter Publish action failed due to an S3 reading issue.
• Fixed an issue with the Flex master due to a NoObjectFoundException error AT failure with reason NoObjectFoundException.
• Consul KV workflowMaxConcurrentPerAccount is no longer used as the default Account Limit.  workflows are not added to the account level limit if the account level limit is not configured in account metadata.
• Fixed an issue in Dalet CUT where after rendering a project, the audio was one frame longer.
• Fixed an issue where content published to Facebook using the Facebook Publish action did not publish properly, resulting in unplayable media on Facebook. 
• Fixed an issue where an attempt to view a spanned media asset caused an error. 

Enhancements FlexMAM

• Added support for images in Dalet CUT: when a JPEG image size is less than 5 MB when placed on the Dalet CUT timeline, no proxy image is used. If larger than 5 MB, a proxy image will load.
• Added edited sequence saving support in Dalet CUT: when importing items to Dalet CUT, after Saving as a new asset, a save button has been implemented to save edited sequence.
• Added Trimmer support to Dalet CUT: When you hover over an asset in Dalet CUT, you get the option to open the asset in the Trimmer. Once you click this button, the asset is open in a second screen that allows you to trim the asset to send to the timeline.

Fixed Issues FlexMAM

• Fixed an issue in the FlexMAM Player for Asset Summary page and new reviewer app, where when a user clicks on the play button from created clip then clip plays and ends very quickly.
• Fixed an issue in the Summary view, where users have IN and OUT marks, but upon creating a new one, the range selection is not properly updated.
• Fixed an issue where if a user adds a workspace as a filter chip, it does not appear correctly.
• Fixed an issue where the ‘got to URL’ button was missing on a metadata form.
• Fixed an issue in Dalet CUT where if content failed to load, the default handling for configuration service endpoints were used.
• Fixed and issue in Dalet CUT where an activate and pinned Collection did not appear in after page refresh.
• Fixed an issue in Dalet CUT where an error would appear in the status bar when dragging media from the browser clipbin to the collection area.
• Issue has been resolved in Asset Manage page, where if a timeline comment was made, then when selecting the comment, it wrongly highlighted the duration in timeline section below the player
• Fixed an issue in FlexMAM where the Subtitle icon button was not visible in the summary panel player
• Fixed two issues, the first, where if there were multiple audio tracks, then the scroll bar to see more audio tracks was not present in Reviewer. The second was that in the Summary page, the audio track name was not displayed correctly.
• Fixed an issue where after an upload of a profile picture profile, the User profile icon was not displaying the uploaded picture.
• Fixed an issue where in the Flex Reviewer, once an image has been uploaded to an image metadata field, it was not displaying.
• Fixed an issue where in the Flex Reviewer, the Metadata Drop down values were hidden from view on the finish review screen.
• Fixed an issue in Dalet CUT where the “Created” date field displayed the "DD/MM/YYYY" format in the clipbin and collection folders.
• Fixed an issue in Dalet CUT where the status bar showed an error after a page refresh.
• Fixed an issue where when adding custom field filters to an asset search, they were not showing up as chips not affecting the search results.
• Fixed an issue in the Flex Reviewer, where audio tracks names were partially hidden in audio switcher icon in summary page.
• Fixed an issue with the Flex Reviewer, where the Audio switcher icon was missing in summary page of the reviewer for assets with an *.mpd proxy.
• Fixed an issue where the User’s profile picture was not being displayed in the Dalet Cut App.
• Fixed an issue in the Flex Reviewer, where if you sort by Expirty and Creted date, no assets were being found.
• Fixed an issue in the Flex Reviewer where one cannot filter comments by user.
• Fixed an issue with the Flex Reviewer where Taxanomy taxon options were not displaying.
• An issue has been resolved where on the Action execution page - Launch action configuration the potential job list was missing.