Dalet Flex Folder Resource

Folder Resource

A Folder Resource plugin is an object that references one or more Storage Resources that are used for managing the storage of files. A folder can reference multiple storage resources and you can use a Storage Resource Selector Policy to configure which of the associated Storage Resource to reference. 

Dalet Flex does not communicate directly with storage resources, instead it connects to them via Folder Resources.  Folders are a logical concept in Dalet Flex that are used for configuration, they do not refer to any physical object or location.

Folder Resources do not support nesting.

Creating a Folder Resource

1. In Flex Core, in the New menu, select Resource.

2. In the Create new Resource screen, give the folder resource a Name and an optional Description. Set which accounts and groups have access to the folder by setting its Visibility and select Folder Resource Plugin.

NOTE: Folder Resources are only visible and available in drop down lists for use in actions/workflows on Accounts they have Visibility for. 

3. Click Save and move to the Configuration sub-tab and click Edit.

4. Select the storage resource that the folder connects to from the Storage Resource drop down list.

5. Click Save to finish and and then click Enable.

6. Click Start to start the folder.

Restarting a Folder Resource

A Folder Resource must be started to be active. Failed Folder resources can be automatically restarted a configurable number of times before they are failed in a way that requires manual intervention. 

This is configured using the Consul KV flex/enterprise/resource/failureThreshold . If no value is provided, the default number of retries is 5.

Associating a Folder Resource to Multiple Storage Resources

You can associate a single Folder Resource to mulitple Storage Resources. 

1. Click the icon to associate an additional Storage Resource to the Folder Resource. Select the second storage resource from the drop down list.

2. Click Save to finish.

Storage Resource Selector Policy

Dalet Flex allows you to use a Storage Resource Selector Policy to derive the Primary Storage when a folder Resource has more than one Storage Resource associated with it.

There are three available Storage Selector Policies:

• Custom Configurable Selector Policy
• Failover Selector Policy
• VfsProvider Selector Policy

Select the icon in the Storage Resource Selector Policy section to display the policies.

By using the selector policy, all assets (including proxies), account storage referenced items (keyframes, attachments, metadata files, waveforms) will dynamically switch the SR to source from based on the selected SR. This applies to job execution (selecting the file to use), players etc. 

Custom Configurable Selector Policy

The Custom Configurable Selector Policy is used to select an explicit Primary Storage Resource from the multiple Storage Resources configured for a particular Folder Resource.  

No checks regarding the availablity of the selected Storage Resource are performed.

 To configure Custom Configurable Selector Policy:

1. Select this policy in Folder Resource configuration page.
2. Select the Storage Resource to be used as the Primary Storage. NOTE: You must select one of the Storage Resources configured for this Folder Resource and you cannot use an expression to select the custom storage resource.

3. Save your changes

Failover Selector Policy 

The Failover Selector Policy is used to manage storage locations dynamically so that if a Storage Resource is offline, the selected asset can be accessed seamlessly from an alternative Storage Resource.  This is the recommended selector policy.

For redundancy, in case a Storage Resource goes offline, files may be stored in more than one location. When Failover Selector Policy is enabled for a Folder Resource, the policy will set the first online and accessible storage resource associated with the Folder Resource as the Primary Storage Resource and assets in this location will be used for all relevant jobs and actions. A storage resource is considered inaccessible after multiple availablity checks. The Storage Resources are checked in the order they appear in the FR configuration screen. 

In this example, Storage Resource A is the default primary storage resource if it is online. If Storage Resource A becomes inaccessible, Storage Resource B will be used; if both A and B are down, Storage Resource C will become the primary resource. When a process (job, action etc) asks for the location of an asset managed by this folder, it will be updated which Storage Resource is currently the Primary Storage. Once Storage Resource A comes back online, this will once again be registered as the Primary Storage Resource. The file information for an asset is updated dynamically to match the current Primary Storage Resource. 

NOTE: Dalet Flex DOES NOT synch the content of the different Storage Resource. This can be managed by the Storage Providers, such as AWS and Azure or by setting up workflows that copy the files between the storage locations.

To configure Fallover Selector Policy:

1. Select this policy in Folder Resource configuration page; the policy will set the first online and accessible storage resource associated with the Folder Resource as the Primary Storage Resource.

2. Save your changes.

This screen shot shows the file information for an asset that is associated to a Folder Resource, FO FR, associated to multiple storage resources (FO SR, FO SR 2, FO SR 3) with Failover Folder Selector Policy configured for it.

You retrieve these details using an API call.

You can use API to retrieve information about the Folder Resource to know which of the associated Storage Resources is the current Primary Resource. 

VfsProvider Selector Policy 

The VfsProvider Selector Policy is used to configure the Primary Storage Resource for a  folder associated to multiple Storage Resources based on storage resource protocol such as File, S3, GCS etc. This is an older implementation of selector policy with basic functionality. 

This policy selects the first Storage Resource in the Folder Resource configuration that matches the configured protocol. If no Storage Resource with the correct protocol exists, the first Storage Resource in the folder is used as the Primary Storage.

No checks regarding the availablity of the Storage Resources are performed.

To configure VfsProvider Selector Policy:

1. Select this policy in Folder Resource configuration page.
2. Select the VfsProvider Selector Protocol

3. Save your changes.

Cache Eviction for Selector Policies

Whenever a storage selector policy configuration changes from one primary storage to another primary storage resource you must manually evict the cache. The cache contains details about the Folder Resource and its associated Primary Storage location.

To evict the cache, edit the  Folder Resource as follows:

1. Set the Storage Resource Selector Policy field to empty and save the Folder Resource.
2. Reselect the Storage Resource Selector Policy and save the Folder Resource.