Wait Plugin

Introduction

The Wait Plugin Type is used to add Wait steps to workflows. Different types of Wait actions are available as Plugins. A wait action can add a predefined delay in the workflow, wait to receive a signal from another process or wait for a script to evaluate. Whenever one of the Wait actions is used as a node in a workflow, the workflow waits until the Wait job completes before proceeding to the next node in the workflow.

Wait actions are used in two main scenarios:

• Workflow to Workflow Communication 
• Third-Party or External System Integration

You use them when:

• You are waiting for external input or readiness (e.g., a file to arrive, a review to complete).
• You need inter-workflow communication (e.g., child workflow tells parent that it has completed).
• You want to avoid fixed delays and make the workflow reactive to real events.

This article explains what each of the Plugins does and shows some examples of how to use them.

JEF Delay Workflow

The JEF Delay Workflow plugin adds a fixed wait time until the action completes and the workflow moves on to the next action. It can also be used to add a key value pair to the variables in the current workflow. For example, Action A executes and creates some data, Action B needs to read that data but it is not immediately available. Add a JEF Delay Workflow action to create a delay between the execution of Actions A and B.

Configure the JEF Delay Workflow action as follows:

Number of Seconds (Mandatory)

Set the number of seconds the workflow should wait until executing the next node.

Context Variables Expand the section to add a key value pair to add to the variables in the workflow (supports multiple values)

Name Name of the context variable to add to the workflow

Value Value of the context variable to add to the workflow

Metadata Wait

The Metadata Wait plugin checks the metadata of an asset. If the metadata value matches the value configured in the action, the action completes. If it does not match it waits and periodically rechecks the value.

Metadata Wait plugin is not a JEF action, therefore, it is executed by Flex Job and supports multiple workflow transitions to the next node.

The action completes for one of three conditions:

• When the metadata variable matches the configured value → the job exits by the transition named true
• When the metadata variable does not match the configured value and it reaches the configured timeout → the job exits by the transition named timeout
• When there is no asset or there is no value in the asset metadata field → the job exits by the transition named false

You MUST add multiple transitions to a Metadata Wait node in a workflow, otherwise the workflow will fail if the named transition does not exist.

Configure the Metadata Wait action as follows:

Variable Name (Mandatory) The name of the variable whose value must be matched. You can only set a SINGLE variable.

Values to be Matched Against (Mandatory) The value of the variable that must be matched.

Timeout The maximum time the job runs before completing and moving to the next node regardless of the value of the metadata variable.

Polling Time (Mandatory) Period Polling Time Period (seconds) that the action checks the values of the given variable for a match.

Wait for Signal

The Wait for Signal plugin pauses the workflow until the node receives a signal from another Flex process/workflow or from a third-party app to indicate that it can continue. The Signal can be sent using an action that uses a Message plugin type or via API or using a script. See an example of a Signal script below here.

The signal that is sent is hard-coded and is “-SIGNAL”. Regardless of its source (Signal Job Command Plugin, API or SDK via scripts) the signal adds a workflow variable that is displayed in the Variables tab for the workflow. The key will follow the pattern NODE-{name of the workflow node that received the signal}-SIGNAL and the value is the time at which the signal is received. The time is represented as a String using the ISO 8601 format, without millisecond and with timezone: YYYY-MM-DDThh:mm:ssZ. The timezone is always GMT+0.

For example, you use a Wait for Signal action to signal your workflow rather than defining a delay of 12 hours. This makes the workflow reactive to an event rather than relying on a time delay.

Wait for Signal plugin is not a JEF action, therefore, it is executed by Flex Job.  

Example of how to use a Wait for Signal action in a workflow

To use a Wait for Signal action to stall a workflow until you receive a signal to continue, you configure a:

• Signal Job action
• Wait for Signal action
• Workflow containing the Wait for Signal action

Create a Signal job to send a signal to a Wait for Signal node in a workflow, using Message Plugin Type  >  Signal Job Command Plugin.

1.You can optionally specify a specific workflow definition.

2.In the configuration of the Signal job, specify the destination of the Signal being sent. It is mandatory to set the exact name of the workflow node you are signalling. 

3.You can also set or modify variables in the target workflow instance. 
NOTE: The variables you send are only updated if the target workflow is Running.

4.Create a Wait for Signal action to use as node in a workflow to receive a signal, using Wait Plugin Type  >  Wait for Signal Plugin. 

5.The Wait for Signal action, does not require any additional configuration. 

6.Create a workflow with the Wait for Signal action.

When this action is used as a node in a workflow, the workflow waits for the Signal Job created above to send a Signal to the specific node, in this example “any name here”, specified in the Signal Job to complete the job. If there are multiple instances of running workflows which contain the specified Wait for Signal node in the Signal job, they will all receive the Signal and progress to the next node.

JEF Wait for Named Signal

The JEF Wait for Named Signal plugin pauses the workflow until it receives the Named Signal configured in the action. Once the Named Signal is received by the waiting node, the workflow continues. The signal can be sent by Named Signal Job or by API or using a script.

The JEF Wait for Named Signal action includes an additional mandatory configuration to set whether the job should wait for a signal before executing or not.

For example, Action A sends a network call to a customer’s external system to perform a transcoding operation. Action A is completed after sending the call. The next node in the workflow can then be a JEF Wait for Named Signal action that waits for the external system to send the configured Named Signal once the transcode is completed. Once the Named Signal arrives, the job is completed and the workflow continues. 

Example of how to use a JEF Wait for Named Signal action in a workflow

To use a JEF Wait for Named Signal action to stall a workflow until you receive a signal to continue, you configure a:

• Signal Job action
• JEF Wait for Named Signal action
• Workflow containing the JEF Wait for Named Signal action

Create a Signal job to send a signal to a JEF Wait for Named Signal node in a workflow, using Message Plugin Type >  JEF Signal Plugin.

In the configuration of the Signal job, specify the name of the Signal being sent. This is the name that is used in the JEF Wait for Named Signal action configuration.

Configure the JEF Wait for Named Signal action as follows:

Signal Name (Mandatory) The name of the Signal that the job waits to receive – this must be the same as the name in the JEF Signal job action.

Should job wait for signal before executing? (Mandatory) Select one of the options (Expression/Source Control Script/Groovy Script) to add an additional condition to evaluate whether the job waits for a signal before executing. 

If the Groovy Script evaluates to true, the “job” completes and the workflow continues to the next node even if the signal has not been received.

If the Groovy Script evaluates to false, or any other value other than true, for example a string etc, the JEF Wait for Named Signal job waits for the signal to be received and the job does not complete.

Wait for Signal Expression An expression that evaluates to true or false. For example, if you set the value of the Expression to false, the action will always wait until the Named Signal is received. If your use case is to always wait for the Named Signal before executing the node, set the Expression to false.

Source Control Script Expand to add the configurations to use a Source Control Script that evaluates to true or false. 

Groovy Script Add a Groovy Script that evaluates to true or false.  In the example in the screen shot, the script only evaluates to true if the counter stored in the job variable “Shivesh” reaches 20, this means that the Wait for Named Signal job will wait for the Named Signal to arrive unless the counter stored in the job variable “Shivesh” reaches 20, in which case the script evaluates to true and the job completes even if the Named Signal has not yet been received.

Requires Lock Set the type of lock used on the object while the job is running: Shared, Exclusive/None. See this article for information on Lock Types.

Create a workflow with the JEF Wait for Named Signal action.

When the JEF Wait for Named Signal action is used as a node in a workflow, the workflow waits for the JEF Signal job created above to send the Named Signal and any node that is waiting for this Named Signal, in this example "Wait 2", then completes. If there are multiple instances of running workflows which contain the specified Named Signal in the Wait for Named Signal node, they will all move to the next node.

Using Signals to Update Workflow Variables

Signals are used to communicate with workflows that are using Wait for Signal/JEF Wait for Signal nodes.  A Signal Workflow can also be used in a script to update workflow variables in a running workflow even if it does not contain a Wait node.

For example, if you have a child workflow and need to set workflow variables in the caller or parent workflow you can use signalWorkflow to do this. In this case, because you want to set workflow variables, rather than send a Signal, the node name is not important and the node does not actually need to exist in the workflow.

In this sample Groovy script, you use workflow service to retrieve the parent workflow ID and send the variables to workflow node "Start 1" for the parent workflow. Since the signal is not important, nodeName can be anything and the workflow variables will still be updated in the specified workflow PROVIDED the workflow is running.

  def execute() {
    def parentWorkflow = flexSdkClient.workflowService.getWorkflow(context.job.workflow.id)
    assert parentWorkflow != null: "Error: workflow with ID $context.job.workflow.id does not exist"

    /**
     * It's important that the map values are of the type String.
     * Other variable object types will result in unpredictable behaviour.
     */
    Map<String, String> variablesToSet = [
            "variable-name1": "some value",
            "variable-name2": "another value"
    ]

    def newSignal = NewSignal.builder()
            .nodeName("Start 1")
            .variables(variablesToSet)
            .build()
    flexSdkClient.workflowService.signalWorkflow(parentWorkflow.id, newSignal)
}

You can track if a Signal has been sent because it is displayed in the String Variables of the workflow. From the above example:

JEF Wait for Script

The JEF Wait for Script action executes the script configured in the action repeatedly based on the polling interval. Once the script evaluates to true, the job is completed and the workflow continues. The job continues evaluating the script as long as it returns a value of false and the job does not complete. If the Timeout duration is reached before the script evaluates to true, the job completes and moves to the next node - IT DOES NOT FAIL.

NOTE: JEF Wait for Script actions is a JEF plugin; therefore, it is executed by the Job Async Executor service. They DO NOT support multiple workflow transitions to the next node.

The Update Workflow Variables option allows updated variable values to be passed to the script each time the script iterates. The modification of the workflow variables made by the JEF Wait for Script itself are persisted in the History tab of the workflow and can be used in subsequent iterations of the script. This option refreshes and updates the workflow variables from outside changes in between executions of the JEF Wait for Script, for instance, if the workflow graph has a fork. And in the other leg, a script might have updated a workflow variable, or a signal might have been received; this signal might contain workflow variable updates as well. If the option is not enabled, the script of the JEF Wait for Script will reuse the workflow’s variables in the state they were when the job was created.

Once the script evaluates to true, the final variables are updated in the Variables tab and the job completes. This is useful when using a JEF Wait for Script action to process a long list of objects, such as to export details of large numbers of assets to a CSV.

Configure the Wait for Script action as follows:

Add a script using one of the available methods: Source Control Script, Internal Script or Import Script

In the example in the screen shot, the script checks the status of a job submitted to an FSP service, as long as the job is still active, the script evaluates to false and the job does not complete and the workflow continues to wait. Once the job completes, the script evaluates to true and the job is completed and the workflow continues to the next node.

Timeout Maximum time allowed for the job to run (e.g., 60s, 10m). Before running the script again in each polling iteration, if this time has passed since the script’s first execution, the job completes as "COMPLETED" with result "false" and moves to the next node.

Polling Time Period The time (seconds/minutes) between successive iterations of the script.

Update Workflow Variables Select Yes to use the latest workflow context variables when executing successive iterations of the script until the job completes.

Requires Lock Set the type of lock used on the object while the job is running: Shared, Exclusive/None. See this article for information on Lock Types.

Example of Wait Actions in Workflows

Wait actions are very useful for controlling processes that call sub workflows or external resources.

Example 1a – Wait for Script

In this example, the "Send FSP Req" node, is a Wait for Script action that sends a request to FSP to create a job and then waits until it receives a notification that the job has been created successfully (pass) or not (failed). Once the script evaluates to true, the workflow proceeds to the next node and uses the pass/fail status to determine where to go next.  The Check FSP Job node is a Wait for Script action that waits to receive a signal from FSP that the render job has completed successfully or not. Once the script evaluates to true, the workflow proceeds to the next node and uses the complete/failed status to determine which End node to proceed to.

Example 1b – Wait for Script

In this example, the node "Wait for extract technical info on live ingest" node, is a JEF Wait for Script action that monitors a Live Asset ingest.

This workflow is launched from a Live Ingest workflow and runs in parallel with the parent Live Ingest workflow. For it to function correctly, the parent Live Ingest workflow must have extracted some technical metadata from the live ingest asset.

The "Wait for extract technical info on live ingest" node waits until the Live Ingest parent workflow has extracted technical metadata from the live asset. The script checks to see that the asset in context has Media Context (which is present once the parent workflow has extracted technical metadata).

Once the wait action completes, the rest of the workflow begins, processing live sporting metadata and adding it to the live asset as Timeline Annotations (i.e. Markers). To add Timeline annotations, technical metadata for an asset is required, so the workflow “waits” until that is available.

NOTE: In the script above, the workflow string variable assetHasMediaContext is only set to true  when the script returns true.

When a JEF Wait for Script node ends and transitions to the next node, it could be for one of two reasons: -

• The Job completed successfully (job action returns true)
• The Job timed out

By setting the string variable assetHasMediaContext to true, the decision action after the wait can determine if the wait action completed successfully (i.e. returned true because the asset has technical metadata) or timed out (asset still has no technical metadata). If the variable does not exist or evaluates to false, the asset still has no technical metadata; the decision action ends the workflow and does not attempt to add timeline annotations.

Example 2 - Wait for Signal

In this example, the workflow is triggered by a user adding information to a wizard. The workflow updates a set of information inside the wizard panel and the user must decide which information to use. To pause the workflow so the user can add additional information a Wait for Signal action, the "waitForWizardSignal node", has been added.

Once the user adds the information to the wizard and clicks OK,

a signal is sent to the waitForWizardSignal node and the workflow continues.

In this case, the Signal is sent by the wizard by adding this parameter that points to the Wait for Signal node in the workflow to the External URL field in the wizard configurationsecondaryCheckSignalNodeName=waitForWizardSignal.

NOTE: JEF Wait for Script actions are useful to add a loop that allows the workflow to wait for important information, however it is vital to manage any loops correctly. The workflow loop between the "Get Stats Perform Fixtures from UUI" node and the "Create Text Placeholders" node is carefully managed – the script in the "Get Stats Perform Fixtures from UUID" node prevents infinite loops (max number of loops is limited to 200).

If this is not managed properly, you can have a scenario where a workflow can run indefinitely resulting in many jobs being triggered. In the example below, a workflow was running for over two years and there are over 10,000 loops. Scenarios like this can significantly impact Flex performance.