Flex Scripting: Things to Avoid

When writing internal or external scripts it is possible to write code that will cause unnecessary load or operational issues to Flex. The following is a list of things that should be avoided/paid attention to.

Performance considerations

getXXX,updateXXXanddeleteXXXSDK methods (e.g.assetService.getAsset(1)) involve the database and are therefore relatively “expensive” operations. Calling these methods repeatedly can have performance implications, so the following should be avoided:

def first(assetService, assetId) {
    def asset = assetService.getAsset(assetId)
    // Do some processing 
}

def second(assetService, assetId) {
    def asset = assetService.getAsset(assetId)
    // Do some processing 
}

def execute() {
    def assetService = flexSdkClient.assetService
    first(assetService, 1L)
    second(assetService, 1L)
}

In the above example we are fetching the asset twice, which means 2 REST API calls. The following is better:

def first(asset) {
    // Do some processing
}

def second(asset) {
    // Do some processing
}

def execute() {
    def assetService = flexSdkClient.assetService
    def asset = assetService.getAsset(1L)
    first(asset)
    second(asset)
}

Long running loops

Another performance consideration is loops. If you execute a long running loop (e.g.for,doetc.) this will tie up one of the executor in the Job Execution Framework. Therefore you should aim to exit loops as soon as possible.

// bad
def found
for (def asset in assets) {
    if (asset.id == assetId) {
        found = true
    }
}
// good
def found
for (def asset in assets) {
    if (asset.id == assetId) {
        found = true
        break
    }
}

Fetching large amounts of data

When you fetch data from Flex you may, depending on the search parameters, cause the search execution to occur on ElasticSearch. The database by default will only return a maximum of 10,000 results, which means a script will throw an error whenever it exceeds that.

For example the following script searches for all assets for the textabc. By default Flex will return 100 results, so we have to get the data in batches of 100.

def assetService = flexSdkClient.assetService
def offset = 0L
def limit = 100L
def assetApiQuery = AssetApiQuery.builder()
        .searchText('abc')
        .offset(offset)
        .limit(limit)
        .build()

def assets = assetService.getAssets(assetApiQuery)

def allAssets = new ArrayList(assets.getAssets())
while (!assets.getAssets().isEmpty()) {
    offset += limit
    assetApiQuery.offset = offset
    assets = assetService.getAssets(assetApiQuery)
    allAssets.addAll(assets.getAssets())
}

If you have more than 10,000 matching assets you will eventually get the following error:

Error with job: {"timestamp":"2022-09-27T14:05:46Z","status":400,"error":"Bad Request","message":"Validation failed for 
object='searchRequest'. Error count: 1","errors":[{"codes":["page.outOfBounds.searchRequest.page","page.outOfBounds.page",
"page.outOfBounds.int","page.outOfBounds"],"defaultMessage":"Page 33 with page size300 tries to return results beyond 
ElasticSearch's limit of 10000","objectName":"searchRequest","field":"page","rejectedValue":33,"bindingFailure":false,"code":
"page.outOfBounds"}],"path":"/api/object/ids"}

The solution, assuming that your script must to run against a large amount of data (which should be avoided if possible), is to sub-divide the search in further chunks. An example of this is to load all data based on the created data, starting from a known date (for exmaple, the date that the Flex system went live). e.g.

def getAssetsOlderThan(def date, def searchText) {
    def assetService = flexSdkClient.assetService
    def offset = 0L
    def limit = 100L
    def assetApiQuery = AssetApiQuery.builder()
            .searchText(searchText)
            .offset(offset)
            .createdTo(date)
            .limit(limit)
            .build()

    def assets = assetService.getAssets(assetApiQuery)
    def allAssets = new ArrayList(assets.getAssets())
    while (!assets.getAssets().isEmpty()) {
        offset += limit
        assetApiQuery.offset = offset
        assets = assetService.getAssets(assetApiQuery)
        allAssets.addAll(assets.getAssets())
    }
    return allAssets
}

def execute() {
    def date = new Date().parse("dd/MM/yyyy", '01/01/2021')
    def assets = new ArrayList()
    use(TimeCategory) {
        while (date.before(new Date() + 1.days)) {
            // plus 1 day to ensure we get all assets from the current day 
            assets.addAll(getAssetsOlderThan(date, 'abc'))
            date = date + 100.days
        }
    }
}

Sleeping the Current Thread

Sleeping the current thread viaThread.sleep()should be avoided if possible because it ties up job execution resources.