pythonpython-requeststaipy

How to Implement Delayed Retry Pattern on Taipy Core Task


I want to implement a retry pattern on my task API.

Can I do this without doing a try/catch myself? Can the next task wait few seconds before retrying if there is an error?

import requests
from taipy import Config
import taipy as tp


def api():
    r = requests.get('<url>')
    return r.json()


scc = Config.configure_scenario_from_tasks("sc",
                                          [Config.configure_task("api_call",
                                                                 api,
                                                                 [],
                                                                 [Config.configure_data_node("api_result")])
                                          ])

if __name__ == '__main__':
    tp.Core().run()
    scenario = tp.create_scenario(scc)
    tp.submit(scenario)

    print(scenario.api_result.read())

Solution

  • At the moment, there are no built-in retry patterns in Taipy Core.

    However, a scenario can be subscribed to a callback. At each change of a status job (the execution of a task), this callback is called. If the job status is failed, you can execute some code. For example, you can re-submit the scenario:

    from taipy.core import Scenario, Job
    import taipy as tp
    
    MAX_RETRY = 5
    
    def subscribe_to_retry(scenario: Scenario, job: Job):
        print(scenario.name, job.status.name)
        if 'retry' not in scenario.properties.keys():
            scenario.properties['retry'] = 0
        if job.status.name == 'FAILED' and scenario.properties['retry'] < MAX_RETRY:
            scenario.properties['retry'] += 1
            scenario.submit()
    
    # this mechanism will be applied to all the scenarios
    tp.subscribe_scenario(subscribe_to_retry)
    

    You could also directly put a retry pattern in your Taipy task's function.

    def funct(retry=3):
        try:
           ...
            return ...
        except e:
            if retry > 0:
                return funct(retry-1)
            raise e