I have recently created an R package that makes use of sparklyr possibilities. I invoke the package main function from opencpu and pass as argument an url with all my data as a stream. Data stream is successfully analysed in a distributed way via spark and provides some results.
My only problem is that it needs a lot of time to complete the execution part. I tried to invoke my package both via opencpu.call and opencpu.rpc but both of them make me wait until the end of the process.
Since opencpu is an amazing approach of microservice architecture it would be extremely useful to have the possibility of really asynchronous calls.
Is something of the following supported or planned to be supported an the near future?
Option A: receive instantly a sessionid (even though the process is still executed). Then the client is responsible to ask for the status of the process using his sessionid.
Option B: define a callback url that ocpu server triggers, passing the sessionid upon the completion of the analytic process execution.
Thank you very much for you help!
No, current OpenCPU does not support background jobs. You'll have to create a middle layer yourself that performs the request that does the waiting on behalf of the user.