I have a python project, which scrapes a urls using Selenium and adds the events collected in several Google Calendar (each url is associated to a person and to a calendar) and I would like to publish it and run it periodically on a cloud computing service. The code saves a log file for each user the url was scraped for; the overall computing time may soon reach 1500 seconds.
I'm a complete NEWBIE when it comes to running codes from servers.
I currently have three AWS Lambda functions running periodically that only update calendars, without having to store files, having very short execution times (15-20 seconds).
I tried PythonAnywhere but I'm having inconsistent results. I'm looking for advices for Cloud Computing Services which are relatively easy to use in executing and scheduling python projects as well as storing results in files. Needless to say, I know for what I want to achieve, I'll have to rely on a paid service.
I would also be happy to use AWS but I'm lost in the offer of product they have.
Thank you so much.
UPDATE: After trying Fargate, I've given it a go to Github Actions and it works smoothly. Thanks!
I believe you can still use your lambda function and you should change your script so it will store the log file within an S3 bucket.
following your comment: You can use AWS Fargate in order to achieve that