We have a microservices app that are running on PCF. Some of the microservices are able to generate log files in its log folder. Is there a way to automate to download these log files and save it to a shared folder or remote container (like google drive and the like)?
Your suggestion and advice is highly appreciated.
Thank you.
In the perfect world, you would not write things to the local filesystem that you need to keep. It's OK to write cached filed or artifacts you can simply recreate, but you shouldn't put anything important there.
https://docs.cloudfoundry.org/devguide/deploy-apps/prepare-to-deploy.html#filesystem
The local file system exposed to your app is ephemeral and it's not safe to store important things there even for a short period of time. You could certainly try to set up a process that runs periodically and sends log files out of your container to somewhere else. However, when your app crashes you're going to lose log messages, probably the important ones that say why your app crashed, because your sync process isn't going to have time to run before the container is cleaned up.
What you want to do instead is to configure your applications to write their logs to STDOUT or STDERR.
https://docs.cloudfoundry.org/devguide/deploy-apps/streaming-logs.html#writing
Anything written to STDOUT/STDERR is automatically captured by the platform and sent out the log stream for your app. You can then send your log stream to a variety of durable locations.
https://docs.cloudfoundry.org/devguide/services/log-management.html
Most applications can easily be configured to write to STDOUT/STDERR. You've tagged spring-boot on this post, so I assume your apps are running Spring Boot. By default, Spring Boot should log to STDOUT/STDERR so there shouldn't be anything you need to do.
What might be happening though is that your app developers have specifically configured the app to send logs to a file. Look in the src/main/resources/application.properties or application.yml file of your application for the properties logging.file.path or logging.file.name. If present, comment out or remove them. That should make your logs to STDOUT/STDERR.