oracle-databaseamazon-s3data-synchronizationoracle-cloud-infrastructure

Look for best approach replicating oracle table in S3 bucket


My problem:

I need a data pipeline created from my organization’s Oracle DB (Oracle Cloud Infrastructure) to an AWS S3 bucket. Ideally, I would love for there to be some mechanism for oracle to push new data that has entered the database to be pushed to an S3 bucket as it is added (in whatever format).

Question:

Is this possible with Oracle native, specifically Oracle Cloud Infrastructure?

Or would is there a better solution you have seen?

Note: I have seen AWS has the Data Sync product, this seems like it could facilitate with this problem, however I am not sure if it is suitable for this specific problem.


Solution

  • An S3 bucket is object storage; it can only hold complete files. You cannot open and update an existing file like you would in a normal file system, even just to add new rows. You will need to construct your whole file outside of Oracle and then push it to S3 with some other mechanism.

    You may want to consider the following steps:

    THEN:

    OR:

    OR: