apache-hudi

Apache Hudi Upsert/Insert/Deletes at the same time


Can we run write operation type Upsert and Delete at the same time and same table?

Is Apache Hudi meta get corrupted??

Please help here to do the same using other if any solutions.

Thanks in Advance !!


Solution

  • With Hudi, you can upsert and delete records in the same query, without corrupting the Hudi metadata, to achieve this you have two options:

    Update:

    If you want to run them in two separate queries, they are considered as 2 concurrent writes, you can activate OCC (optimistic concurrency control) which allow concurrent writes when there is no overlap (DELETE from partition X, and INSERT in partition Y), but when both queries are writing to the same partitions, they will both fail.