apache-sparkbigdatadata-processingdata-lakekappa

Does a Kappa Architecture use a data lake or not?

Kukreja in “Data Engineering with Apache Spark, Delta Lake, and Lakehouse” says that a Kappa architecture has no data lake. Microsoft in https://learn.microsoft.com/en-us/azure/architecture/data-guide/big-data (see picture) mentions a “long term store” without saying what it actually is. It uses that data to “re-compute”. For me this is a data lake.

Does a Kappa Architecture use a data lake or not?

enter image description here

Solution

  • No, it does not.

    But others have a more realistic perception. It states recompute... Imagine a mistake has been made, handy to be able to correct. From the long term store.