While moving data to Sink, I'm getting this error in GCP Data fusion pipeline. Can someone help?
GCS path cdap-job/dd5d2bba-9cce-11ed-8666-56bac137a1c0 was not cleaned up for bucket gs://df-5999901975431383890-d2icduix5qi63hnabcpaqiyllq due to The specified bucket does not exist..
I tried recreating the temp buckets as it appeared in the log but it keeps on changing. I had deleted few temp buckets from the list and I suspect that caused this issue.
As of today, with every Data Fusion instance created there is always a bucket that is created with it that has a name similar to the one that you shared in your message, i.e - df-5999901975431383890-d2icduix5qi63hnabcpaqiyllq. This bucket is visible in GCS, on the project you created you instance on.
You are probably getting this error because you deleted this bucket from GCS. You can either recreate the bucket or your Data Fusion instance.