Prometheus pod crashlooping on error "opening storage failed"

I restarted the Prometheus pod and now the Prometheus pod is crashlooping. Found this error in the logs:

level=error ts=2021-06-09T09:27:29.066Z caller=main.go:758 err="opening storage failed: block dir: \"/prometheus/01F6J0P4KBBWVJD2M8B1PE7C5E\": open /prometheus/01F6J0P4KBBWVJD2M8B1PE7C5E/meta.json: no such file or directory"

Noticed that the 01F6J0P4KBBWVJD2M8B1PE7C5E folder only has the chunks folder in it.

Any idea why this occurs and is there a way to fix this?

So you have already found correct github related issues and asked there also.

• Still seeing "opening storage failed: block dir" on more recent version of prometheus #7090

• does not start up after corrupted meta.json file

And seems currently the only way to fix the problem is to delete 01F6J0P4KBBWVJD2M8B1PE7C5E folder that contains empty meta.json either nothing (link)