How to read celery results from redis result backend

We have a (django) celery setup with one redis db as broker and another as result backend.

When inspecting the keys in the result backend redis, we see 485K keys of type string and 7 keys of type none. An example of a key is: celery-task-meta-26351389-c077-4ba6-8e34-66731f6636b3. I assume 26351389-c077-4ba6-8e34-66731f6636b3 is the task id. Is it expected to not see more complex types ?

When looking at the results through the flower interface we see:

• 376K processed
• 8K failed
• 368K succeeded
• 86 retried

When going to the failed view at /tasks?state=FAILURE we only see 2 failed tasks there. When clicking on the UUID we are taken to the task view say task/fdb7b30f-d9cf-4b60-a1b4-11d6aee241b3 and we see details, including args, exception and trace.

When going to the success view /tasks?state=SUCCESS we see 10K entries. But for the earliest tasks in the list, the corresponding task view, tells us Unknown task '377cc204-cb5e-41bc-9aa8-194679ac9bce'. For later tasks we get to see some information.

In the end, the result backend redis uses almost 500MB memory but only little information seems to accessible.

Is there a way we could access the details of all these failed tasks ? (Possibly adapting our setup that looks a bit broken).

As far as I know Flower saves the tasks info in memory (RAM) and it's not related to the celery backend.

Celery backend is used to retrieve the results of your async tasks with its own TTL mechanism (simple Redis ttl in your case).

Flower just registered to the broker's queues and peek at all events.

If you want Flower to keep more tasks (aka increase the history size) I think you should increase the max-tasks in flower's configuration. (default=10k). More details in the docs.