How to read celery results from redis result backend
We have a (django) celery setup with one redis db as broker and another as result backend.
When inspecting the keys in the result backend redis, we see 485K keys of type string and 7 keys of type none. An example of a key is: celery-task-meta-26351389-c077-4ba6-8e34-66731f6636b3. I assume 26351389-c077-4ba6-8e34-66731f6636b3 is the task id. Is it expected to not see more complex types ?
When looking at the results through the flower interface we see:
- 376K processed
- 8K failed
- 368K succeeded
- 86 retried
When going to the failed view at /tasks?state=FAILURE we only see 2 failed tasks there. When clicking on the UUID we are taken to the task view say task/fdb7b30f-d9cf-4b60-a1b4-11d6aee241b3 and we see details, including args, exception and trace.
When going to the success view /tasks?state=SUCCESS we see 10K entries. But for the earliest tasks in the list, the corresponding task view, tells us Unknown task '377cc204-cb5e-41bc-9aa8-194679ac9bce'. For later tasks we get to see some information.
In the end, the result backend redis uses almost 500MB memory but only little information seems to accessible.
Is there a way we could access the details of all these failed tasks ? (Possibly adapting our setup that looks a bit broken).
As far as I know Flower saves the tasks info in memory (RAM) and it's not related to the celery backend.
Celery backend is used to retrieve the results of your async tasks with its own TTL mechanism (simple Redis ttl in your case).
Flower just registered to the broker's queues and peek at all events.
If you want Flower to keep more tasks (aka increase the history size) I think you should increase the max-tasks in flower's configuration. (default=10k). More details in the docs.
- Field error in object 'target' on field '': rejected value []; codes [typeMismatch.target.,typeMismatch.,typeMismatch.java.util.Date,typeMismatch]
- Binary Safety redis
- Getting A component required a bean named 'redisTemplate' that could not be found. for multi different RedisTemplates configuration
- Redis: possible to expire an element in an array or sorted set?
- How to create a redis cloud connection url with an auth password
- As redis keeps data in memory, what can be the maximum size of data stored in Redis
- MISCONF Redis is configured to save RDB snapshots
- Spring boot caching with redis,key have \xac\xed\x00\x05t\x00\x06
- Running Django process in the background
- How to separate redis database for same two app in node.js
- Redis command times increase over time on same load
- StackExchange.Redis sync-ops and conn-sec
- How do I delete everything in Redis?
- Redis Error 8 connecting localhost:6379. nodename nor servname provided, or not known
- If Redis is single threaded, how can it be so fast?
- Why Redis is single threaded (event driven)
- Redis is single-threaded, then how does it do concurrent I/O?
- Unit rabbitmq-server.service could not be found
- Is it possible to set default ttl for all keys in redis?
- Changing a type of param in application.yml
- Install Redis on Alpine based image in docker
- How do I move a redis database from one server to another?
- Does Redis need locking?
- Spring Boot Application in Docker Unable to Connect to Redis Container (Connection Refused)
- redis (ledis) become slow after flushdb
- redis-cli how to AUTH using password and issue a command?
- How to delete all data from Redis using RedisTemplate in java
- How to create an index in Couchbase in order to enumerate though all the keys?
- Is BLPOP supported in Redis Lua functions?
- How to implement auto-refresh with Redis cache