With the aid of this question here, I've been busy taking over an woefully undocumented gearman setup from a previous dev. Funnily enough, I've never seen such a woefully poor excuse for documentation as the gearman official website either, so have spent a while writing my own for the repo.
Anyway, I'm doing ok regardless as I'm persistent, so all good, except my staging site, which is unusable as it will not load memcache, due to stuck jobs.
So on to monitor, I run:
gearadmin --status | sort -n | column -t
Production workers show
0 0 26
all done, great!
On staging, I was left with
2550 0 0
...basically every product failing.
So I ran
/usr/bin/gearman -t 1000 -n -w -f what_i_need_to_clear > /dev/null
This seemingly clears a job queue. lol. Almost. Now I have
5 4 4
How do I clear those? as the jobs just back up behind it. I just want to reset my staging to
0 0 0
I'm using a federated table setup with PHP/MySQL/Sphinx/Memcache
Can anyone shed some light on this, or show me how to debug those jobs?
This is an old question I know but in case anyone else has the same question in the future... the answer to why your final jobs didn't clear was that had workers running which were working on those jobs.
The three columns in the status output:
5 4 4
That means 5 jobs in the queue, 4 jobs in progress, 4 workers available.
So while your gearman -w > /dev/null was consuming jobs, 4 jobs were locked by the other 4 workers. The fact you're left with 5 I guess was probably a race condition, one job got returned to the queue by another worker while your drain was running.
You can avoid this by stopping the other workers while you drain the queue, or like Kris said in the comments, stop gearmand, clear the persistent storage and restart gearmand.