No, you are not storing the job itself, just a flag that indicates that the job was not done/or even stored. What's more, there really are various ways to handle failure of storing a job on a queue, but in most cases it's just the absence of the desired effect that can be a giveaway. Ie. you know that the job either didn't finish or wasn't even sent to the queue if it's effect (ie. updating the user's friend list from facebook) was not done.
Also, take a look at #5 for a great monitoring stats solution.
> No, you are not storing the job itself, just a flag that indicates that the job was not done/or even stored
You must be storing enough representation of the job to re-send it though, no? You couldn't do this with a simple flag for all job types I imagine.
> in most cases it's just the absence of the desired effect that can be a giveaway
Sure, but not all jobs create side effects you can easily look for (e.g. sending an email). And secondly, you'd have to create a "sweeper" process for every type of job you create then.
At the point where all the other "normal" mechanisms monitoring storing the job on the queue have failed I would suppose I have a far more greater problem on that system. Personally I would not use flags for jobs types, I would monitor them with flower. Also, we need to keep in mind that it's one thing retrying sending a job to the queue and a whole other thing retrying a job already on the queue that failed for some reason. In my experience so far there a mechanisms that deal with both of these issues effectively and none involved using the webapps database for this.
Thanks, I will take a look at the Pythonic way of doing these things. Celery does look a lot more industrial-grade than most of the Ruby solutions so my curiosity is piqued.
Also, take a look at #5 for a great monitoring stats solution.