Hello everyone and immediately to the point. There is some problem, let it be foo ()
def foo(): capture m, use m, leave m, do smth Next, there are n sets containing m elements. The set is one of the parameters for foo (). When the task starts, it must find a free element in the set given to it and capture it, perform some operations, release and continue to do some work. In theory, the elements of m are full-fledged mutex-objects, practically - these are files. In an hour about 30 thousand tasks of foo are filled in the broker. The implementation through file locking had some problems: All the “workers” engaged in processing tasks use one set, i.e. if there is one element in it, then everyone is idle and waiting for him to be released. At the same time, the broker has tasks for other sets that have free elements. If you return tasks to the broker with the help of raise, then in principle everyone is in business, but since the priority queue, this method does not always work. Once lock was hung on the file, but it did not appear and all the "workers" hung on waiting. It was not possible to reconstruct the error, but it is scary if this happens on the battle server. With KazooClient.Semaphore it was possible at least not to let the workers into the sets, which have all the elements blocked, and inside there are lock files again. Actually, it seems to me that the task is pretty clear.
Currently used technologies: Django1.11 + celery + RabbitMQ + kazoo?
Now, actually, the questions are: 1. Is it possible to turn m objects into mutex objects and pass them into a task in order to try to find a free one inside and, in this case, refuse a lock? It seems that the necessary technologies are present in kazoo, but, unfortunately, the deadline is near and I would like to work, not learn. 2. Sometimes a large pile of tasks for one set with high priority can fall into the queue, and if I understood everything correctly, then all the “workers” will hang up again, because will take the task, do not find a free "m" and fall back. It seems to be solved, if for such tasks to create a separate basket, but even here they can begin to juggle the first ten tasks.
If anyone has an idea, but rather a link to the code that solves my problem. I really do not want to write the task allocator myself. Thanks for attention.
-Ofair,--scheduler. If there are not enough opportunities or it doesn’t fit at all, see which ready-made job schedulers, workload automation systems are closest to your case (from different categories: Slurm, k8s) - jfs