Good evening everyone!
Colleagues, please tell me if there is any proven way to solve the problem:

  1. there is a function F that captures the posix th rwlock synchronization primitive , before working with data D
  2. there is another function that also captures rwlock , works with data D, and in the process of working with this data calls function F.

Is there any way to work with rwlock with this order of calls and avoid deadlocks and out of sync (in case you release rwlock before entering function F)? I know that using a recursive mutex would not have these problems, but I just need a read-write lock, and as far as I know it does not support recursive capture, at least in FreeBSD.

UPD. @avp, yes, I want to use the pthread_rwlock_t primitive. blocking it for reading, it will be possible to "walk" through the list, blocking for writing - edit the list. The problem is that some threads need:

  1. find some list items and delete them,
    and in others:
  2. delete function can be called separately

For example, if we only worked with this list item in the stream, and it has exhausted itself, then we don’t need to look for it. In the first variant, the search and delete operation cannot be logically separated, and it would be expedient to simply grab the write lock, delete everything that fell under the conditions, and release, before starting the search. But the delete function is called from other threads, which means it must acquire the same lock. Here you can do what VladD has suggested, if there is still no logic in the delete function, just make a wrapper. But this add. There is logic - there are still a lot of checks done before deleting, which have nothing to do with this list, it would be more reasonable to execute them not under blocking, if possible, of course. Maybe I can take all these checks somewhere, for example in the same wrapper, then in general it should work.

But maybe there are still some ways to synchronize such schemes, which are more convenient to use? So that many threads could run along the list and search for something at the same time, and only one could edit the list, forbidding everyone else to work with it.

  • @margosh, all your code and can I redo it? - avp
  • No, this is a processing of the server, which has so far worked consistently - margosh
  • @margosh, it means there were no locks in the program, and can I write my own? I mean, to solve your problem, you just need to write a couple of functions, say, lock_22(), unlock_22() (I don’t know what to call it) that will safely make a nested lock. By the way, do you really have this nested (balanced) lock-unlock, or maybe several lock (do you really need the first one) and only one unlock? If in principle such a solution is suitable, then I can try to sketch. - avp

2 answers 2

@margosh , for some reason I don’t see any special problems after this description.

The only thing is that when you get wrlock, you need to check if the item has already been removed by another thread? And all functions should work by performing this check.

IMHO in any case can not do without it, because You cannot "raise" the lock level from READ to WRITE. All n rdlock (if there were several of them in the stream), in any case, should be released before wrlock.

And perhaps this solution will do. Of course, it works only in the case when the flow for further work does not matter whether the item is actually removed.

Search functions (streams) make an rdlock to the list. Found items (candidates for deletion) are placed in a certain set (they remain in the list, put a pointer or some item ID in the set). Naturally, a mutex and a semaphore are associated with it, on which the stream hangs, which will delete the elements. That he does wrlock.

Something like that.

UPDATE

@margosh , I don’t know for sure if you would like to see such a code, but I decided to approach the nested wrlock quite formally and I hope this will be useful for you.

The lock22() function can be called already having a lock, the number of unlock22() calls must match the number of nested lock22() calls lock22() . To exit from all levels of nesting and loss of lock, call force_unlock22() .

The address of the struct lock22 structure that contains the pthread_rwlock_t lock is passed to all functions. The structure must be initialized before use by calling init_lock22() .

It is important . All other calls to pthread_rwlock_rdlock() and other similar functions must refer to this particular lock field in struct lock22 .

 // lock22.c avp 2013 nested pthread_rwlock_wrlock, pthread_rwlock_unlock #include <stdio.h> #include <stdlib.h> #include <pthread.h> /* #include "lock22.h" put next lines to lock22.h */ struct lock22 { pthread_t owner; int cnt; pthread_rwlock_t lock; }; extern int init_lock22 (struct lock22 *p, const pthread_rwlockattr_t *a), lock22 (struct lock22 *p), unlock22 (struct lock22 *p), force_unlock22 (struct lock22 *p); /* end of "lock22.h" */ // returns 0 if OK int init_lock22 (struct lock22 *p, const pthread_rwlockattr_t *a) { p->owner = 0; p->cnt = 0; return pthread_rwlock_init(&p->lock, a); } static pthread_mutex_t locki = PTHREAD_MUTEX_INITIALIZER; // returns 0 if OK int lock22 (struct lock22 *p) { int rc = pthread_mutex_lock(&locki); if (rc) return rc; pthread_t self = pthread_self(); if (!pthread_equal(p->owner, self)) { // try to acquire wrlock pthread_mutex_unlock(&locki); #if TEST printf ("CNT %d owner %p self %p\n", p->cnt, (void *)p->owner, (void *)self); #endif if (rc = pthread_rwlock_wrlock(&p->lock)) return rc; // acquired wrlock p->cnt = 1; p->owner = self; } else { // already has wrlock p->cnt++; rc = pthread_mutex_unlock(&locki); } return rc; } // returns 0 if OK int unlock22 (struct lock22 *p) { int rc = pthread_mutex_lock(&locki); if (rc) return rc; pthread_t self = pthread_self(); if (!pthread_equal(p->owner, self)) { pthread_mutex_unlock(&locki); return -1; // logical error. Only owner thread can call unlock22() } if (--p->cnt == 0) { // last nested lock rc = pthread_rwlock_unlock(&p->lock); p->owner = 0; } pthread_mutex_unlock(&locki); return rc; } // returns 0 if OK int force_unlock22 (struct lock22 *p) { int rc = pthread_mutex_lock(&locki); if (rc) return rc; pthread_t self = pthread_self(); if (!pthread_equal(p->owner, self)) { pthread_mutex_unlock(&locki); return -1; // logical error. Only owner thread can call unlock22() } rc = pthread_rwlock_unlock(&p->lock); p->owner = 0; p->cnt = 0; pthread_mutex_unlock(&locki); return rc; } #if TEST /******************************************************* Testing code *******************************************************/ struct lock22 l22; void test (char *msg) { printf ("%s lock22() %d\n", msg, lock22(&l22)); printf ("%s cnt = %d\n", msg, l22.cnt); printf ("%s unlock22() %d\n", msg, unlock22(&l22)); printf ("%s cnt = %d\n", msg, l22.cnt); } void * ptest (void *a) { puts("Thread go"); test("ptest1"); printf ("ptest lock22() %d\n", lock22(&l22)); printf ("cnt = %d\n", l22.cnt); test("ptest2-nested"); test("ptest3-nested"); printf ("ptest unlock22() %d\n", unlock22(&l22)); printf ("cnt = %d\n", l22.cnt); puts("Thread exit"); } int main (int ac, char *av[]) { pthread_t th; void *ptres; printf ("init_lock22(): %d\n", init_lock22(&l22, NULL)); test("main1"); test("main2"); printf ("main lock22() %d\n", lock22(&l22)); printf ("cnt = %d\n", l22.cnt); pthread_create(&th, NULL, ptest, NULL); puts ("Enter for continue"); getchar(); test("main3-nested"); test("main4-nested"); printf ("main unlock22() %d\n", unlock22(&l22)); printf ("main cnt = %d ???\n", l22.cnt); pthread_join(th, NULL); printf ("fin cnt = %d\n", l22.cnt); puts("End"); } #endif

Made in ubuntu, not carefully tested.

Broadcast in ./a.out

 gcc lock22.c -pthread -DTEST

If something is not clear, ask.

UPDATE-2

I init_lock22() that for normal use in init_lock22() you need to add an argument - attributes of the locale. Then man pthread_rwlock_init will clearly correspond.

The code above corrected.

  • I am not yet ready to answer the question of whether there will pass a similar scheme there, the code is not at hand, but I don’t really like the idea of ​​creating another stream to remove items from the list, for the reason that there are enough streams there, difference from the estimated number of cores on which it will be executed. >> I cannot "raise" the blocking level from READ to WRITE I would not want to raise, but rather recursively grab write to a function where the list is passed and deleted. - margosh
  • But only now, having found the next element for deletion, rdlock will have to be given away, and after deletion, to take it again and here is an unpleasant moment. The element from which we were going to continue viewing the list of m. someone already deleted. And if all the details are not carefully thought out, then for coherent lists this is fatal. And if you start from the beginning of the list, it is quite ugly. - avp
  • @ avp, rdlock - only for functions in which deletion does not occur, where I am going to delete an element, I repeat, it would be convenient for me to capture wrlock before starting the search for elements that fit the condition. - margosh
  • @margosh, wrote the lock-unlock code for nested calls in UPDATE of your response. - avp
  • @avp: oh, you wrote a recursive mutex on the basis of non-recursive :) - VladD

Hm Try to divide the functions into parts, one of which takes the lock, and the other performs the work, knowing that the lock is taken.

Freeing a lock is a bad idea: after all, at this moment another thread can take it and change the state of the objects with which you worked before releasing the lock.

That is, you will have:

 F: take the lock call F_realwork release the lock Other: take the lock do some work call F_realwork do some more work release the lock

By the way, think, with good chances, you just do not need the F version with blocking.

  • one
    in my case it will not work - a function that is called internally can be called from other threads independently, and it needs a lock. This could be done like this: duplicate the nested function F, remove the lock in one of them, and call a copy without a lock from a given stream, and a copy with a lock from the others, but this is some ugly solution to the problem. - margosh
  • one
    @margosh: well, let other threads call F , and where necessary, F_realwork . - VladD
  • I do not like this solution to the problem, maybe something is better to eat? It seems to me that this is a quite common problem, for sure in the pros it is solved by some other primitives. UPD: sorry, you meant a wrapper, maybe a wrapper and go, I'll take a closer look. If there are more ideas, I will be glad to read it. - margosh
  • one
    @margosh, are you going to use pthread_rwlock_rdlock / pthread_rwlock_wrlock? I think you know that when pthread_rwlock_rdlock is received you cannot call pthread_rwlock_wrlock. This should be tracked in the program logic. - avp
  • one
    @margosh, so (if I understood correctly) does the nesting problem really exist only for wrlock? - avp