Greetings.
Who can explain the essence of this situation and how to fix it?

Here I create a simple class with 2 std :: string fields and fill the vector with objects of this class. I look at the loaded memory, then "I try" to clear it, but nothing comes out.

#include <iostream> #include <vector> #include <string> #include <stdlib.h> using namespace std; class Foo { public: string s1; string s2; Foo( string s1, string s2 ) { this->s1 = s1; this->s2 = s2; }; ~Foo() { }; }; void printState( string msg ) { cout << msg << endl; system("ps aux | grep test"); cin.ignore(); } int main( void ) { printState( "Program has been started" ); int const N = 100000; int i; vector < Foo * > vec; for ( i=0; i < N; i++ ) { string s1 = "The some string. Bla bla bla.."; string s2 = "123456789012345678901234567890"; Foo * tmp = new Foo( s1, s2 ); vec.push_back( tmp ); } printState( "Load has been completed" ); for ( i=0; i < N; i++ ) { delete vec[i]; } printState( "Unload has been completed" ); return 0; }

Result

 Program has been started root 24601 0.0 0.0 11720 1060 pts/8 S+ 16:07 0:00 ./test root 24602 0.0 0.0 3956 568 pts/8 S+ 16:07 0:00 sh -c ps aux | grep test root 24604 0.0 0.0 6028 680 pts/8 S+ 16:07 0:00 grep test Load has been completed root 24601 2.5 0.2 28328 17488 pts/8 S+ 16:07 0:00 ./test root 24605 0.0 0.0 3956 568 pts/8 S+ 16:07 0:00 sh -c ps aux | grep test root 24607 0.0 0.0 6028 680 pts/8 S+ 16:07 0:00 grep test Unload has been completed root 24601 2.6 0.2 28328 17488 pts/8 S+ 16:07 0:00 ./test root 24608 0.0 0.0 3956 568 pts/8 S+ 16:07 0:00 sh -c ps aux | grep test root 24610 0.0 0.0 6028 680 pts/8 S+ 16:07 0:00 grep test
  • one
    use grep test | grep -v grep grep test | grep -v grep :) - VladD

4 answers 4

Specially drove your code under valgrind'om. Even increased N 10 times (for reliability). All 142777208 bytes of memory were returned to the system. In this example, there are no leaks.

But there are many other possibilities, how you can start all your memory, without leakage. Some people do not know that if all elements are removed from a vector, then it does not give all the memory back (the destructor will of course return everything, but until it is called — part of the memory is held — you can read it here ).

Another way is memory fragmentation. If you constantly create small objects of different sizes, then after some time the memory will be "full of holes." That is, the part in fact, 90% of the memory is free, but in reality - 16 bytes are occupied, 152 is free, and so is the whole memory. And if you want to allocate 200 bytes of memory - nothing happens - there is no memory (the figures are taken from the head without reference to a specific architecture and compiler).

  • The fact is that I follow the memory through htop (top, ps) and it obviously grew there with the same speed, as in this example and there (real application). Maybe it depends on the compiler, I compiled with g ++ - Anton Lakotko
  • I also compiled with g ++. And to trust htop (and other similar) it is necessary to interpret accurately and correctly. Run under valgrind and see. - KoVadim
  • By the way. if I make a call to the vector destructor, vec. ~ vector (); then in the same htop the memory index decreases. those. there is some truth. Ok I will try valgrind - Anton Lakotko
  • such a vec.~vector(); may cause serious problems. - KoVadim
  • Can you explain? - Anton Lakotko

In fact, delete returns the memory to the allocator pool. And when the allocator returns this memory to the system, and whether it returns at all, depends only on the whim of the allocator itself (and on the system on which it runs!).

If you do not like it, you can write your own allocator. This is not so difficult, but you will need to be able to take the memory from the system and return it.

  • one
    As far as I know in * nix memory (allocated in a heap) the system can not be returned. Especially worry about this is not worth it. Unused virtual pages will not interfere with other applications (and yours too). - avp
  • one
    @ Anton: look for a problem somewhere else. The fact is that if the memory is freed and goes to the allocator, the next selection will receive this most previously returned piece of memory, and the memory consumption will not grow anymore. The fact that the memory grows means that some code does not return memory to the allocator , that is, does not cause delete . This is the real problem. - VladD
  • one
    @ Anton Lakotko: I had another completely in mind: you need to dig in the direction of code that does not cause delete! Allokator likely works correctly. - VladD
  • one
    @avp> As far as I know in * nix memory (allocated on the heap), the system cannot be returned. Memory in * nix can be returned. This is a jubilant on where this piece is located. Memory allocation in glibc occurs in two ways: via brk and via mmap. It depends on the size of the requested piece. Small ones are allocated in the general space of "arena", the boundary of which is moved by the call to brk and memory can be freed only if the block to be released is at the end of this area. Large ones are placed and released individually, but their size is a multiple of the size of the memory page. see man mallinfo - sercxjo
  • one
    @avp, I think that will be replaced. In the case of openSUSE, this is where the new gnome interface came out. - gecube

@ Anton Lakotko , maybe this testik will help you with something.

 #include <stdio.h> #include <stdlib.h> #include <malloc.h> int pri_mallinfo (struct mallinfo *mi) { printf ("arena\t\t%d non-mmapped space allocated from system\n" "ordblks\t\t%d number of free chunks\n" "smblks\t\t%d number of fastbin blocks\n" "hblks\t\t%d number of mmapped regions\n" "hblkhd\t\t%d space in mmapped regions\n" "usmblks\t\t%d maximum total allocated space\n" "fsmblks\t\t%d space available in freed fastbin blocks\n" "uordblks\t%d total allocated space\n" "fordblks\t%d total free space\n" "keepcost\t%d top-most, releasable (via malloc_trim) space\n", mi->arena, mi->ordblks, mi->smblks, mi->hblks, mi->hblkhd, mi->usmblks, mi->fsmblks, mi->uordblks, mi->fordblks, mi->keepcost); } main (int ac, char *av[]) { puts ("Hi"); struct mallinfo mi = mallinfo(); pri_mallinfo(&mi); int i, s = 0; char *p[1000]; for (i = 1; i < 1001; i++) { p[i-1] = malloc(i*i); s += i*i; } printf ("\nmalloc %d times total %d bytes\n",i-1,s); mi = mallinfo(); pri_mallinfo(&mi); for (i = 0; i < 1000; i++) free(p[i]); puts ("\nfree all"); mi = mallinfo(); pri_mallinfo(&mi); printf ("\nreturn half non-mmaped via malloc_trim(%d) to system\n", mi.keepcost); malloc_trim(mi.keepcost/2); mi = mallinfo(); pri_mallinfo(&mi); exit (puts("Bye") == EOF); }

We broadcast and run

 avp@avp-ubu1:~/hashcode$ gcc mal.c avp@avp-ubu1:~/hashcode$ ./a.out Hi arena 0 non-mmapped space allocated from system ordblks 1 number of free chunks smblks 0 number of fastbin blocks hblks 0 number of mmapped regions hblkhd 0 space in mmapped regions usmblks 0 maximum total allocated space fsmblks 0 space available in freed fastbin blocks uordblks 0 total allocated space fordblks 0 total free space keepcost 0 top-most, releasable (via malloc_trim) space malloc 1000 times total 333833500 bytes arena 16015360 non-mmapped space allocated from system ordblks 1 number of free chunks smblks 0 number of fastbin blocks hblks 638 number of mmapped regions hblkhd 319283200 space in mmapped regions usmblks 0 maximum total allocated space fsmblks 0 space available in freed fastbin blocks uordblks 15884224 total allocated space fordblks 131136 total free space keepcost 131136 top-most, releasable (via malloc_trim) space free all arena 135168 non-mmapped space allocated from system ordblks 1 number of free chunks smblks 0 number of fastbin blocks hblks 0 number of mmapped regions hblkhd 0 space in mmapped regions usmblks 0 maximum total allocated space fsmblks 0 space available in freed fastbin blocks uordblks 0 total allocated space fordblks 135168 total free space keepcost 135168 top-most, releasable (via malloc_trim) space return half non-mmaped via malloc_trim(135168) to system arena 69632 non-mmapped space allocated from system ordblks 1 number of free chunks smblks 0 number of fastbin blocks hblks 0 number of mmapped regions hblkhd 0 space in mmapped regions usmblks 0 maximum total allocated space fsmblks 0 space available in freed fastbin blocks uordblks 0 total allocated space fordblks 69632 total free space keepcost 69632 top-most, releasable (via malloc_trim) space Bye avp@avp-ubu1:~/hashcode$

I admit, in my comment (that in * nix memory is not returned to the system) was wrong. As you can see part of the dynamically allocated memory is automatically returned after free (), a part can be returned forcibly (via malloc_trim).

  • At the moment I am rewriting my server. because check through valgrind showed a lot of leaks. The main loop has already been rewritten and now it works stably. I can only rewrite business logic classes in parallel testing them for fluidity. If interested I can share the result of the work. Thank you all for the advice. - Anton Lakotko
  • sharing interesting cases is always necessary. - KoVadim

I rewrote my server using valgrind (thanks to @KoVadim ) There are no more leaks and the memory keeps at the 1st level. The essence of the problem was simple, as I wrote, my server is multi-threaded and, under certain circumstances, I waited for the completion of the thread not through the wait (pthread_join), but through the actually declared completion flag. And everything worked perfectly through this flag, except for memory, and so it turns out that pthread_join removes everything that was created by pthread_create Now everything works fine. Thanks again for all the help, and @KoVadim is separate!

  • It was enough for you to run threads in the detached state. Something pthread_attr_setdetachstate (& attr, PTHREAD_CREATE_DETACHED); before pthread_create (& th, & attr, ...) - avp
  • @ Anton Lakotko: then you should have accepted @ KoVadim's answer. - VladD