I wrote an application on JavaFX that parses the pages via webengine, then with the help of jsoup the values ​​of the elements are taken and displayed in a table. The problem is that the more the application works, the more it eats RAM (after a couple of hours of work it can eat under 8 GB). I started it with jprofiler, but all the parameters seem to be normal and the heap does not increase. I do not understand what the problem is

jprofiler

  • Debag mode tried? Look at what's going on in your loops, is the connection closing correctly, are you using a String in loops? - alexandr gaiduchok
  • Is the code itself possible to see? You have a clear leak - GenCloud
  • @GenCloud, drive.google.com/file/d/0B-Bsa-UzKnsUbGU3ZURRbzFPUVE/… - mckay
  • The operating system is not Unix-like random? - Andrey M
  • @AndreyM, no, windows 7 - mckay

2 answers 2

Based on the code:

You have a memory leak in PermGen (jdk 1.7) or Metaspace (jdk 1.8) due to improper use of intern ()

https://habrahabr.ru/post/79913/

https://stackoverflow.com/questions/10578984/what-is-string-interning

The second point: Jsoup itself does an excellent job with getting the page, if you need not get the WebEngine page anymore, you better use only Jsoup

https://jsoup.org/cookbook/input/load-document-from-url

The third point:

doc = null; state = null; interests = null; name = null; str = null; city = null; lastActivity = null; age = 0; iAmt = 0; Runtime.getRuntime().gc(); Runtime rt = Runtime.getRuntime(); try { rt.exec("free -m"); } catch (IOException e) { e.printStackTrace(); }

Remove and this piece of code. The reason is that zeroing here will not help, zeroing primitives all the more, manual invoking gc will not help.

http://www.oracle.com/webfolder/technetwork/tutorials/obe/java/gc01/index.html

Try setting -XX: MaxPermSize = ??? m to check for overflow perm gen

  • That code was already outdated. Now made changes to the document by reference. It gave a small effect after setting the size of the history of the engine to 0. The volume of consumption is growing, but not so intensively, so apparently the main leak is not a story. Removed intern (). I would love to use only jsoup, but as far as I know, it does not work with js on the site. Metaspace in the region of 30mb. Of course, zeroing does not give sense, but if you do not cause gc, then the consumption immediately grows to ~ 1.7 gb, against 400-500 mb - mckay
  • @mckay so what is the profit? Resize history or intern ()? Very interesting .... - alexandr gaiduchok
  • one
    The size of the story. At the very beginning I had no intern (), then I found information that it seemed like intern () could help solve the problem, but there was no point anyway. But one fig, even with a zero history, a couple of hours of work and operative end. - mckay
  • @mckay Issue as an answer, so that those who have a similar question are not searched in the comments. Thanks - alexandr gaiduchok

The problem was partially solved by setting the size of the engine history to 0

engine.getHistory (). setMaxSize (0);

In my cases, after a cold start, 20 minutes of the program, consume instead of ~ 1gb, ~ 830mb. But all the same, the longer the program runs, the more it consumes.

upd: the problem lies in jdk8. When loading a large number of pages, WebEngine has a noticeable memory leak. Jdk9 problem solved