There is a web project, with each request (and there are tens of thousands per day), something like this happens:

скачивается некоторый файл по ссылке в файл -> файл считывается -> происходит некая программная обработка, сохраняется в другой файл -> происходит ещё одна обработка -> я отправляю запрос, где результативный файл прикрепляется в multipart/form-data

Total three files: downloaded (via FileUtils.copyURLToFile ), converted to the desired format (I work with audio files) and processed file (effective, which I send).

Or I receive a request -> I generate it myself and save the file -> process it and load it.

It turns out at least 2 files, maximum 3. The error began to appear:

 Jul 29, 2017 7:38:05 PM org.apache.tomcat.util.net.NioEndpoint$Acceptor run SEVERE: Socket accept failed java.io.IOException: Too many open files at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method) at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422) at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250) at org.apache.tomcat.util.net.NioEndpoint$Acceptor.run(NioEndpoint.java:682) at java.lang.Thread.run(Thread.java:748)

I already asked a similar question ( Too many open files on tomcat server, tomcat 9 ), but there was only one file for each question, and all I had to do was read all the files into a string, but now I don’t know how to do in this situation. Do not save the files is unlikely to succeed, and I do not really understand how to apply try-with-resources here. I would be grateful for the help.

Increasing the limits also failed: ( https://askubuntu.com/questions/941031/sysctl-etc-security-limits-conf55-invalid-syntax-continuing ), I tried to do ulimit -n 8192 , now with ulimit -n and 8192 displayed, but I'm not sure that it works as it should.

  • Do you manage to reproduce the problem with the help of the generated queries? If your code, which physical files opens to replace with stubs, is it still possible to reproduce the problem? How many simultaneous connections do you have? Does your server start child processes? - jfs
  • @jfs 1. As for generating queries, I’m probably not going to imitate it, but I’m almost sure that the result will be the same. 2. What are the stubs? More? 3. How to determine the number of simultaneous connections? As far as I know, with each new request, a new connection is opened (and a stream for it)? In this case, it is possible that the maximum can be up to a couple of hundred connections at the same time. 4. The server does not start any child processes, at least I myself do not. - Peter Samokhin
  • About stubs: just do not open system files when prompted, but use artificial files from memory (so as not to consume fd). If the problem persists, then it’s not about your three files. How to monitor connections? - look at lsof, ss, sysdig. - jfs
  • @jsf besides tomkata, mongodb still works, and a total of more than a dozen connections were not running (I now understand what you asked). As for files from memory, if a new stream is created for each request, doPost / doGet is called each time, How and where can I save files into memory, so that I can access them every time? - Peter Samokhin
  • @jsf I save the audio file in mp3 format, convert it to wav using javazoom jlayer , process it with the help of the sonic library, then load and send it. Either I get the text, generate a voice recording from it using amazon polly , convert again, process it with the same library and load it again. If I could do all this without saving anything in the files, it would be excellent, but I don’t know, to be honest, how to do it, because I’m not jlayer the same jlayer library jlayer that it jlayer not a file, but a stream of bytes . With other libraries as well. - Peter Samokhin

0