Found the following:

import requests from requests_futures.sessions import FuturesSession from concurrent.futures import ThreadPoolExecutor XML = '<TAG>bla</TAG>' res = [] for i in range(150): session = FuturesSession() res.append(session.post(u'http://192.168.1.100:9000/Service/rest/BlaValidation', headers={}, data=XML)) for j in range(150): res[j].result() res = [] print(threading.active_count())

As a result, despite the fact that the session object, return objects, as well as all objects from futures are destroyed, 151 will be displayed on the screen, i.e. 150 flows generated as a result of the work (and yes, there will be 150 of them, i.e. each new session will create a flow!).
Is it possible to fix this somehow?

If I replace session formation as follows:

 session = FuturesSession(executor=ThreadPoolExecutor(max_workers=12))

Streams will still be 151. For good reason. The only way to limit the number of threads is to put the formation of the session out of the cycle, then even after the "death" of the session object, 12 child threads will remain.

Is it possible to somehow still complete the streams? Because my process architecture still provides for the multiple creation of different sessions, and the number of possible threads per process x32 is not so large, considering the stack and the “strapping”. And when the number of threads exceeds 500-700, the process will no longer work.

    1 answer 1

    In the second variant, a new executor object is created from the question for each session. Maybe try?

     executor=ThreadPoolExecutor(max_workers=12) for i in range(150): session = FuturesSession(executor=executor)

    PS could not resist: http://exeggutor.jpg.to/

    • @actionless, but this does not eliminate the main problem - when creating an executor, a pool is created for the threads, which can then be created, but they cannot be freed. - Arkady
    • so and close the session at the end? - actionless
    • by the way, it will still run synchronously due to: for j in range (150): res [j] .result () see here: docs.python.org/3/library/… - actionless