I have requests for information gathering at the service. One request from the client in a loop makes 1000 calls to the database. There is no possibility to merge all small requests into one big one.

All this happens like this:

rows = [] for receiver in receivers: rows.append(db_connection.connection('mybd').select(query, [begin, end, receiver]))

Is there any easy way to do a few DB connections and parallelize queries?

  • A simple way is to start several threads, organize a queue. To queue to push requests, the free flow takes the request from the queue, executes it. Plain Producer-Consumer. But this is a bad, bad solution - it would be good to write non-sting SQL. - m9_psy
  • Look in the direction of asynchronous queries in the database: For example asyncpg - Wolkodav
  • in such a substitution, any approach used for concurrent programming is appropriate. The best result will, of course, be given by methods that use the features of your database queries. If you do not take into account, the methods are exactly the same as for example, for creating a set of http requests (thread / connection pool, async. IO). - jfs

0