1. User entered id.
  2. I do request to API of other service with these id.
  3. I get for example the total number of records 19,000.

Now I need to go through each record from these 19000, get the value of this record and then determine the maximum value of these 19000. API for a single request gives a maximum of 200 records.

I did this:
1. Cycle for from 1 to 19000/200
2. For each pass of the cycle, I write in the array the maximum value of 200.
3. When for has reached the end, I select the maximum value from the recorded array.

It works, but it works for a long time.

How can this be improved or should it be done differently?

  • one
    1) leave; 2) select the maximum of 201 (200 entries + previous maximum), save in a variable; 3) In the variable - the maximum for the entire sample. Ps. And you can not immediately request the maximum from the service? say, specify the desired sorting descending and take the first record ... - Akina
  • @Akina, 2, 3 is a good idea. After the change, I will write about the result. No, the service does not provide this) - Fitstd

1 answer 1

Will not help. Suppose you have a good api and is responsible on average for 0.05 s (taking into account all the overhead costs of sending, processing and delivering the response to your piece of iron). 19000/200*0,05 - which is just under 5 seconds of real time. Suppose an even faster and more stable api with a response time of 0.01 s — this is still one second of real time.

It doesn't matter what and how you do it on your side, if 99% of the time you are just waiting for the result of an API call.

If the API owner is not offended by this and does not begin to squeeze the limits on the number of requests (hundreds of requests per second from one of your scripts - this is quite a lot), you can take multi curl and execute requests to api in parallel. In a sad case, if the API is not based on HTTP, then write your binding on non-blocking sockets . And again, there is a big chance to fly into the limits of using the API. And even in general, to put a flurry of requests on an external system, if it is not ready for such an intensive request.

More democratic - bring the user progress bar. And fill it as requests are processed. Those. Immediately make it clear in the interface that the task takes time and that the user has not been forgotten, his task is being processed.

And optimally, of course, it would be to convince the external system to add the appropriate opportunity.

  • As an option: 1. In any case, add a progress bar. 2. Save the data to my database (there are still only two short text fields) and already in the “place” to make a selection from the database. - Fitstd