Using simple_html_dom I make 15 http requests in a loop.
The order of requests takes a lot of time.
Is there an easy way to do these requests in parallel?

  • unless run 15 scripts in parallel. - teran
  • will not work, url's are formed in the process, I basically need something to launch each request with separate processes ... - Amirhon
  • 2
    Maybe multicurl? - Edward
  • Unfortunately I could not find and understand a normal and understandable example for myself, if anyone can help by example, I will be grateful - Amirhon

1 answer 1

  1. The most common method is to use the RabbitMQ messaging service. In this case, you will need to immediately give any answer. But at the same time, the real answer will be only after the queue is completed. That is, the basic idea is not to entrust complex operations to php, but to a third-party service that will execute scripts asynchronously, in a queue and at the same time β€œrelease” php for other operations. This is the most ideal option, but also quite complicated.

  2. As @Eduard wrote, use curl_multi_init .

Allows asynchronous processing of multiple cURL descriptors.

The scheme is simple: we initialize multicurls and we initialize resources, we execute requests and we receive data, we close all descriptors

// 1 ΠΈΠ½ΠΈΡ†ΠΈΠ°Π»ΠΈΠ·ΠΈΡ€ΡƒΠ΅ΠΌ ΠΌΡƒΠ»ΡŒΡ‚ΠΈΠΊΡƒΡ€Π» ΠΈ ΠΈΠ½ΠΈΡ†ΠΈΠ°Π»ΠΈΠ·ΠΈΡ€ΡƒΠ΅ΠΌ рСсурсы $mh = curl_multi_init(); // ΠΈΠ½ΠΈΡ†ΠΈΠ°Π»ΠΈΠ·ΠΈΡ€ΡƒΠ΅ΠΌ ΠΌΡƒΠ»ΡŒΡ‚ΠΈΠΊΡƒΡ€Π» foreach ($urls as $url) { $chs[] = ( $ch = curl_init() ); // создаСм массив рСсурсов ΠΊΡƒΡ€Π» curl_setopt($ch, CURLOPT_URL, $url); // УстанавливаСт ΡƒΡ€Π» curl_setopt($ch, CURLOPT_HEADER, 0); // Π²Ρ‹ΠΊΠ»ΡŽΡ‡Π°Π΅ΠΌ Π·Π°Π³ΠΎΠ»ΠΎΠ²ΠΊΠΈ Ссли Π½ΡƒΠΆΠ½ΠΎ ... // устанавливаСм Π΄Ρ€ΡƒΠ³ΠΈΠ΅ ΠΎΠΏΡ†ΠΈΠΈ ΠΏΠΎ-нСобходимости (Π½Π°ΠΏΡ€ΠΈΠΌΠ΅Ρ€ Ρ‚Π°ΠΉΠΌΠ°ΡƒΡ‚ запроса, Π·Π°Π³ΠΎΠ»ΠΎΠ²ΠΊΠΈ, ΠΏΡ€ΠΎΠ²Π΅Ρ€ΠΊΠ° Π²Π΅Ρ€ΠΈΡ„ΠΈΠΊΠ°Ρ†ΠΈΠΈ ...) curl_multi_add_handle($mh, $ch); } //2. выполяСм запросы ΠΈ ΠΏΠΎΠ»ΡƒΡ‡Π°Π΅ΠΌ Π΄Π°Π½Π½Ρ‹Π΅ $status = null; do { curl_multi_exec($mh, $status); // выполняСм запросы Π² Ρ†ΠΈΠΊΠ»Π΅ $info = curl_multi_info_read($mh); // ΠΏΠΎΠ»ΡƒΡ‡Π°Π΅ΠΌ ΠΈΠ½Ρ„ΠΎΡ€ΠΌΠ°Ρ†ΠΈΡŽ ΠΎ Ρ‚Π΅ΠΊΡƒΡ‰ΠΈΡ… опСрациях $ch = $info['handle']; if (is_array($info)) { $content = curl_multi_getcontent($ch); // ΠΏΠΎΠ»ΡƒΡ‡Π°Π΅ΠΌ содСрТимоС } } while ($status == CURLM_OK); // 3 Π·Π°ΠΊΡ€Ρ‹Π²Π°Π΅ΠΌ всС дСскрипторы foreach ($chs as $ch) { curl_multi_remove_handle($mh, $ch); curl_close($ch); } curl_multi_close($mh); // Π·Π°ΠΊΡ€Ρ‹Π²Π°Π΅ΠΌ ΠΌΡƒΠ»ΡŒΡ‚ΠΈΠΊΡƒΡ€Π» 
  • @ Amirhon did not understand the question - Kostiantyn Okhotnyk
  • one
    It turns out that the text was not fully inserted, the question was how to get out of <div class = "a"> <h1> info </ h1> <div> - info, but I found the answer, thanks - Amirhon