Using simple_html_dom I make 15 http requests in a loop.
The order of requests takes a lot of time.
Is there an easy way to do these requests in parallel?
- unless run 15 scripts in parallel. - teran
- will not work, url's are formed in the process, I basically need something to launch each request with separate processes ... - Amirhon
- 2Maybe multicurl? - Edward
- Unfortunately I could not find and understand a normal and understandable example for myself, if anyone can help by example, I will be grateful - Amirhon
1 answer
The most common method is to use the RabbitMQ messaging service. In this case, you will need to immediately give any answer. But at the same time, the real answer will be only after the queue is completed. That is, the basic idea is not to entrust complex operations to php, but to a third-party service that will execute scripts asynchronously, in a queue and at the same time βreleaseβ php for other operations. This is the most ideal option, but also quite complicated.
As @Eduard wrote, use curl_multi_init .
Allows asynchronous processing of multiple cURL descriptors.
The scheme is simple: we initialize multicurls and we initialize resources, we execute requests and we receive data, we close all descriptors
// 1 ΠΈΠ½ΠΈΡΠΈΠ°Π»ΠΈΠ·ΠΈΡΡΠ΅ΠΌ ΠΌΡΠ»ΡΡΠΈΠΊΡΡΠ» ΠΈ ΠΈΠ½ΠΈΡΠΈΠ°Π»ΠΈΠ·ΠΈΡΡΠ΅ΠΌ ΡΠ΅ΡΡΡΡΡ $mh = curl_multi_init(); // ΠΈΠ½ΠΈΡΠΈΠ°Π»ΠΈΠ·ΠΈΡΡΠ΅ΠΌ ΠΌΡΠ»ΡΡΠΈΠΊΡΡΠ» foreach ($urls as $url) { $chs[] = ( $ch = curl_init() ); // ΡΠΎΠ·Π΄Π°Π΅ΠΌ ΠΌΠ°ΡΡΠΈΠ² ΡΠ΅ΡΡΡΡΠΎΠ² ΠΊΡΡΠ» curl_setopt($ch, CURLOPT_URL, $url); // Π£ΡΡΠ°Π½Π°Π²Π»ΠΈΠ²Π°Π΅Ρ ΡΡΠ» curl_setopt($ch, CURLOPT_HEADER, 0); // Π²ΡΠΊΠ»ΡΡΠ°Π΅ΠΌ Π·Π°Π³ΠΎΠ»ΠΎΠ²ΠΊΠΈ Π΅ΡΠ»ΠΈ Π½ΡΠΆΠ½ΠΎ ... // ΡΡΡΠ°Π½Π°Π²Π»ΠΈΠ²Π°Π΅ΠΌ Π΄ΡΡΠ³ΠΈΠ΅ ΠΎΠΏΡΠΈΠΈ ΠΏΠΎ-Π½Π΅ΠΎΠ±Ρ
ΠΎΠ΄ΠΈΠΌΠΎΡΡΠΈ (Π½Π°ΠΏΡΠΈΠΌΠ΅Ρ ΡΠ°ΠΉΠΌΠ°ΡΡ Π·Π°ΠΏΡΠΎΡΠ°, Π·Π°Π³ΠΎΠ»ΠΎΠ²ΠΊΠΈ, ΠΏΡΠΎΠ²Π΅ΡΠΊΠ° Π²Π΅ΡΠΈΡΠΈΠΊΠ°ΡΠΈΠΈ ...) curl_multi_add_handle($mh, $ch); } //2. Π²ΡΠΏΠΎΠ»ΡΠ΅ΠΌ Π·Π°ΠΏΡΠΎΡΡ ΠΈ ΠΏΠΎΠ»ΡΡΠ°Π΅ΠΌ Π΄Π°Π½Π½ΡΠ΅ $status = null; do { curl_multi_exec($mh, $status); // Π²ΡΠΏΠΎΠ»Π½ΡΠ΅ΠΌ Π·Π°ΠΏΡΠΎΡΡ Π² ΡΠΈΠΊΠ»Π΅ $info = curl_multi_info_read($mh); // ΠΏΠΎΠ»ΡΡΠ°Π΅ΠΌ ΠΈΠ½ΡΠΎΡΠΌΠ°ΡΠΈΡ ΠΎ ΡΠ΅ΠΊΡΡΠΈΡ
ΠΎΠΏΠ΅ΡΠ°ΡΠΈΡΡ
$ch = $info['handle']; if (is_array($info)) { $content = curl_multi_getcontent($ch); // ΠΏΠΎΠ»ΡΡΠ°Π΅ΠΌ ΡΠΎΠ΄Π΅ΡΠΆΠΈΠΌΠΎΠ΅ } } while ($status == CURLM_OK); // 3 Π·Π°ΠΊΡΡΠ²Π°Π΅ΠΌ Π²ΡΠ΅ Π΄Π΅ΡΠΊΡΠΈΠΏΡΠΎΡΡ foreach ($chs as $ch) { curl_multi_remove_handle($mh, $ch); curl_close($ch); } curl_multi_close($mh); // Π·Π°ΠΊΡΡΠ²Π°Π΅ΠΌ ΠΌΡΠ»ΡΡΠΈΠΊΡΡΠ»
- @ Amirhon did not understand the question - Kostiantyn Okhotnyk
- oneIt turns out that the text was not fully inserted, the question was how to get out of <div class = "a"> <h1> info </ h1> <div> - info, but I found the answer, thanks - Amirhon