Parallel PHP Script Execution

Question

Hello. I make a function to import the contents of a .csv file into a MySQL database. Records of more than 40,000 plus must be checked for duplicates. No effort was able to stuff the work time in a reasonable 30 seconds. Decided to run in the background using an AJAX request. And other AJAX-ohms from time to time to check the status of implementation.

The first script cyclically rewrites the file in the database and writes the current value in the $ _SESSION ['import'] variable.

public function action_importcore() { $auth = Auth::instance(); if ($auth->logged_in()) { if (isset($_GET['file'])and(isset($_GET['group']))) { $filename = $_SERVER['DOCUMENT_ROOT'].'/bases/'.$_GET['file']; $group = ORM::factory('group', $_GET['group']); $_SESSION['fullimport'] = filesize($filename); $fp = fopen($filename, 'r'); $colum = fgetcsv($fp, 255, ';'); set_time_limit(0); while ($dt = fgetcsv($fp, 255, ';')) { $person = ORM::factory('person', array($colum[1] => $dt[1])); if ($person->loaded()) { if (!$person->has('group', $group->id)) $person->add('group', $group); } else { $person->set($colum[1], $dt[1])->set($colum[0], $dt[0])->save(); $person->add('group', ORM::factory('group', $group)); } $_SESSION['import'] = ftell($fp); } } else { $this->request->redirect('/'); } } else { $this->request->redirect('/'); } }

The second reads the value of $ _SESSION ['import'], considers the percentage and gives it to AJAX.

 public function action_importping() { $auth = Auth::instance(); if ($auth->logged_in()) { if (isset($_SESSION['import'])and(isset($_SESSION['fullimport']))) { echo round($_SESSION['import']/$_SESSION['fullimport']*100); } else { echo 50; } } else { $this->request->redirect('/'); } }

The problem is that during the execution of the first script, the second one does not respond. No script at all! The web server (by the way, Apache 2.2.22 under Windows) gives pictures and static pages, but hangs when you request to any .php script. As soon as the first script exits, all the others can run.

Help to understand what is the reason?

Also with the help of curl (it turns out that the server will query itself, but oh well) - I can make several connections to one script at once, without waiting for an answer (set the timeout).
40 thousand is a garbage, the processing should fit in seconds at 10 on a standard hosting.
But since there are problems with the time limit - terminate the script after every n lines processed and start again.
In addition, as far as I remember, the session is recorded in the file only when the script completes its work.
in your case, while the first script is running, the second one does not have access to the session, or at least does not see changes in the $_SESSION['import'] variable.
Indifferent, 40 thousand - this is garbage, the processing should fit in seconds in 10 - how?
The script that I posted stuffed 42,000 lines into the database in just over an hour.

Sh4dow Sh4dow 13.8k sixteen 33 · Accepted Answer · 2012-11-13T12:06:45

Problem in session: when launching a script where a session is used, the script locks the session file. The next one accordingly waits for unlocking, then it starts.

An option like this:

create a label [ ключ (session_id); значение ( [ ключ (session_id); значение ( $ import ) ]

in the import script do so

 <? session_start(); define('SID', session_id()); session_write_close(); // имеем на руках session_id и закрытую сессию ?>

in the import process, insert a pair into the nameplate (SID, $import)
in the validation script, match the value (you can not close the session, it is fast)

Option easier and "knee"

  <? // import.php session_start(); define('SID', session_id()); session_write_close(); // code, code etc file_put_contents(SID.'.data', $import) ?> <? // check.php session_start(); $import = (int) @file_get_contents(session_id().'.data'); ?>

Tin ... Isn’t it easier to enter a new temporary status file? Why so scoff at a poor session?

Parallel PHP Script Execution

1 answer 1

More articles: