I want to collect all the information from a specific category with one call to Phantomjs (walk through page-by-page navigation, switching between pages). The fact is that when the site is clicked on the page number / arrow, the target block changes with the information and the URL of the page via ajax, i.e. page overload does not occur. Phantomjs does not understand this and continues to take information from the initially loaded page.

The option to run page.open several times with a URL change is not very good, it will take too long to load and the load will be too large.

Sample code:

page.open(url, function (status) { var data = page.evaluate(function() { //Пока стрелка вправо активна, нажимаем на неё while ($('.arrow_right').hasClass('disabled') === false) { //Логика сбора данных //Нажимаем на стрелку var ev = document.createEvent("MouseEvents"); ev.initEvent("click", true, true); document.querySelector('.arrow_right').dispatchEvent(ev); //Так подозреваю, что здесь что-то надо сделать для обновления данных } }); console.log(data); phantom.exit(); });

Type link site.ru/category/page/1/

Tell me, is there such an opportunity at all or not?

    1 answer 1

    To solve this problem, you can wait for the desired state until the data on the page is updated by checking this condition, for example, every 50 milliseconds. The following is an example of the implementation of such a function of the receiver using promises Q:

     var Q = require('q'); var currentUrl = page.url; waitState(function urlChanged() { // ждем смены урла return page.url !== currentUrl; }).then(function() { // будет выполнено, когда дождались состояния смены урла }) function waitState(state, timeout) { console.log('Старт ожидания состояния ' + state.name); var limitTime = timeout * 1000 || 20000; var startTime = new Date(); return wait(); function wait() { return Q() .then(function() { if (state()) { console.log('Дождались состояния ' + state.name); return; } else if (new Date() - startTime > limitTime) { var errorMessage = 'Таймаут ожидания состояния ' + state.name; console.log(errorMessage); throw new Error(errorMessage); } else { return Q.delay(50).then(wait); } }) .catch(function(error) { throw error; }) ; } }