I am writing a parser on python.
The pages I need are only seen by an authorized user. I logged in via chrome / firefox, but the parser is given an authorization page.
Where to look?
Timofei Bondarev
6,940 3 gold marks 32 silver marks 51 bronze marks
Mikhail Boroda Mikhail Boroda
6
- depends on the site, somewhere it is enough to transfer the header , somewhere cookies, sometimes even browser automation using selenium webdriver or phantomjs will not help. - jfs
|
2 answers
The fact is that most likely your parser does not support authorization. The fact that you entered the site through the browser does not mean anything, because the browser and your parser are different applications.
In order for your parser to be able to use browser data to enter, it must be passed to it as a cookie. You can see which cookies the site puts in the Chrome developer console (Ctrl + I):
Cobalt cobalt
585 2 silver marks 17 bronze marks
|
You can look for the python library mechanize. There are examples with authorization and further work with the site. It is described simply, the examples are simple. I think the difficulties will not arise.
mixt mixt
21 2 bronze marks
|