original page

there is a page. It reflects a certain number of ads.

how to make python reflect more ads on one page,
then to load the whole page into a variable?
I use the module to work with html

import requests
from bs4 import BeautifulSoup

number of announcement

  • What's the problem? url='http://dom.59.ru/realty/sell/residential/new/1.php?order=DateUpdate&dir=desc&PriceUnit=1&AreaUnit=1&expand=0&PriceUnit=1#1.php%order=DateUpdate&dir=desc&PriceUnit=1&AreaUnit=1&expand=0&PriceUnit=1' and then as usual ... - MaxU
  • the problem is that python loads only the first 50 ads. and I need the page to display more, for example 300 ads. - Andrei Kruzlik

1 answer 1

For the number of ads displayed, the cookie value is list_col_pp. You can try to request data as follows:

 import requests cookies = requests.cookies.RequestsCookieJar() cookies.set("list_col_pp", "200") response = equests.get("http://dom.59.ru/realty/sell/residential/new/1.php?order=Da&dir=desc&PriceUnit=1&AreaUnit=1&expand=0&PriceUnit=1", cookies=cookies)

Or you can request using POST request as it is done when changing the number of displayed ads:

 response = requests.post("http://dom.59.ru/realty/sell/residential/new/1.php?order=Da&dir=desc&PriceUnit=1&AreaUnit=1&expand=0&PriceUnit=1", data={"action": "get_block", "name": "main"}, cookies=cookies)