https://plastinfo.ru/trade/sell/raw/ - the site from which you want to extract data

To get a row from a table, I use the following code:

import urllib.request from bs4 import BeautifulSoup def get_htlm(url): response = urllib.request.urlopen(url) return response.read() def parse(html): soup = BeautifulSoup(html,"html.parser") div = soup.find('div',class_ = 'trade_board_border') table = div.find('table') tbody = table.find('tbody') row = tbody.findAll('tr') print(row) def main(): parse(get_htlm('https://plastinfo.ru/trade/buy/raw/')) if __name__ == '__main__': main()

As a result of execution, it returns only the 1st row from the table.

 [<tr class="odd"><td>13:25</td><td><a class="trade" href="/trade/buy/raw/526624/" onclick="pageTracker._trackEvent('trabe', 'buy', 'object', 526624);"></a></td></tr>]

How to make the result all the lines? I use Python 3, BeautifulSoup 4, Visual Studio

    1 answer 1

    I tried to get the items that interest you. I ran into the same problem. It seems that the remaining rows of the table are loaded not after weaning the response , but later.

    Adding page1 , page2 , etc. in the tail of the link also gives nothing.

    You have to switch to selenium to simulate the user and actually load the data on the page.