Please tell me, I get the page, I find the necessary data, this is a table. How to decompose data from a table into a pair of key - value? using BeautifulSoup

<tr> <td> Принципал </td> <td> МОРТП "АНЗАСТ" </td> </tr>

my code

  import urllib.request import urllib.parse from bs4 import BeautifulSoup url = 'https://fedresurs.ru/messages/4073AEF1951F13998374D373A64CB497' def getpage(lnk): with urllib.request.urlopen(lnk) as response: html = response.read() return html soup = BeautifulSoup(getpage(url)) table = soup.find('table', class_='mesview') # print(table.prettify()) for tr in table.find_all('tr'): print(tr.td)

At the output, get the dictionary key: value Complicated by the fact that there is a nested table in one tr. How to handle it correctly?

  • Your goal is to get a "flat" (single-level) dictionary or nested, corresponding to the source table? - MaxU
  • MaxU, "Flat", and the nested table convert the Beneficiary: Name, Identifiers: TIN, it will be quite enough. - ss_beer
  • Can a nested table have more than one beneficiary? And if so, how do you imagine making this a flat structure? - MaxU
  • Maybe in this case, just take the first. Of course, if you make full compliance with the source data - it will be cool, but, unfortunately, I don’t know either of these methods :( - ss_beer

1 answer 1

I still decided to make a nested list for the beneficiaries, so as not to lose information. Selecting the first item from this list is easy ...

 import re import requests from bs4 import BeautifulSoup url = 'https://fedresurs.ru/messages/4073AEF1951F13998374D373A64CB497' r = requests.get(url) soup = BeautifulSoup(r.content, 'html.parser') data = {} # iterating tables with the <table class="mesview"> attribute for tab in soup.find_all('table', attrs={'class':['mesview']}): for row in tab.find_all('tr'): tmp = row.find_all('td') if len(tmp) == 2: data[re.sub(r'\s{3,}', '', tmp[0].text.strip())] = \ re.sub(r'\s{3,}', '', tmp[1].text.strip()) bnfcr = [] # iterating table(s) with the <table class="mandatoryAssessmentTable"> attribute for tab in soup.find_all('table', attrs={'class':['mandatoryAssessmentTable']}): # parsing column names from table's headers cols = [col.text for col in tab.find_all('th')] for row in tab.find_all('tr'): tmp = row.find_all('td') if len(tmp) == len(cols): bnfcr.append({cols[0]: tmp[0].text.strip(), cols[1]:tmp[1].text.strip()}) data['Бенефициары'] = bnfcr print(data)

Result:

 {'': 'Изменяемый срок действия', 'Бенефициары': [{'Бенефициар': 'МАДОУ № 13 "УМКА"', 'Идентификаторы': 'ИНН: 5047086394 ОГРН: 1075047010114'}], 'Гарант': 'БАНК "ВОЗРОЖДЕНИЕ" (ПАО)(ИНН: 5000001042, ОГРН: 1027700540680)', 'Дата выдачи': '26.12.2016', 'Дата завершения действия': '02.03.2018', 'Дата начала действия': '26.12.2016', 'МАДОУ № 13 "УМКА"': 'ИНН: 5047086394 ОГРН: 1075047010114', 'Номер гарантии': '063-158-БГ-2016', 'Обстоятельства выплаты': 'Получение требования Бенефициара с указанием, в чем состоит нарушение Принципалом основного обязательства, в обе спечение которого выдана гарантия.', 'Описание основного обязательства': 'Исполнение обязательств по контракту/договору', 'Принципал': 'МОРТП "АНЗАСТ"(ИНН: 5047000703, ОГРН: 1025006178010)', 'Сумма': '947 917.98RUB', 'Тип сообщения': 'Сообщение о выпуске независимой гарантии', 'Торги': '№4от16.12.2016'}
  • Thank you very much. Went to understand :) - ss_beer