I met this with respect to input , but never with respect to list Below is the code element. What is my mistake and how to fix it?

 import requests from bs4 import BeautifulSoup from lxml import * urls = ['https://www.sima-land.ru/kanctovary/shkolnye-tovary/opticheskie-pribory/', 'https://wmasteru.org/threads/Простой-парсер-товаров-с-amazon-с-помощью-beautifulsoup.22880/'] class PageDownload(urls): def ___init___(self): self.urls = urls def taking_things(self): for url in self.urls: s = requests.get(url) b = BeautifulSoup(s.text) self.title = b.select('soup.html.head.title') print(self.title) p = PageDownload() p.taking_things()

Further error text

 Traceback (most recent call last): File "C:/Users/kotov_or/PycharmProjects/PageDataDownloader/main.py", line 6, in <module> class PageDownload(urls): TypeError: list expected at most 1 arguments, got 3

Thanks in advance for your reply.

  • Specify in which line the error, and preferably the full text of the error. - insolor Nov.
  • Updated the question. - AgeofCreations
  • By declaring the class “class PageDownload (urls)” you indicated that you want to inherit it from the object “urls”. - MaxU

1 answer 1

In your code, something strange is going on, corrected:

 import requests from bs4 import BeautifulSoup class PageDownload: def __init__(self, urls): self.urls = urls def taking_things(self): for url in self.urls: s = requests.get(url) b = BeautifulSoup(s.content, 'html.parser') self.title = b.select_one('head > title') print(self.title) urls = ['https://www.sima-land.ru/kanctovary/shkolnye-tovary/opticheskie-pribory/', 'https://wmasteru.org/threads/Простой-парсер-товаров-с-amazon-с-помощью-beautifulsoup.22880/'] p = PageDownload(urls) p.taking_things()

See:

  • class PageDownload(urls): - in brackets after the class name indicates the type from which that is inherited
  • def ___init___(self): - here is a typo 3 underscores, instead of two

Ps.

I would also recommend replacing:

  • b = BeautifulSoup(s.text) to b = BeautifulSoup(s.content) , and preferably with a parser, for example: b = BeautifulSoup(s.content, "html.parser") or b = BeautifulSoup(s.content, "lxml")

  • And the select itself looks strange - the soup element should not be on that page, and you want to take the title of the page from the title tag, then: title = b.select('head > title')

Pps

Regarding the error, I wondered why it came out this way and sketched a small example:

 class Foo: def __init__(self, *args, **kwargs): print('__init__:\nargs={}\nkwargs={}'.format(args, kwargs)) urls = Foo() print() class PageDownload(urls): def __init__(self, urls): self.urls = urls

Console:

 __init__: args=() kwargs={} __init__: args=('PageDownload', (<__main__.Foo object at 0x000001D2FE773F60>,), {'__module__': '__main__', '__qualname__': 'PageDownload', '___init___': <function PageDownload.___init___ at 0x000001D2FE7782F0>}) kwargs={}

As can be seen from the console, the code called upon inheritance passes several parameters to the constructor.

It turns out that the TypeError: list expected at most 1 arguments, got 3 error TypeError: list expected at most 1 arguments, got 3 occurred when more parameters were passed to the list constructor than the one calculated - i.e. 1 parameter was expected, and 3 was received.

  • As for the postscript, I took this particular method from the official documentation of BeautifulSoup. But thanks for the advice. So do. - AgeofCreations
  • one
    @AgeofCreations, it’s understandable :) You didn’t quite understand that what’s in select is the css selector, and in the css-selector soup.html.head.title is to find such an element: <soup class="html head title"></soup> , and there was something else in the documentation - this is the writing of the tag at the BeautifulSoup object itself, i.e. in your case it would be: title = b.html.head.title . Do you understand? Kst, if anything, select returns a list, but if you need 1 element, call select_one - gil9red
  • one
    meta_tags = b.select('head > meta') , it will find tags that are direct children head - gil9red
  • one
    Please write in more detail, I am not a telepath :) You are talking about getting a tag using the example of <meta property="og:description" content="В наличии 571 шт. по цене от 10.04 руб. Закажите прямо сейчас!"> ? - gil9red
  • one
    Try this: b.select_one('meta[property="og:description"]') , and this way: b.select_one('meta[property=og:description]') - gil9red