I want to write a service on the bottle framework, which will parse information from other sites once in a while and add it to the database. And when you receive a request, output the summarized information from the database and output it in the form of an HTML page. The question is: how to force the application on the bottle to independently collect information from other sites once a certain time?

from bottle import route, run, template def parser(time=n): # функция парсинга сайтов запускаСмая Ρ€Π°Π· Π² n-ΠΌΠΈΠ½ΡƒΡ‚ @route('/report/') def index(): return template(# суммированная ΠΏΠΎ ΠΎΠΏΡ€Π΅Π΄Π΅Π»Π΅Π½Π½Ρ‹ΠΌ критСриям информация ΠΈΠ· Π‘Π”) run(host='localhost', port=8080)
  • I propose to divide the tasks: write a script that will parse and add information to the database, and the web server will read and generate a report from the database at the request of the report - gil9red
  • In the original, I did, but those who gave me this task insisted on a multi-threaded web service. - Valentin Fedyakov

1 answer 1

Out of the box this is not able to bottle.

The easiest way is to write a separate script for this and execute it once an hour / day. For example, using cron .

A more advanced way is to use the queue manager, for example, Celery .

  • Celery in my opinion is too bulky for such a task. Found as an option solution through multiprocessing. paste.ofcode.org/9ziuqcBB9DVbmf8Fzr9Bnt How correct would it be to write like that? - Valentin Fedyakov
  • It is better to post a site and a script, then they will be independent. Now, if something breaks while parsing, the site will be unavailable, and this is not very. In addition, the constantly working process will gradually eat off the memory. - Lebedev Ilya
  • those. in the function parser to call, in the body of an infinite loop with a slip, exec (script with parser)? - Valentin Fedyakov
  • No, just once in en minutes run python parse.py , which will update the data and stop execution. - Lebedev Ilya
  • "Out of the box of bottle this does not know how" being technically correct, it can still distort the truth: bottle is distributed as a single file without dependencies, except for a small core of functionality to create a wsgi application, everything else is at the discretion of the user : which ORM, i18n, auth , cache, etc to use (there are separate packages, but it is not necessary to use them). That is, bottle does not include it now and it is very likely that it will not include a queue of distributed tasks in the future (you can use different solutions separately). cron is one of the simple and (relatively) reliable solutions - jfs