Hello. How can you correctly track the appearance of new materials on a particular site? Now I have a regular application in which a request is sent to the site every few minutes (using a timer) and a list of materials on the main page is obtained, and then compared to what is already there and materials that are not there are displayed in the corresponding field. Only now I understand that at such a pace I can soon catch a ban, especially if requests are sent too often. So I thought that maybe there is some normal way to track the appearance of materials? the site has rss.

  • one
    If у сайта есть rss. why not use it? - Anton Komyshan
  • Moreover in the .NET Framework there is a tool for working with rss out of the box ... - Anton Komyshan
  • Can you tell me how to work with rss? - Andrey Fedorov
  • Ok, got it. But about the requests: they, it turns out, everything is also sent using a timer? Or is there some way that will automatically receive updates on the machine itself? - Andrey Fedorov
  • It depends on where you are doing the request. Maybe the site has just an API for interacting with it. You can check for updates with the HEAD method and look at the Last-Modified header. Or watch the stauts code (304 Not Modified). It also happens that the feed sends in the body an additional parameter GUID or a special token or id of the feed or creation time - by which you can easily see if there are new elements there or not. To make a request in this case you will have to cycle / timer. - Anton Komyshan

0