There is an asp.net parser application from many sources (VKontakte groups). The response from each is about 200 ms. Hundreds of groups give a delay already in 20 seconds, and there may be several hundreds. It is necessary to somehow do such an operation in the background, i.e. not at the moment when the user entered the site. In the background, data will be collected and stored in the database, for example. A user visiting the site will simply receive them from the database. Question: how to make the application work without user intervention? Something must initiate the execution of a code that, for example, will do things on a timer. As an option - a micro application working around the clock on a home PC, which makes http request to the main application. Multithreading helps a little, but still processing time is very long - do not offer it.

  • You can make a web service that will collect information and put it in the database. - Trymount
  • one
    Usually such processing, if a little and once in a while, is done with the help of the scheduler. If the job is permanent, then a separate worker is started. For example, the scheduler calls some kind of application url which initiates processing. - Eugene
  • those. scheduler and will be a separate application that causes the main? - olegall

1 answer 1

I agree with the opinion that you need to do the service.

But the logic can be different, you can make it so that the service would thresh everything, but the relevance of the data will be with a long delay.

You can make it so that the service is given a task to calculate specific data. Then from the asp.net application we give the task to the service and immediately rather than waiting for the timeout we give an answer that the request was formed and return the request id, then when the service completes the task it signals the client.

You can inform in different ways, if in the forehead, it is possible after the completion of the task, the service will create a record in b.d. and javaScript every 5 - 10 seconds, pull the result of the task by the previously obtained task id.