Good day, dear!

There are several applications, each of which accesses a web service, collects data from it and processes it. It is necessary to apply periodically (the data changes with the passage of the day, imagine that this is the exchange rate from the Central Bank website). Question: how to synchronize applications so that they do not save the necessary data twice? Allow access to the service only one application can not.

If there was only one client, I would do this: I would take the data, save it to the database and the database cache, check whether there is any such data at the next call.

But the situation is more complicated. You can save data in a shared database, but since there is no synchronization, there is no guarantee that clients will not start at the same time and simultaneously bring the same course value into the database (for 15:00, for example). What are my options?

  1. Somehow to lock a DB while one of clients is fulfilled. Like lock or synchronized in Java. It can be done?

  2. Load all into one table with a query like insert if unique , and then read from there. In principle, the normal option, although it seems to me more resource-intensive than the first.

  3. Save the index of the next client in the table so that the rest will not start at the same time (that is, spread them out in time).

What do you think?

  • 2
    And you can not place a service in front of the database, to which customers will turn and who will single-handedly manage data update requests? - yozh

3 answers 3

And who prevents to do so. We make one application that knows when and how to access an external service, how to read and write.

And all your services are connected to this application. And it either gives from the cache, or makes a request and gives. This scheme will solve many questions - synchronization, data relevance, debugging. Also, imagine that an external service has changed the API / address. With the right approach, all applications will not even notice that the external service has changed / fell / sent obviously incorrect data.

    Point 1 is clearly bad ideologically, even if you find such an opportunity. Because You want to keep customers waiting. Although if there are few customers, then maybe they can suffer.

    Point 3 is bad because in general it is approaching the 1st.

    Point 2 - or some of its variations - good. About the same said yozh in his comments. Those. the point is that clients do not save data directly, but save the "request for data saving", and then either a separate service or a stored procedure in the database disassembled these saved requests and single-handedly determined what to save and what was not necessary.

    The β€œrequest to save” is apparently saved in a table containing the client number, the data he wanted to save and the time when he provided them, as I understand it.

      Either you are checking at the database level or at the application level (for example, web service).
      at database level:

       Π°) ΠΏΠ΅Ρ€Π²ΠΈΡ‡Π½Ρ‹ΠΉ ΠΊΠ»ΡŽΡ‡ Π² Ρ‚Π°Π±Π»ΠΈΡ†Π΅(Π°Ρ…) Π±) Ρ‚Ρ€ΠΈΠ³Π³Π΅Ρ€Ρ‹ с) Π²Π·Π°ΠΈΠΌΠΎΠΈΡΠΊΠ»ΡŽΡ‡Π°ΡŽΡ‰ΠΈΠ΅ запросы Ρ‚ΠΈΠΏΠ°: insert into dest select * from input where not in ( select * from dest )

      at the application level:

       Π°) самым простым ΠΈ Π±Π°Π½Π°Π»ΡŒΠ½Ρ‹ΠΌ Ρ€Π΅ΡˆΠ΅Π½ΠΈΠ΅ΠΌ являСтся Π΄ΠΎΠ±Π°Π²Π»Π΅Π½ΠΈΠ΅ синхронизации ΠΌΠ΅Ρ‚ΠΎΠ΄ΠΎΠ². Π±) ΠΊΠ°ΠΊ Π²Π°Ρ€ΠΈΠ°Π½Ρ‚ - использованиС ΠΎΡ‡Π΅Ρ€Π΅Π΄ΠΈ Π·Π°Π΄Π°Π½ΠΈΠΉ. Π³Π΄Π΅ ΠΊΠ°ΠΆΠ΄Ρ‹ΠΉ ΠΊΠ»ΠΈΠ΅Π½Ρ‚ добавляСт своС Π·Π°Π΄Π°Π½ΠΈΠ΅ Π½Π° ΠΎΠ±Π½ΠΎΠ²Π»Π΅Π½ΠΈΠ΅ Π‘Π”. Ρ‡Ρ‚ΠΎ ΠΏΠΎΠ·Π²ΠΎΠ»ΠΈΡ‚ ΠΈΡΠΊΠ»ΡŽΡ‡ΠΈΡ‚ΡŒ ситуации дублирования.

      Validation at the application level may adversely affect system performance. in terms of performance, it is more convenient at the database level to do this.

      ps there is a suspicion that you have incorrectly designed database once you have such a problem