There is a queue of some atomic operations, in my case these are portions of data that are loaded into the database, it can also be, for example, a queue of files when copying. I want to calculate and deduce the rate of these operations, for example, so that at any moment I can see that 20 operations have been completed in the last minute or, for example, 120. I do not need super accuracy - a deviation of 2-5 units is considered acceptable. In parallel, even portions of the data can be added to the same queue from the outside, so it’s impossible to simply print the number of elements in the queue. How to implement it?

While typing the question, there was about this solution:

using System; using System.ComponentModel; using System.Linq; using System.Timers; class Program : INotifyPropertyChanged { static void Main(string[] args) => new Program().Run(); int count = 0; int[] tablet = new int[60]; public int Tempo => tablet.Sum(); public event PropertyChangedEventHandler PropertyChanged; void Snick() { ++count; } void Cut() { int s = DateTime.Now.Second; tablet[s] = count; count = 0; PropertyChanged?.Invoke(this, new PropertyChangedEventArgs(nameof(Tempo))); } void Run() { Timer timer = new Timer(1000); timer.Elapsed += (s, e) => Cut(); timer.Start(); // ... while (true) { // ... Snick(); } timer.Stop(); } }

But I still don’t fully like it, even if it introduces an array and a timer, if I suddenly want to output the rate in 1 s - the load from the timer can become significant, it’s also not clear what sections to break.

Maybe someone has a ready-made solution that is better?

  • on the interval of 1 s, the load from the timer will not be significant, for 1 ms it is possible, although also in doubt, the timer in a separate stream works. But to use DateTime.Now for counting is nonsense, get a static counter for a method call, timer to fix the value of the counter and reset it. - rdorn
  • Yes, but I have more than 40 such queues in different streams and add another one to each stream - it can already be expensive. But again, I can use timer one for all the queues ... - Andrew NOP
  • Why everyone, timer one. to each thread you will need to add only the increment counter of completed actions - rdorn
  • By the way, I ’ve looked at the Stopwatch.GetTimestamp () method now: msdn.microsoft.com/en-us/library/… - it just takes its value from DateTime.Now - Andrey NOP

2 answers 2

Why so hard?

Such a simple code should work:

 TimeSpan timeout = TimeSpan.FromMinutes(1); long OperationsDuringLastMinute { get; private set; } async void RegisterOperation() { OperationsDuringLastMinute++; await Task.Delay(timeout); OperationsDuringLastMinute--; }

At the end of each operation, call RegisterOperation (on the main thread).

If you do not have a UI context, then, as @Raider suggests in the comments, you will need atomic operations.

 long operationsDuringLastMinute; long OperationsDuringLastMinute { get { return Interlocked.Read(ref operationsDuringLastMinute); } } async void RegisterOperation() { Interlocked.Increment(ref operationsDuringLastMinute); await Task.Delay(timeout); Interlocked.Decrement(ref operationsDuringLastMinute); }
  • I do not fully understand the work of tasks, but this code will not create 100 (by the number of completed operations) hanging in the background of nothing that does nothing? Or is it somehow everything should be cunningly optimized in the end? - Andrey NOP
  • @Andrey: No, of course, no 100 threads! Tasks hanging in await do not occupy any thread at all. - VladD
  • Well then this is exactly what I need, thank you! - Andrei NOP
  • @ Andrei: Please! Glad that helped. Be sure to read and understand async / await, this is a very powerful technique. - VladD
  • Task.Delay I advise you to also deal with async / await, and specifically with Task.Delay . This code will create 100 timers (according to the number of operations performed), each of which, after a minute, will try to take a stream from the Thread Pool (or return to the UI synchronization context, but this is not our case) to perform the decrement operation, which is not yet atomic . Compared to this, the author’s version with 1 timer and an unnecessary array does not look so crooked. - Raider

If you need to track the performance of any lengthy operation, regardless of itself, in any case, you will need 2 threads, and some kind of shared memory area where the overall progress will be recorded.

The timer is not needed here: firstly, another one is taken for it (without going into details) an unnecessary stream from the thread pool, secondly, its accuracy is completely lame, and it was not created for that.

For measuring the exact time stamps in .NET there is a class Stopwatch , but since your task is to count the number of operations in the last minute, then a variable with the previous measurement in combination with Thread.Sleep will be enough. If desired, Stopwatch and Thread.Sleep can be combined to get a more accurate result.

For clarity, I used quantization by the second, but the interval is easily changed by the method parameter:

 public static class Program { private static readonly ConcurrentQueue<string> Results = new ConcurrentQueue<string>(); static void Main(string[] args) { Task work = Task.Run(new Action(DoWork)); MeasureRate(work, Results); Console.WriteLine("Completed!"); } private static void MeasureRate<T>(IAsyncResult work, IReadOnlyCollection<T> results, int timing = 1000 /* 1 second */) { int previousCount = 0; do { Console.WriteLine("Current rate: {0} entries per second", results.Count - previousCount); previousCount = results.Count; Thread.Sleep(timing); } while (!work.IsCompleted); } private static void DoWork() { var randomizer = new Random(); for (int i = 0; i < 100000; i++) { Thread.Sleep(randomizer.Next(100)); Results.Enqueue("Result " + i); } } }
  • Your option does not suit me, because you calculate the average speed for all the time, but I need exactly the number of operations for the last minute, since the work is organized in the following way: while there are elements in the queue - we work with them, when they are finished - we fall asleep for half an hour (or an hour), then again check the status of the queue. - Andrei NOP
  • I have one data source - a process-scanner and 40 processes working with a database, this scanner scans a certain folder (s) and, when files appear in them, forms chunks of data and places them in a queue of one of 40 processes. After processing all the files, he also falls asleep for an hour. - Andrei NOP
  • Then you just need to save the previous measurement in a variable, and every minute to read the difference. I updated the sample code. - Raider
  • Well, I understood your idea and took note, but you don’t quite see what the int array in my version is for [60] - i.e. I do not need to get the number of operations for each specific minute, but for the last minute, that is, every second I have continuously the number of operations in the last 60 seconds. Those. at 00:01:00 I will display the number of completed during the interval 00: 00: 00-00: 01: 00; at 00:01:01 - the number of completed during the interval 00: 00: 01-00: 01: 01, etc. In my example, the data is updated every second, if I want to update every 5 seconds, then an array of 12 numbers is enough, for example. - Andrey NOP
  • Those. counter, which will be reset to zero every minute and run again? And why do you need an array if you only need the last minute? - Raider