Good afternoon, dear community.

Please conduct a code review and give advice. There is a program. It needs to run the same external EXE file N times with different input parameters. Wait until all the running processes are completed and collect the results into a single, say, array.

How to use parallelism and asynchrony (if the latter is needed here) with maximum efficiency?

I have so far made such a decision:

List<Tuple<int, int, string, string>> processignDataList = ... //список ΠΊΠΎΡ€Ρ‚Π΅ΠΆΠ΅ΠΉ с Π²Ρ…ΠΎΠ΄Π½Ρ‹ΠΌΠΈ Π΄Π°Π½Π½Ρ‹ΠΌΠΈ Task[] stackTasks = new Task[processignDataList.Count]; // массив всСх тасков List<Tuple<int, string>> tasksResults = new List<Tuple<int, string>>(); //список ΠΊΠΎΡ€Ρ‚Π΅ΠΆΠ΅ΠΉ с Ρ€Π΅Π·ΡƒΠ»ΡŒΡ‚Π°Ρ‚Π°ΠΌΠΈ ΠΎΡ‚Ρ€Π°Π±ΠΎΡ‚ΠΊΠΈ тасков int i = 0; // счСтчик, Π½ΡƒΠΆΠ΅Π½ Ρ‚ΠΎΠ»ΡŒΠΊΠΎ для указания элСмСнта массива foreach (Tuple<int, int, string, string> pair in processignDataList) { stackTasks[i] = Task.Factory.StartNew(() => tasksResults.Add(Tuple.Create(pair.Item1, pairProcessing(cSettings, pair)))); i++; } Task.WaitAll(stackTasks);

The string pairProcessing() method:

  • takes properties from the special object cSettings , input parameters from pair,
  • Runs EXE as Process - process.Start();
  • there is also a limit to the endless hangup - process.WaitForExit(3000)
  • then check that process.Close();
  • StandardOutput Analysis, and Based on It
  • forming a certain string and its return as a result of the work of the pairProcessing method

Do I achieve with this solution run multiple copies of EXE and parallel execution? Are there any additional pairProcessing() to the pairProcessing() method, are there any requirements for it, or can it be any? Do you know the solutions better? (and the solution is always better ...) Are there any shoals? Thank.

  • "Do I achieve with this solution launching multiple copies of EXE and parallel execution of them?" - easier to try and see. IMHO, the approach is correct: start the processes and wait for the completion of all. - Vladimir Martyanov
  • one
    In my opinion, the task here is not needed. Instead of a list of tasks, create a list of processes. Start and wait for the end of the process, not the task. - Alexander Petrov
  • @ VladimirMartyanov View ... The idea is simple, but effective, thank you. wtf.jpg.wtf/c7/86/… - there is multithreading and three separate processes. Now I’ll still check whether it is possible to throw out tasks from this chain and leave some processes, as Alexander advises. - Natalia
  • If it's just a list of processes, then it should go if you are not reading their stdout. (And if you read, you have to think.) - VladD
  • @VladD generally read. In addition to starting the process, I also have to read its stdout, analysis, and possibly some other actions. I will be uncomfortable without a task. - Natalia

1 answer 1

Firstly, you do not have synchronization of access to tasksResults - there may be an unpleasant race.

Secondly, if there are a lot of processes, you may run out of threads in the pool, which will undesirably limit parallelism. However, too many processes will still not be able to work in parallel.

The first problem is solved quite simply - instead of writing to the list inside the task, you must allow the task to return the value:

 List<Task<Tuple<int, string>>> stackTasks = new List<Task<Tuple<int, string>>>(); foreach (Tuple<int, int, string, string> pair in processignDataList) { stackTasks.Add(Task.Factory.StartNew(() => Tuple.Create(pair.Item1, pairProcessing(cSettings, pair)))); } Tuple<int, string>[] tasksResults = Task.WhenAll(stackTasks).Result;

You can also use linq:

 Tuple<int, string>[] tasksResults = Task.WhenAll( from pair in processignDataList select Tuple.Create(pair.Item1, pairProcessing(cSettings, pair))) ).Result;

or so:

 Tuple<int, string>[] tasksResults = Task.WhenAll( processignDataList.Select(pair => Tuple.Create(pair.Item1, pairProcessing(cSettings, pair)))) ).Result;

To solve the second problem, it is necessary to move from tasks to threads. Or increase the size of the thread pool with ThreadPool.SetMaxThreads and ThreadPool.SetMinThreads