Help me figure out how to parallelize loops ..
Now let's say I have some method that should be called with different input parameters in the loop. The result is written to the array.
Each loop iteration is independent of the others. How to parallelize this process?

And how can I be in this situation:

public class MainClass(){ public void main(){ Generator g = new Generator(); for (int i = 0; i < 10; i++){ ArrayList<String> res = g.funcion(i); // здесь все должно выполняться последовательно System.put.println(res.size()); } } } public class Generator(){ public function(int i){ // Вот тут то и хотелось бы распараллелить выполнение метода function2 ArrayList<String> res = new ArrayList<String>(); res.addAll(function2(i)); res.addAll(function2(i*i)); } public function2(int i){ // ... } }
  • In C # it would be: public IEumberable <string> function (int i) {return new [] {i, i * i} .AsParallel (). AsOrdered (). Select (function2); } On the other hand, you write in the question that the iterations are independent, and in the code that the loop in main should be executed sequentially. Which of these is true? - VladD
  • asked a few questions: one - how to parallelize the cycle of independent iterations; second on the example - Stas0n
  • @ Stas0n: well, I understand, these are independent questions. In .NET it is easy to parallelize, in Java it should also be, we wait for the experts to catch up. - VladD
  • 2
    Why do you always want something parallel? - IronVbif
  • @VladD, very nicely parallels on c ++, already pleases the eye) Only I have no pluses, but java - Stas0n

4 answers 4

If each iteration of the loop from main does not depend on the others, then it would be more logical to run each iteration in a separate thread (and not function2 calls as you wish). Then you will need a list of the output of each thread and some thread pool that tracks whether they have completed the work. After launching all the threads, the main thread will fall asleep, and when all the threads have completed, the pool will wake it up, it will go through the results and display them on the screen. For storing the work results of streams, it is better to use not a simple List , but thread-safe. This can be obtained either with Collections.synchronizedList(List<T> list) , or with the help of ReentrantLock , or by wrapping calls to this list in a synchronized block. ThreadPoolExecutor can be used as a thread pool. This article briefly describes how to use it. And you can wait for the release of Java 8, in which all this is implemented at the API level.

  • @ fori1ton, the fact is that the iterations in main depend on each other, it is impossible to break them into threads. The question was how to parallelize the function2 () method in the function method of the Generator class. - Stas0n
  • All of the above can also be used for paralleling function2 . Not really, by the way, it’s clear if you want to parallelize function2 calls, or something inside it. In the first case, there will be no big win, because you have only two challenges. In the second, it depends on what happens in function2 . The best way to parallel cycles. - fori1ton
  • @ fori1ton, winnings will be) function2 method eats up the lion's share of the program's work time ... - Stas0n
  • @fori1ton if function2 calls can be made independently, then why do you claim that iterations in main depend on each other? - rfq
  1. There is something like LinQ for Java - the LambdaJ library can be used to treat collections as certain entities without looping through their insides. Not sure that the output will be parallel execution - it already depends on the compiler. But syntactically it will be quite nice as in SQL.
  2. Real parallelization can be done with Thread 's, but you will not like the result ...
  • @Barmaley,> but you don’t like the result ... What do you mean? - Stas0n
  • 2
    @ Stas0n syntactically it does not look very ... - Barmaley

Create a class that will implement the Runnable interface.

 public class YourClass implements Runnable{ private int myInt; public YourClass(int _myInt){ myInt = _myInt; } @Override public void run(){ // Твой код } }

Instead of

 res.addAll(function2(i*i));

Write

 Thread thread = new Thread(new YourClass(i)); thread.start();

Transfer the code from function2 to run ()

  • @ dimka3210, and without the introduction of the class will not work out somehow? And how by the way to make so that the result of paralleling is assigned to any one variable? Well, i.e. How to make function2 (i) be one ArrayList <String> and function2 (i) different? - Stas0n
  • I would put the result of parallelization in HashMap - dlarchikov
  • one
    It would be more logical not to use Runnable, but Callable. He is able to return values. - IronVbif

something like this:

 public abstract class ThreadHelper { public static <T> void exec(Callable<T> ...callable) throws InterruptedException { // validate input if (null == callable || 0 == callable.length) { return; } List<Callable<T>> tasks = Arrays.asList(callable); // special for @IronVbif int cores = Runtime.getRuntime().availableProcessors(); // execute thread group ExecutorService threadpool = Executors.newFixedThreadPool(cores*2); threadpool.invokeAll(tasks); threadpool.shutdown(); } @SuppressWarnings("unchecked") public static void main(String[] args) { final List<Object> result = new LinkedList<Object>(); // callable one Callable<Object> callable = new Callable<Object>() { @Override public Object call() throws Exception { for (int i = 33; i < 77; i++) { result.add(i); Thread.sleep(15); } return null; } }; Callable<Object> callable2 = new Callable<Object>() { @Override public Object call() throws Exception { for (int i = 97; i < 133; i++) { result.add(String.valueOf((char)i)); Thread.sleep(25); } return null; } }; try { exec(callable, callable2); } catch (InterruptedException e) { System.out.println("failed: " + e.getLocalizedMessage()); } System.out.println(result); } }

google :

 java parallel execution
  • It is better to take the creation of a pool beyond exec and set it to a size comparable to the number of hardware threads in the processor. Because in such code it is potentially possible to transfer 1000000 objects and you will start the same number of threads. - IronVbif