There is a script, I want to make it multi-threaded.

Suppose there is a certain array of hashes containing

my %hello : shared; print Dumper(\%hello); $VAR1 = { 'a' => '1', 'b' => '2', 'c' => '3', 'e' => '4', 'f' => '5', 'g' => '6' };

And a certain function which, let's say, just print'om displays the elements of this array on the screen.

What I want: I want to run it somehow in, say, 2 threads. I see it this way: Since the array is shared, we run in an infinite loop until something in the array is our function that simply “takes” the value from it and does something with it.

The question is: how can this be explained to the machine on perl and will it work at all? :)

UPD

Something like that, just so that the streams do not do the same actions (Now they just in turn output all the contents of the hashes array completely):

 #!/usr/bin/perl use threads; use threads::shared; my $threads = 2; my @threads; my %hello : shared; sub test { my $num=shift; print "+ Thread $num started.\n"; foreach my $key (keys %hello) { print $key, $hello{$key} . "\n"; } } for my $t (1..$threads) { push @threads, threads->create(\&test, $t); } # Дожидаемся окончания работы всех потоков foreach my $t (@threads) { $t->join(); } exit(0);
  • The first thing that gave Google on request is "perl multithreading": habrahabr.ru/post/128477 is a great example, though they have an array and not a hash. But no one bothers to work with the shared variable as with a normal one. By the way - you can delete a hash element using delete - Mike
  • Hi @Mike, yes I read it, a lot of things, but how to do it for my needs - I don’t think :( The bad from me, apparently, is a perl programmer. - user226699
  • And if you add to the delete($hello{$key}) function delete($hello{$key}) , the first thread does all the work, only somehow “crookedly”, sometimes stitching together lines and, in my opinion, duplicating them ... apparently I don’t understand something, did not take into account something. - user226699
  • Yes, I understood your problem. Now I’ll try to figure it out, never dealt with the streams :) I can say that I need to block the variable in order to get the value and delete it ... - Mike
  • @Mike Type is probably probably like that on stackoverflow.com/questions/23753538/… I’ve got it on a long time ago, but this is some kind of hard tin plate, I thought it’s somehow easier to do. Actually, I wrote a multithreaded script 2 years ago to scan the network, but there I used “shell” and this time I wanted to do it in a normal way. Last time, I did this in the system("/usr/bin/nmap -p80 -PN -n -T5 --defeat-rst-ratelimit --nsock-engine select --open --min-parallelism 254 -oG - $ip| awk '/open/{print \$2}' >> $path"); function system("/usr/bin/nmap -p80 -PN -n -T5 --defeat-rst-ratelimit --nsock-engine select --open --min-parallelism 254 -oG - $ip| awk '/open/{print \$2}' >> $path"); - user226699

2 answers 2

I do not know how much work it really is, I had to debug the ideone, because multi-threaded perl was not at hand ...

 use threads; use threads::shared; use Time::HiRes qw(usleep); my $threads = 2; my @threads; my %hello : shared = (a,1,b,2,c,3,d,4); sub test { my $num=shift; print "+ Thread $num started !.\n"; while(1) { my ($key,$val); { # block for lock lock(%hello); $key=(keys %hello)[0]; $val=$hello{$key}; delete $hello{$key} if($key); } last unless $key; print "Thread $num: $key $val\n"; usleep(100*int(rand(10))); } } for my $t (1..$threads) { push @threads, threads->create(\&test, $t); } # Дожидаемся окончания работы всех потоков foreach my $t (@threads) { $t->join(); } exit(0);

At least each of the values ​​is printed 1 time. I had to use sleep, otherwise one thread managed to process everything.

The basic idea: blocking the hash, retrieving one key, storing the value, deleting the key, unlocking the hash (no explicit block removal is provided, the block is removed at the end of the code block). In addition to receiving and deleting an element while taking a lock, we do not do anything; work happens after the lock is released. If this is not done, then our threads will be executed strictly in turn, and not simultaneously.

  • OO And what a strong witch my ($key,$val); { ... } my ($key,$val); { ... } ? - user226699 7:26
  • @ user226699 Curly brackets to my have nothing to do. They in themselves, they frame the syntactic block (the block is not obliged to begin with all sorts of for / while). This syntactic block is needed so that after it a blockage is removed, because it acts within the block - Mike
  • Obviously ... I don’t think that perl is a human ridbl :) Do you think it’s better to do it through an array of hashes or through a regular array? - user226699
  • @ user226699 depends on the task, if you need quick access to meaningful keys, then a hash, and if just a queue for work, then an array. At least with an array, no blocking is needed if you use shift / pop. And if now the hash - it may well leave less to alter. On the other hand, it is necessary to quickly pull out and remove an element, which means that it will be impossible to use $ hash {$ key} below the code, the element is then deleted. - Mike
  • @ user226699 By the way, what do you call an array of hashes? In the example you have just a hash :) - Mike

I sketched it on Mojo::IOLoop , but I spat and still fell off on AnyEvent :-) Additionally, I know how to limit the processing (launch of a maximum of M simultaneous workers in N seconds). Although, if we talk about restrictions, it could not be so. But this is a completely different area.

 #!/usr/bin/env perl # ----------------------------------------------------------------------------- use Modern::Perl; use AnyEvent; use Const::Fast; use Data::Lock qw/dlock dunlock/; use Data::Printer; # ----------------------------------------------------------------------------- const my $MAX_WORKERS => 10; # количество одновременных запросов (M) const my $INTERVAL => 1.0; # интервал (N) const my $DEBUG => 1; my %data = map { $_ => $_ } ( 1 .. ($MAX_WORKERS * 10) ); my %result; my %dups; # ----------------------------------------------------------------------------- while (keys %data) { my $tstart = AnyEvent->time(); my $cv = AnyEvent->condvar(); # формируем $MAX_WORKERS одновременных запросов: for ( 0 .. $MAX_WORKERS ) { my $id = $_; dlock(%data); my $key = ( keys %data )[0]; if ($key) { $cv->begin(); my $val = $data{$key}; $dups{$key} = $val if $result{$key}; $result{$key} = $val; say "[worker $id] $key => $val"; delete $data{$key}; dunlock(%data); # # здесь делаем что нужно # $cv->end(); } dunlock(%data); } $cv->recv(); last unless keys %data; my $sleeping = $INTERVAL - ( AnyEvent->time() - $tstart ); # слишком быстро - поспим: if ( $sleeping > 0 ) { printf( "sleeping for %.3f seconds...\n", $sleeping ) if $DEBUG; my $sleep = AnyEvent->condvar(); my $timer = AnyEvent->timer( after => $sleeping, cb => $sleep ); $sleep->recv(); } else { say 'no sleeping needed' if $DEBUG; } } say 'Done, dups:'; p %dups;