[Parallel] processing is, in sharp contrast to just a [Concurrent] processing, guaranteed to start / perform / finish all thread-level and/or instruction-level tasks executed in a parallel fashion and provides a guaranteed finish of the simultaneously executed code-paths.
0
votes
1answer
58 views
Concurrence of Julia Parallel Computing
I am new to Julia and studying Julia parallel computing recently.
I am still not clear about the accurate mechanism of Julia's parallelism including macros \@sync and \@async after I read the relevant ...
0
votes
0answers
6 views
Implement parallel processing of for loop
Looking to make the following code parallel- it reads in data in one large 9gb proprietary format and produces 30 individual csv files based on the 30 columns of data. It currently takes 9 minutes per ...
1
vote
2answers
39 views
C# Parallel.ForEach opening up too many connections from the pool
The full error received is this:
Timeout expired. The timeout period elapsed prior to obtaining a connection from the pool. This may have occurred because all pooled connections were in use and max ...
0
votes
0answers
12 views
Unexpected result from MPI isend and irecv
My goal was to send a vector from process 0, to process 1. Then, send it back from process 1 to process 0.
I have two questions from my implementation,
1- Why does the sending back from process 1 ...
-1
votes
1answer
26 views
What are examples of dependent work that will prevent async-await from having any benefit?
Quick question about async-await.
If I have a chunk of code like
object foo;
Task t1 = LongRunningTaskThatSetsFoo(out foo);
UseFooToDoSomething(foo);
await t1;
then what will happen? Because ...
0
votes
0answers
14 views
Python: parallelize to processing the dataframe
I have a big file and I want to parallelize that.
It's a part of them
member_id event_type event_path event_time event_date event_duration
2523365 05.11.2016 05.11.2016 0:32 url youtube.com/...
0
votes
0answers
21 views
The omp_get_thread_num() is always returning 0 in UNIX
I am executing the code in UNIX. The following statement is always printing 0 in place of %i:
printf("thread %i under execution", omp_get_thread_num());
Initially I used to get correct result. But ...
0
votes
1answer
40 views
Why parallel for loop fails in python
I want to do a for loop parallel:
def multiple(a, b):
return a * b
Parallel(n_jobs=2)(delayed(multiple)(a=i, b=j) for i in range(1, 6) for j in range(11, 16))
But it gives me an error message:
...
0
votes
3answers
76 views
Java 8: How can I convert a for loop to run in parallel?
for (int i=0; i<100000; i++) {
// REST API request.
restTemplate.exchange(url, HttpMethod.GET, request, String.class);
}
I have a situation where I have to request a resource for 100k users ...
0
votes
0answers
24 views
Run asynchronous function in R
I have the following code:
myfun <- function() {
fun2()
return(1+1)
}
I want fun2() is invoked and then moved to the next lines without waiting for the result. Function fun2 does not return ...
-2
votes
0answers
12 views
Parallel Programming on R: Print function inside foreach loop does not run
I just try to do parallel programming on R. I modify one of my web scrapping script so it runs parallel but certain line "do not work".
Before the change, I put print function in a loop so I know ...
2
votes
1answer
52 views
Calculate all permutations of a collection in parallel
I need to calculate all permutations of a collection and i have a code for that but the problem is that it is linear and takes a lot of time.
public static <E> Set<Set<E>> ...
0
votes
1answer
18 views
MATLAB parallel loop with slight delays
I have a parallel loop in matlab shown below. Is there a way to make sure the loops are not running simultaneously but with a slight delay? I am open to use batch or spmd also but have limited ...
8
votes
1answer
418 views
Strange environment behavior in parallel plyr
Recently, I have created an object factor=1 in my workspace, not knowing that there is a function factor in the base package.
What I intended to do was to use the variable factor within a parallel ...
0
votes
3answers
35 views
execute batch files in parallel and get exit code from each
How can I execute set of batch files from single batch file in parallel and get the exit code from each. When I use start it executes the batch file in parallel (new cmd window) but don't return the ...
0
votes
1answer
16 views
Are data races intrinsic to imperative programming and an obstacle for parallel computing?
I don't study this area of computing to be honest. Actually my references is some web and academic articles then I'm insecure but curious about some concepts of parallel computing.
I've formulated ...
1
vote
0answers
16 views
Spark Direct Stream is not creating parallel streams per kafka partition
We are facing performance issue while integrating Spark-Kafka streams.
Project setup:
We are using Kafka topics with 3 partitions and producing 3000 messages in each partition and processing it in ...
5
votes
1answer
2k views
Initializing MPI cluster with snowfall R
I've been trying to run Rmpi and snowfall on my university's clusters but for some reason no matter how many compute nodes I get allocated, my snowfall initialization keeps running on only one node.
...
0
votes
0answers
6 views
Difference between GAS model and BSP model
I know Pregel's graph computation framework is based on BSP and GraphLab is based on GAS. What is the difference between BSP and GAS? and Advantage and disadvantage of each one?
-1
votes
0answers
28 views
parallel backends for Rstudio-Server [on hold]
I am trying to find a parallel backend for the foreach library to run on Rstudio-server.
I have 1 machine with 8 cores and 100 gb memory running Ubuntu, so I believe I am looking for multicore ...
0
votes
1answer
14 views
Nunit running tests in parallel not working
I added [assembly: Parallelizable(ParallelScope.Fixtures)] to my AssemblyInfo.cs.
now I'm running the test dotnet test and see in logs that test executed in normal order (fixture after fixture, in ...
1
vote
1answer
35 views
MPI_Scatter a 2D array in other 2D arrays
I want to scatter a 2D array in other 2D arrays (one for each process) using this specific way of allocating memory.
int (*matrix)[cols] = malloc(sizeof *matrix* rows);
I keep getting this error:
...
0
votes
1answer
30 views
Detect available and idle cores in R
I frequently use parallel::detectCores() in R, to get the number of CPU cores on a host, for parallelized computation. I would like to get a count of the number of available and idle cores for my ...
-1
votes
0answers
47 views
MPI with 1 thread is faster than 8 threads in my code
there is my code that I bring it below . in the first version, I don't use MPI send or receive . I just allocate some matrix then utilizing from them ...
the hardware which I use is Intel® Core™ i7-...
0
votes
0answers
6 views
receiver payment not showing in paypal account,paypal adaptive parallel payments [on hold]
Hi all and thanks for any help given, i have bought a magento marketplace ecommerce store from a well know supplier,i also purchsed there paypal adaptive magento add on which they have installed for ...
paypal
3
votes
2answers
108 views
Parallel::ForkManager makes subroutine 1000x slower
I have a subroutine, which I've serially optimized as much as I can, approximately like
sub overlap {
my $hash_reference = shift; # pass the hash to the subroutine
my %h = %{ $...
1
vote
1answer
16 views
Mpi_File_iwrite fails to write; aio_write() error: Invalid argument
if(my_rank==0)
{
sprintf(str,"<?xml version=\"1.0\" ?>\n");
MPI_File_iwrite(fp,str,strlen(str), MPI_CHAR,&request);
a=a+strlen(str);
sprintf(str,"<!DOCTYPE ...
0
votes
0answers
20 views
Parallel Mergesort in python 2.7 doesn't show result
I've tried to parallel mergsort and do this with 2 function:
one of them is: def merge(left, right) and another is :def mergesort(L)
I gave each function to one process but do doesn't show any things....
0
votes
1answer
20 views
IProgress<T> and Parallel.ForEach Sync Issues
I'm running into a sync issue involving reporting progress inside of a Parallel.ForEach. I recreated a simplified version of the problem in a Console App. The example actually only uses one item in ...
0
votes
3answers
46 views
Triggering Parallel.For in c# with sleep
I have a Parallel.For loop which I use to peform a lot of HTTP request at a certain point when a scheduled task occurs like this:
Parallel.For(0, doc.GetElementsByTagName("ItemID").Count, i => {
...
0
votes
0answers
12 views
Parallel jpeg compression algorithm
I am learning about parallel programming, and have a project about JPEG compression. But JPEG compress algorithm is hard with me. Can anyone give me a parallel algorithm about problem? Thanks!
0
votes
0answers
58 views
C# Split List<T> into groups using TPL Parallel ForEach
I need to process a List<T> of thousands of elements.
First I need to group the elements by year and type, so I obtain a List<List<T>>. Then for each internal List<T> I want ...
0
votes
0answers
35 views
data.table: parallel execution of row-wise function
I want to apply a function to some colums in every row of a data.table. I do this using something like this:
require(data.table)
## create some random data
n = 1000
p = 1000
set.seed(1)
data.raw <...
1
vote
1answer
26 views
Parallel gnu passing arguments in shell script
I would like to pass an argument to a c++ program in a shell script that just prints it to the console. However, I want to use gnu parallel to do it. At the moment I have a shell script,
#!/bin/sh
...
0
votes
1answer
14 views
why only MainThread reaches my function?
import threading
in this function I want to do my coding stuff for each thread but only mainthread enters this function none of thread which has been created in main function enters this function
...
0
votes
1answer
20 views
RBM no improvement with OpenACC on the code yet
RBM algorithm is open source algorithm
the source code is available here: https://github.com/yusugomori/DeepLearning/tree/master/cpp
I tried to get improvement with OpenACC by different ways but the ...
-1
votes
1answer
21 views
How to gather row and column partitioned matrix with MPI?
How would I go about gathering a partitioned MPI matrix?
Initially I have a matrix which I break down into several row and column-wise sub-matrices, used for the floyd warshall algorithm.
When I ...
6
votes
4answers
2k views
How to rename huge amount of files in Hadoop/Spark?
I have an input folder that contains +100,000 files.
I would like to do a batch operation on them, i.e. rename all of them in a certain way, or move them to a new path based on information in each ...
0
votes
1answer
42 views
Rcpp causes segfault RcppArmadillo does not
I'm currently trying to parallelize an existing hierarchical MCMC sampling scheme. The majority of my (by now sequential) source code is written in RcppArmadillo, so I'd like to stick with this ...
0
votes
1answer
41 views
How to listen port parallel in python + linux
everyone!
I have a python script, which process input from COM-port.
import sys
for string in sys.stdin:
some_calculation(string)
It runs like this:
cat -v /dev/pts/2 | python3 'process.py'
I ...
0
votes
0answers
24 views
ClusterMap in R on a forked cluster is much slower than mapply. What else can I try?
I am quite new to parallel programming, and have picked up bits about R from various (sometimes contradictory) sources.
I inherited some code and have tried to parallelise it, I think that data ...
0
votes
0answers
8 views
how to solve with Uilization dependant on latency regarding performance huththo
When latency is not so important and the overall utilization is more crucial,
throughput is a good measure of performance.
is this correct? why?
-2
votes
0answers
26 views
Bash + Python, GNU parallel [closed]
I have some input via COM-port.
while :
do
bash -c "cat -v /dev/pts/2 | python3 ppp.py"
done
where ppp.py is just:
import sys
for string in sys.stdin:
print(string)
I want to do the same, but ...
1
vote
4answers
705 views
Search image pattern
I need to do a program that does this: given an image (5*5 pixels), I have to search how many images like that exist in another image, composed by many other images. That is, i need to search a given ...
0
votes
0answers
43 views
Nested parallel for loop… “Parallel outer for loop” in “parallel inner for loop as a function”
What I want to run a function that contains a for loop (supposed to run in parallel) in a parallel outer loop. So it looks like the following:
void myfunction(){
...
#pragma omp parallel for
...
5
votes
2answers
3k views
Is MPI_Reduce blocking (or a natural barrier)?
I have the code snippet below in C++ which basically calculates the pi using classic monte carlo technic.
srand48((unsigned)time(0) + my_rank);
for(int i = 0 ; i < part_points; i++)
{
...
4
votes
3answers
77 views
Change for loop to Parallel.For loop
I can change my loop
for (int i = 0; i < something; i++)
to:
Parallel.For(0, something, i =>
But how to do this with this loop?:
for (i = 3; i <= something / 2; i = i + 2)
Thanks for ...
-1
votes
2answers
31 views
Distributed Computing and Parallel Programming C++ Framework For Network Clustered PCs [on hold]
I am attempting to create an application in C++ that uses parallel programming and distributed computing in C++. This would be my first parallel programming and distributed computing project. I was ...
0
votes
0answers
31 views
CUDA Nsight Error Message
I get this error message when I try to debug (run) my code using CUDA Nsight on Mac (XQuartz):
esk95@en-openmpi04:~$ /bin/sh -c "cd \"/home/esk95/Documents/parallel-programming/Spam-Filter/bin\";...
6
votes
2answers
2k views
Parallel.ForEach stalled when integrated with BlockingCollection
I adopted my implementation of parallel/consumer based on the code in this question
class ParallelConsumer<T> : IDisposable
{
private readonly int _maxParallel;
private readonly Action&...
