Tagged Questions
1
vote
0answers
34 views
Python: subprocess memory leak
I want to run a serial program on multiple cores at the same time and I need to do that multiple time (in a loop).
I use subprocess.Popen to distribute the jobs on the processors by limiting the ...
0
votes
0answers
14 views
Can install and run MATLAB Distributed Computing Server on a Desktop version of linux?
I installed MATLAB Distributed Computing Server on a Desktop version of linux (Mint) but when i run the command ./lmstart to start the licence manager it does not start. It also does not fill the log ...
0
votes
1answer
12 views
How to correctly instance a IPython cluster made of local and remote machines
I'm facing big problems in instancing a cluster for IPython. I think the doc is pretty complicated.
My system is made of the following things:
Local computer, Ubuntu 12.04 x86_64, IPython 2.1, 8 ...
-4
votes
0answers
45 views
how to use virtualCL to execute a simple matrix multiplication on multiple sytem.?
I have more than 2 computer as same config (intel i7 with 1GB graphics AMD card) and I want to run matrix multiplication of 1M size using distributed computing. I need full details stating with ...
0
votes
1answer
33 views
Creating a progress screen for MATLAB distributed computing applications
I'm running a highly-parallelized MATLAB application on a cluster. I'm going to build a wait screen for the user and I'd like to display the progress of the cluster job. Here's some context:
...
1
vote
3answers
59 views
cluster r script isn't reading in RData datasets properly
I'm trying to run this job on our cluster, and I keep getting this "object of type 'closure' is not subsettable" error. It basically runs this function "do_1()" on a bunch of nodes. The closure object ...
0
votes
2answers
97 views
OpenMPI: Simple 2-Node Setup
I'm having trouble running an OpenMPI program using only two nodes (one of the nodes is the same machine that is executing the mpiexec command and the other node is a separate machine).
I'll call the ...
0
votes
0answers
24 views
Parallelizing Matlab codes on a cluster
I have a Matlab code and I'm trying to change the parameters in it and run it on a cluster. Each set of parameters will generate two .csv files. The parameters are alpha and T in the code. I want to ...
1
vote
1answer
76 views
Running doRedis- Object not found even when it's been exported
I'm testing the doRedis package by running a worker one machine and the master/server on another. The code on my master looks like this:
#Register ...
r <- foreach(a=1:numreps, .export(...)) ...
2
votes
2answers
90 views
Parallel Processing in R using “parallel” package
I have two data frames:
> head(k)
V1
1 1814338070
2 1199215279
3 1283239083
4 1201972527
5 404900682
6 3093614019
> head(g)
start end state value
1 16777216 16777471 ...
0
votes
0answers
30 views
sun grid engine synchronization point
I am creating a set of tasks in a cluster using the -qsub command.
Inside my matlab code I would like to make a synchronization point to check whether all of my workers are finished execution or not. ...
0
votes
0answers
36 views
View GNUPLOT figures from cluster backend
I'm currently trying to run some quick diagnostic tests on some parallel code. I'm submitting the code through a batch system to the cluster backend through .pbs scripts. I'm capturing the diagnostic ...
0
votes
1answer
39 views
A set of cluster python workers doing scheduled tasks
I would like to build a set of cluster workers (ie droplets in digitalocean or so).
Each worker would be doing a periodic task and sending the results periodically to the main application.
Here is ...
0
votes
1answer
40 views
Parallel vs Distributed computing----the dividing line
I am studying the differences b/w parallel and distributed systems. I have been told that the division is blurring. Also, clusters can be viewed both as parallel and distributed systems (depending on ...
0
votes
1answer
61 views
R - makeCluster hangs on localhost in Linux
As the title says, makeCluster and makeSOCKcluster from the snow package hang when running on a linux machine..
cl <- makeCluster(4, type="SOCK)
cl <- ...
1
vote
1answer
216 views
Running parallel Jobs
I am working on titan cluster which is consisting of 464 HP blade systems, two head nodes and a virtualized pool of login (submit) nodes.Each node has eight cores (two quad-core processors), and ...
0
votes
1answer
67 views
What is a farm, as opposed to a grid or a cluster?
I'm trying to understand does "farm" mean in computing, so that, if a farm can be exemplified as a cluster, there must be something that a farm is independently of what a cluster is.
How is it ...
0
votes
0answers
42 views
Virtual Machines and OpenMP
I'm new to parallel computing and I'm trying to improve performance on a computing system. I'm interested in calculating the rise and set times for 100,000 stars from an observatory everyday for some ...
0
votes
3answers
104 views
What's the best way for me spread my programs execution across multiple servers? [closed]
I have a program which runs for a long time, up to a few of weeks on a single server. I have split this to run in fairy large parallel chunks within the code. I've been using OpenMP to do this. The ...
0
votes
0answers
67 views
how to make a fork cluster
Sorry for asking trivial questions. I am using 'makeForkCluster' function in Parallel package to create clusters on Linux. But there isn't any response (no error, no warning and success). I would ...
7
votes
1answer
190 views
How to improve speed in parallel cluster processing
I'm new to cluster processing, and could use some advice as to how better to prepare data and/or the calls to functions from the parallel package. I have read thru the parallels package vignettes, so ...
0
votes
1answer
170 views
Running openmp on cluster
I have to run an openmp program on a cluster with different configuration (such as different number of nodes).
But the problem I am facing is that whenever I am trying to run the program with say 2 ...
0
votes
0answers
67 views
MPI_Scatter taking foever to complete
I am working on parallel LU decomposition using MPI, where I send bunch of contiguous rows from original matrix and retrieve them later after some computation. Scatter and Gather was working fine but ...
1
vote
1answer
79 views
Advice needed! Cluster programs for Java?
I am looking for a cluster program which I can compute Java job in parallel. I looked at Rockscluster and Hadoop. The problem of using Rockscluster was that it required scripts in the Unix to run ...
1
vote
1answer
185 views
Registration timeout on PBS cluster with Ipython parallel
I'm trying to set up ipython parallel on a linux cluster using PBS scheduling.
I was following instructions at http://www.andreazonca.com/2013/04/ipython-parallell-setup-on-carver-at.html (the ...
0
votes
1answer
568 views
How to send a variable of type struct in MPI_Send()?
basically i have coded a program in C using MPI where in the struct variable is to be sent in a ring fashion to the processes and based on the value received from that variable the work for that ...
0
votes
0answers
204 views
matlab parallel processing on several nodes
I have studied pages and discussion on matlab processing, but I still don't know how to distribute my program over several nodes(not cores). In the cluster which I am using, there are 10 nodes ...
1
vote
1answer
693 views
R: making cluster in doParallel / snowfall hangs
I've got two servers on a LAN with fresh installs of Centos 6.4 minimal and R 3.0.1. Both computers have doParallel, snow, and snowfall packages installed.
The servers can ssh to each other fine.
...
1
vote
1answer
435 views
Initializing MPI cluster with snowfall R
I've been trying to run Rmpi and snowfall on my university's clusters but for some reason no matter how many compute nodes I get allocated, my snowfall initialization keeps running on only one node.
...
3
votes
1answer
339 views
Parallel R on a Windows cluster
I've got a Windows HPC Server running with some nodes in the backend. I would like to run Parallel R using multiple nodes from the backend. I think Parallel R might be using SNOW on Windows, but not ...
1
vote
0answers
387 views
Octave Parallel Code Example
Can anyone provide an example octave code to submit to the cluster assuming the parallel package for octave is installed in the cluster? What I am saying, do I just use parfor as in Matlab parallel ...
2
votes
1answer
228 views
How to set up doSNOW and SOCK cluster with Torque/MOAB scheduler?
In continuation of this question (http://stackoverflow.com/questions/17222942/allow-foreach-workers-to-register-and-distribute-sub-tasks-to-other-workers), what is a best practice to connect doSNOW ...
3
votes
1answer
212 views
Allow foreach workers to register and distribute sub-tasks to other workers
I have an R code that involves several foreach workers to perform some tasks in parallel. I am using foreach and doMC for this purpose. I want to let each of the foreach workers recruits some new ...
0
votes
0answers
233 views
Using MPI with parallel HDF5
I am new to MPI and I try to run it with parallel HDF5 . So I setup a cluster of 2 nodes with GFS2 and DRBD and in my shared folder I compiled an example provided by the HDF group : ...
0
votes
1answer
2k views
Weak vs Strong Scaling Speedup and Efficiency
I have a theoretical question. As you know, for the analysis of scaling, the speedup is defined as S(N) = T(1) / T(N) where T(i) is the runtime with i processors. The efficiency is then defined as ...
0
votes
1answer
64 views
How to send parallel jobs in pbs_torque and add the output?
I'm using pov-ray (Rendering tool) in Linux. I've to parallelize my code so I can use 10 computers at the same time for rendering, I've to use torque_pbs to accomplish this task. I have an idea how to ...
0
votes
1answer
194 views
Hadoop cluster for non-MapReduce algorithms in parallel
The Apache Hadoop is inspired by the Google MapReduce paper. The flow of MapReduce can be considered as two set of SIMDs (single instruction multiple data), one for Mappers, another for Reducers. ...
0
votes
0answers
125 views
Installing additional threaded version of Tcl/Tk and run code on certain nodes in a cluster
I need to paralleling a code written in tcl/tk language,in a cluster have 12 nodes my IP is registered to access to 4 nodes, every node have 4 processors with 2 cores each. the OS is linux centos 64 ...
8
votes
3answers
6k views
Create a cluster of co-workers' Windows 7 PCs for parallel processing in R?
I am running the termstrc yield curve analysis package in R across 10 years of daily bond price data for 5 different countries. This is highly compute intensive, it takes 3200 seconds per country on a ...
0
votes
1answer
286 views
Connecting 2 laptops for MPI
I am new to MPI but after playing around with few sample MPI codes(in c) I got some familiarity with it. But when I tried to connect two laptops(via college LAN)
I am unable to get the things done.
I ...
2
votes
1answer
211 views
Generate random number within a range (0-100k) in a cluster environment
I have to generate random number within a range (0-100.000) in a cluster environment (many stateless Java based app servers + Mongodb) - so every user request will get some unique number and will ...
mongodb
0
votes
1answer
180 views
PBS running multiple instances of the same program with different arguments
How do you go about running the same program multiple times but with different arguments each instance on a cluster, submitted through a PBS. Also, is it possible to designate each of these programs ...
0
votes
1answer
159 views
Are all cluster computing libraries compatible with starcluster?
I have a highly threaded application running on amazon EC2. I would to convert this application to a cluster on EC2. I would like to use starcluster for this as its easy to manage the cluster.
...
0
votes
1answer
80 views
Finding the amount of execution time spent on each processor in a Beowulf cluster
I have downloaded an LU Decomposition program from the following link http://www.cs.nyu.edu/wanghua/course...el/h3/mpi_lu.c and the programming is running very well...The reason for me writing this ...
1
vote
0answers
304 views
R code parallelized using plyr and doMC: error message: Error in do.ply(i) : task 1 failed - “could not find function ”getClass“”
I'm just getting started learning how to use remote supercomputers for execution of parallelized code. I got a lot of initial help from this previous post, as well as one particularly helpful and ...
1
vote
1answer
196 views
Handling various resource requirements in an IPython cluster
I am using a PBS-based cluster and running IPython parallel over a set of nodes, each with either 24 or 32 cores and memory ranging from 24G to 72G; this heterogeneity is due to our cluster having ...
2
votes
2answers
550 views
GNU parallel load balancing
I am trying to find a way to execute CPU intensive parallel jobs over a cluster. My objective is to schedule one job per core, so that every job hopefully gets 100% CPU utilization once scheduled. ...
0
votes
3answers
357 views
How to increase parallel-processing speed over compute-nodes on linux cluster?
Below is the code relevant to the question I am asking. I am working with a cluster that has 13 compute-nodes and 1 top node (a.k.a. main node). The top node is the node that connects all of the users ...
3
votes
4answers
883 views
Possible to use OpenCL on multi-computers?
As far as I know, the answer is no. OpenCL is designed for multi-cores system.
But, is there any way to use OpenCL on multi-computers ( each computer is a multi-cores system ) ? If not, are any ...
3
votes
3answers
2k views
parallel python, or MPI?
I have a code with heavy symbolic calculations (many multiple symbolic integrals). Also I have access to both an 8-core cpu computer (with 18 GB RAM) and a small 32 cpu cluster. I prefer to remain on ...
