Tagged Questions
6
votes
0answers
74 views
Parallel application in python becomes much slower when using mpi rather than multiprocessing module
Lately I've observed a weird effect when I measured performance of my parallel application using the multiprocessing module and mpi4py as communication tools.
The application performs evolutionary ...
0
votes
2answers
28 views
Was the python script launched by mpirun or mpiexec?
I've coded a python script that can be either launched in stand alone way or with mpi support.
python myscript.py
vs
mpirun -np 2 python myscript.py
How can I know inside the script in which way ...
0
votes
1answer
40 views
Custom boost python module with built in mpi?
Is it possible, in principle, to use the boost python library to create a custom python module in C++ that has MPI functionality? So I wonder if I can include MPI functionality in some of the ...
0
votes
0answers
23 views
2d temps array seems to be doing nothing in my code
I'm trying to teach myself mpi and so I've been writing some code and taking bits and pieces off stackoverflow. I found this code here and made a few changes, however no matter what I change the temps ...
0
votes
0answers
35 views
System calls using Popen in a for loop fail after the first iteration fine in a paralel environment using mpi4py
A simple script below will show that Popen in the simplest case of just sending "ECHO" in a loop only works in the first iteration (run on IBM iDataplex x86 system with RHEL 5). The first iteration it ...
0
votes
1answer
60 views
mpi4py returning rank differences
I want to launch parallel processes from a python script (and, for testing, interactively, but not from ipython), across two different versions of python, and have started out with mpi4py. The two ...
0
votes
1answer
54 views
Reduce a list with operator as summing each element in mpi4py
I am writing a mpi python code. For example, four procs have data below:
data on procs0: [1, 2, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0]
data on procs1: [0, 0, 0, 4, 5, 6, 0, 0, 0, 0, 0, 0]
data on procs2: [0, ...
0
votes
0answers
76 views
parallelize python code + queue submission
I have a Python code containing a for loop that I would like to parallelize, and run on a cluster via a queue.
Here is a copy of the code mycode.py:
import struct
import numpy
from array import array
...
1
vote
2answers
102 views
boost mpi python: undefined symbol: _ZN5boost3mpi5timer14time_is_globalEv [closed]
I inherited a large python system that uses the boost mpi for python.
So I've built boost 1.52 from source, including both the mpi and python components.
And I've built the boost mpi for python ...
0
votes
1answer
72 views
Tips on writing optimised project in Python with C/C++ extensions? [closed]
I have a project written in IDL (interactive data language) that is used to produce a near real time model of the ionosphere by assimilating a heap of different data inputs. IDL is not a great ...
1
vote
1answer
122 views
How to share data between processes in Python using MPI?
I'm attempting to parallelize a script I wrote. Each process needs to do a calculation and store the data to a specific part of an array (list of lists). Each process is calculating and storing its ...
0
votes
0answers
109 views
MPI4Py: Method returns from all processes
I am working on writing a set of parallelized versions of map(), reduce(), and a few other handy operations. The code below returns the correct value once, and then None for the remaining processes. ...
2
votes
1answer
120 views
Bitonic sort, mpi4py
I am attempting to implement the Bitonic-Sort algorithm.
Parallel Bitonic Sort Algorithm for processor Pk (for k := 0 : : : P 1)
d:= log P /* cube dimension */
sort(local datak) /* sequential sort ...
2
votes
3answers
669 views
parallel python, or MPI?
I have a code with heavy symbolic calculations (many multiple symbolic integrals). Also I have access to both an 8-core cpu computer (with 18 GB RAM) and a small 32 cpu cluster. I prefer to remain on ...
1
vote
0answers
108 views
Gathering an array of different sizes on various processes into one array with MPI and mpi4py in Python
I have matrix's on a bunch of different processes but they are all not necessarily the same size so I don't think I can use comm.gather() or equivalents. Therefore, I have written my own function but ...
