Multiprocessing is the use of two or more central processing units (CPUs) within a single computer system.
-1
votes
0answers
7 views
Python3, GUI + IO + BigDataSLowProcess, multiprocessing approach [closed]
Overview of my code:
I am studying multiprocessing, the code below consists of an IO process, a bigData (slow) process, and GUI runs in main line. I labeled them as "exe_"
exe_IO communicates with ...
5
votes
1answer
32 views
Fetch data from co-processor on network in background
My code communicates with a Raspberry Pi (with a static IP address) running on a network. The client sends a signal to the Pi, and the Pi sends back a double (encoded as a string) with the angle ...
4
votes
2answers
216 views
Multiprocess, multithreaded read write on a single file
I've got a job to create a library that would be able to support multithreaded and multi process read and write to a single file. On that single file we would store C# models in an array in JSON ...
2
votes
0answers
18 views
Adaptive scan of function concurrently
This function try to scan the given function in n dimensions in the given range by adaptively divide the area into n+1 points polygon (e.g. triangle in 2D; tetrahedron in 3D)
I tried to seperate the ...
1
vote
0answers
43 views
Optimal use of Python 3's multiprocessing package to create Sudoku grids
I want to create Sudoku grids using Python. As it's pretty slow if I want to create a bunch of grids, I decided to use multiprocessing.
My computer has 4 virtual ...
2
votes
2answers
153 views
Using doParallel to cycle through *.rds files
Overview
I have 165,000 *.rds files that are around 200,000 obs x 13 variables each. These files are unique to grid files (2.5 square mile data) so I need to keep them in their unique grid numbers. ...
1
vote
2answers
170 views
Blocking until condition is met
I have a class called _NodeProcess. It consists of a mainloop (main function) which is run in a separate process and methods ...
3
votes
2answers
39 views
Asynchronous model fitting that allows termination in Python
The problem
When you work with Python interactively (e.g. in an IPython shell or notebook) and run a computationally intensive operation like fitting a machine-learning model that is implemented in a ...
0
votes
0answers
111 views
Simple distributed lock manager server based on zmq
I wrote a simple server to distribute locks on a network. It is used in a multi-server environment with a shared home directory to secure non-multiprocessing safe file writing (...
6
votes
1answer
92 views
Multiprocess Bogosort
I only program by hobby, and I'm trying to improve with Python, so along with practicing coding I decided to venture into multiprocessing. This is a command line and will sort a variable number of ...
6
votes
2answers
293 views
Extracting keywords from 3 billion CSV records
I am processing around 3 billion records using this piece of code. It's pretty slow; it would be really helpful if you could suggest better ways to do this.
Update #1 Based on Nizam Mohammed ...
6
votes
1answer
1k views
Fast Thread- and multiprocess-safe file operations in python under Linux
I'm trying to implement an alternative to python built-in open(), but safe for use in multi-threaded and multiprocessing environment. I'm using advisory locking, ...
8
votes
2answers
710 views
Mandelbrot fractal with MPI
I'm not very familiar with MPI. I've written this little piece of code to draw the Mandelbrot fractal processing each row of the image in parallel. It works but it's really slow. I've written code to ...
0
votes
3answers
73 views
Optimize the process program
The program below is about finding the values below the percentage and splitting them into five different output files. This works great for small input data files. But when I am trying to run it for ...
5
votes
1answer
105 views
Catalogue manipulation
I have a catalogue of 56000 objects and I want to replace some columns with new values for instance replace NaN with -99 and ...
4
votes
1answer
118 views
Filewatcher compiler for watching different files
I'm trying to create a filewatcher program that can watch different files. The files might have subfiles that are being imported into the main file, which is being represented by the file object. I'm ...
7
votes
1answer
1k views
Portable C++ boost::interprocess::mutex
I was looking for long time around to have a portable robust solution for multiprocessing synchronization. Who touche this things know that good solution are boost::iterprocess named sync objects.
...
5
votes
1answer
170 views
Parallelizing scrypt key-derivation function
To review my use of multiprocessing, I don't think it is at all necessary to understand the algorithm, but it's the scrypt key-derivation function.
This uses ...
3
votes
1answer
203 views
Batch program to xcopy from host PC to remote destination with multi-processing
This is a batch program to xcopy from host PC to remote destination with multi-processing.
Kepler, Python-3.x are my environment.
...
