Tagged Questions

3
votes
1answer
288 views

A semaphore-like mechanism for Celery

We're developing a distributed application in Python + Celery for our task queue. Our application requires us to download emails from a remote ISP via IMAP (e.g.: gmail) and we're looking to have be ...
1
vote
1answer
132 views

Updating shared read-only data with Python multiprocessing

I am attempting to use Python's multiprocessing library to experiment with distributed neural networks. At the moment, I have it set up so that a server process creates the neural network and chunks ...
1
vote
1answer
456 views

Data analysis using MapReduce in MongoDb vs a Distributed Queue using Celery & RabbitMq

I am currently working on a project which involves performing a lot of statistical calculations on many relatively small datasets. Some of these calculations are as simple as computing a moving ...
0
votes
1answer
108 views

“Global array” parallel programming on distributed memory clusters with python

I am looking for a python library which extends the functionality of numpy to operations on a distributed memory cluster: i.e. "a parallel programming model in which the programmer views an array as a ...
0
votes
1answer
122 views

Monitor python scrapper programs on multiple Amazon EC2 servers with a single web interface written in Django

I have a web-scrapper (command-line scripts) written in Python that run on 4-5 Amazon-EC2 instances. What i do is place the copy of these python scripts in these EC2 servers and run them. So the ...
0
votes
1answer
83 views

distributed-computing library in the standard library of python

I want to implement a program that will be used by a lot of co-workers, the problem is that I will use sqlserver and pyodbc in my program but in the same time I must not oblige the others who want to ...
0
votes
0answers
8 views

Script for feeding data into Yahoo S4 (Now under Apache)

I know that we have to feed data stream into s4. My question is that the script generating the input stream (that has to be feeded into s4) can be in python or it needs to be in Java only, since S4 ...
0
votes
0answers
22 views

will separating the implementation of game servers rendering and networking code improve performance?

I currently have a game server doing to following - all in c++ code serially - 1)collect every clients interaction, then 2)calculate total effect on system (server has a physics engine ...
0
votes
0answers
58 views

Running many subprocesses, each executing an external program and all share a list of id to process

I have a list of seeds and I want to launch many instances of an external program, each with a seed as input argument. The list is quiet long (~10000) but I want to launch up to ~50 instances in ...
0
votes
0answers
40 views

Appengine Query Parent and Level

The next model is an example: class Category(db.Model): name = db.StringProperty() description = db.TextProperty() level = db.IntegerProperty() And I can have a Parent and one level: def ...