All Questions
Tagged with message-queue python
6 questions
3
votes
1
answer
1k
views
Long-running compute-intensive tasks in APIs: background workers?
I'm working on a design for an HTTP based API that takes in requests to perform a long-running task that requires CPU and RAM intensive processing.
To give an impression of the compute requirements, ...
- 139
0
votes
2
answers
2k
views
How can I create a Python microservice on AWS that both accepts REST connections and processes SQS messages?
Background: I'm working on a new project at work that will run in AWS. We're trying to use a modern microservice architecture and take advantage of cloud technology, but we don't have much experience ...
- 3
1
vote
1
answer
399
views
Persist data downloaded by Celery workers
I'm working in a tool that downloads tweets from Twitter to process them later. For this purpose I'm using Celery with RabbitMQ, sending task with the keywords that must be tracked by the workers.
My ...
1
vote
0
answers
89
views
Improve communication between controller and trackers in a Twitter fetcher tool using RabbitMQ or Apache Flume
I've been working for a time with some researches developing a tool to fetch tweets from Twitter and process them in some way. The first prototype "worked" but became a pain as we used sockets to ...
3
votes
1
answer
2k
views
Is ZeroMQ a good choice to make a Python app and a C# managed assembly work together?
I have a task that involves talking to a .NET-based API (namely AutoCAD) to retrieve data, send commands, and react to events.
I want to separate the API operations and the proper program logic (...
- 133
4
votes
4
answers
1k
views
Architecting Python application consisting of many small scripts
I am building an application which, at the moment, consists of many small Python scripts.
Each Python script processes items from one Amazon SQS queue. Emails come into an initial queue and are ...
- 289