All Questions
Tagged with message-queue distributed-computing
8 questions
0
votes
1
answer
150
views
How to rebalance data across nodes?
I am implementing a message queue where messages are distributed across nodes in a cluster. The goal is to design a system to be able to auto-scale without needing to keep a global map of each message ...
- 115
-1
votes
3
answers
579
views
Avoid race condition using RDBMS transaction
I am working on design of a distributed system which will process some events. For simplicity lets say, multiple instance of same service will be consuming from same queue, every message will have a ...
3
votes
1
answer
1k
views
Long-running compute-intensive tasks in APIs: background workers?
I'm working on a design for an HTTP based API that takes in requests to perform a long-running task that requires CPU and RAM intensive processing.
To give an impression of the compute requirements, ...
- 139
3
votes
2
answers
223
views
Optimistic locking vs separate lock store for background workers
I need to compute the value of a row R on a table based on rows queried on another table B.
I need to do it in a queue worker because the computation of the value can be slow. A message is queued ...
- 39
0
votes
0
answers
52
views
Synchronized computation task distribution
I have a system where a set of Operations needs to be done every day. Each Operation has the following metadata: Type, Entity and a Date (independent of current date). It reads something from the ...
- 147
2
votes
1
answer
138
views
Sequencing distributed tasks with a distributed DB
Background
I have two separate processes, WriteIt() and ReadIt(). One creates records, and the other processes the records in a DB cluster.
Once WriteIt() creates a record, it queues a ReadIt() task ...
- 123
1
vote
1
answer
551
views
Options for repeated distributed task
Scenario
My project has a need to periodically scan the database and notify administrators for particular issues. For example:
Any errors in the logs since last update?
Do values match expected value ...
- 113
22
votes
1
answer
14k
views
What are the solutions to the Distributed Queue problem?
I am trying to learn more about the various ways that the problem of a Distributed Queue may be solved. So I would like know what products, services, implementations and research papers that are ...
- 371