The distributed-computing tag has no usage guidance.
1
vote
1answer
50 views
Distributing an application in 2 or more servers
This is probably a common question and I've been reading a lot about this, but I can't look into a solution.
Let's say I have an enterprise application in one glassfish server located in New York.
...
-2
votes
1answer
95 views
How do I implement autocomplete functionality for a big website? [closed]
I am looking to implement a custom autocomplete functionality for a travel website with huge amounts of data. I am thinking of implementing it using a trie. But how do I handle it if the data cannot ...
2
votes
0answers
78 views
How do I take data offline for a specific tenant, if their data is spread across multiple databases and multiple microservices?
I'm working on an application with about 50 or so microservices that have separate multi-tenanted databases. The problem arose of taking a tenant's data offline meaning that any requests for a certain ...
4
votes
1answer
109 views
Patterns for maintaining consistency in a distributed, event sourced system?
I've been reading on event sourcing lately and really like the ideas behind it but am stuck with the following problem.
Let's say you have N concurrent processes which receive commands (e.g. web ...
0
votes
0answers
73 views
How did “Stub” and “Skeleton” get their names?
In Java remote method invocation, the abstract method on the client-side is called "stub", which communicates with the "skeleton" on the server side and gets back the results. But why the names? For ...
5
votes
0answers
76 views
Distributed Calculation of Geometric Distance between vectors
I am looking at a large scale low latency way to calculate the geometric distance between vectors.
Let's say I have a vector A. It has size 128 and of type 32 bit float. I want to get it's geometric ...
2
votes
2answers
131 views
What does a load balancer return?
When a user hits the load balancer and the load balancer determines which web server to forward to, what happens next? Does the load balancer forward the request and all its data to the webserver, ...
-2
votes
1answer
97 views
Investigating in a highly distributed environment [closed]
I was asked this by an interviewer today and unfortunately he wouldn't share the kind of answer he was looking for so I'm hoping maybe the community can help.
Say you're an engineer at company X and ...
1
vote
0answers
21 views
Split long running work amongst multiple workers
I have a large number of rows in a database each containing a user specific token. For each token I need to maintain an open websocket connection. To scale this up I would like to run multiple ...
1
vote
2answers
188 views
Suggestions to distributed service pattern
We expose some webservices on a server, written in C#. Now we have to distribute the services to multiple servers, so the solution is distributed. And each request is dispatched to one the servers to ...
3
votes
1answer
101 views
Distributing work between TCP Clients
I have an application (run by a Windows Service) which connects to a TCP server (let's call the service 'Listener' from here on).
[It's important to note that the TCP server is out of my reach, and ...
2
votes
1answer
295 views
Scaling websocket client connections (not server) to multiple servers
I wrote a Slack bot which must connect to Slack teams through websocket connections. Since the bot might be used by thousands of team, I will eventually need to distribute the teams across multiple ...
2
votes
0answers
78 views
Error handling in distributed system
This is the common sequence of two distributed components in our Java application:
1 A sends request to B
2 B starts some job J in parallel thread
3 B returns response to A
4 A accepts ...
3
votes
1answer
153 views
Is TCP still considered siginificant overhead compared to UDP? [closed]
I'm working on a distributed application for a graduate class I'm taking. There has been much discussion around implementing a way to maintain peer connection health statuses. Think Gossip. One of the ...
4
votes
3answers
147 views
How to store (and find) updated values in a Distributed Hash Table?
Distributed Hash Tables are a great way to store immutable data in a resilient way.
However, I am wondering about how it would be possible to store new versions of old data in this system.
Because ...
4
votes
1answer
327 views
Distributed long-running jobs
Scenario
Various types of files containing customer's data are being uploaded via FTP, and a POST is being made to a REST API to update a database table, which tells the system (a web app) which of ...
3
votes
1answer
76 views
Coordinate access to resource between workers
I have a database accessed by an several servers, let's call them workers. These workers take one item from the database, do some operations on it and then update it back into the database. The ...
2
votes
1answer
131 views
OAuthv2 for distributed applications
Please note: Months ago I asked this question that received a truly amazing answer and introduction to OAuthv2. I’m now knee deep in the implementation of such a system and have similar (but notably ...
0
votes
1answer
92 views
Is this the correct way to think about the client/server socket relationship?
I'm designing a distributed application, very basic for now. The idea is that once the application is active on all machines (about 5 vms), you can initiate a grep on all log files from all machines ...
3
votes
3answers
115 views
What is an accepted method of timestamping in a distributed system?
In a normal application, a simple call to time.Now() in most languages will suffice for getting a timestamp, but what about in a web application spread over multiple servers? As I understand it time....
1
vote
0answers
96 views
Using a distributed system as an alternative to disk storage
So, one of the differences between "main memory" and disk storage is durability -- the data on hard disks will stay accessible and unaffected even if the host machine crashes or loses power. One can ...
2
votes
2answers
151 views
Starting a Windows task from Linux, what are my options? [closed]
We have server that runs data processing tasks, using Python, Django and Celery, on Linux. Once a batch of data comes in, a number of Celery tasks are scheduled and they run, some of them in parallel, ...
5
votes
2answers
85 views
How do distributed applications verify that the data was processed correctly?
There are a lot of projects out there that rely on the public sphere to process data for them, but I do wonder how they ensure that every computer participating has processed the data properly.
The ...
1
vote
0answers
55 views
JavaBeans vs. ActiveX: Strategic analysis
JavaBeans and ActiveX both serve the same basic function: to facilitate communication among software components within framework "containers." These containers include Web browsers and other document ...
3
votes
1answer
342 views
Single source of truth within an enterprise distributed system
Within an enterprise distributed system I have many services - an ecommerce service, a CRM, support desk, finance, billing. Many of these services share common data, such as Customer data.
These ...
4
votes
1answer
321 views
How to implement efficient heterogeneous microservice data queries?
Our team has an idea of implementing a simple declarative DSL that would let users query the enterprise's domain model via a single interface without caring which specific microservices to call to get ...
2
votes
2answers
68 views
Parallelising processing of users
Update I updated my question to reflect the fact I'm working with a database.
I need to process user actions:
The actions for each user change the user's balance which is then persisted to a ...
3
votes
1answer
78 views
Consistent snapshot
Suppose process A has a consitent snapshot of some entities. And it is constantly receiving updates for this enteties. Now client B connecting to A and it should receive the consistent snapshot of ...
1
vote
1answer
78 views
Is it possible to implement a robust 2 node P2P protocol? [closed]
I am thinking about how to implement a P2P protocol with only two nodes.
At first I read about different DHT implementations in the hope that this scenario would be feasible. Turns out most ...
1
vote
1answer
52 views
Sequencing distributed tasks with a distributed DB
Background
I have two separate processes, WriteIt() and ReadIt(). One creates records, and the other processes the records in a DB cluster.
Once WriteIt() creates a record, it queues a ReadIt() ...
1
vote
0answers
127 views
How is ORB distinct from an endpoint?
What, fundamentally, am I misunderstanding about how to use appclient?
The Application Client Container (ACC) includes a set of Java classes,
libraries, and other files that are required for ...
1
vote
1answer
151 views
Options for repeated distributed task
Scenario
My project has a need to periodically scan the database and notify administrators for particular issues. For example:
Any errors in the logs since last update?
Do values match expected ...
3
votes
3answers
971 views
have we come full circle with microservices, back to very old school approaches?
In terms of software architecture and design, how do microservices "stack up" (pun intended) against middleware? I'm coming from Java, and it seems like as you move away from straight REST as an API, ...
1
vote
1answer
74 views
Matrix distribution to process grid
I need to write a program, which will perform LU-decomposition, etc. The problem is that I don't know about the preferred way to distribute the loaded matrix from the root process to other processes. ...
0
votes
1answer
93 views
Difference between server Jobs and Workers?
I am trying to understand the difference between server workers and server jobs. Google has not been very helpful because the keyword "jobs" is being construed as an employment term... I have yet to ...
2
votes
1answer
346 views
What is eventual consistency?
I've heard people talking about "eventual consistency" (for example in this question and this answer).
What is "eventual consistency"? How does it compare to "transactional consistency"? When does it ...
3
votes
2answers
195 views
Distributing a set of “Jobs” across multiple computers
I'm rather new to distributed computing and would like some assistance with the overall architecture of my application.
My application has Jobs that can be added to a JobQueue. Then one or more ...
1
vote
2answers
202 views
How to minimize networking cost of distributed game database?
We've been working on a networked online game project which will run in near future and
looking for a satisfying solution for our distributed game database. We didn't take networking costs
into ...
1
vote
2answers
208 views
How to deploy software automatically over SSH?
I am dealing with a distributed application which is going to mange a network of Linux-based servers spread among different places. Having the SSH credentials (as the most supported connection ...
1
vote
0answers
43 views
how to combine logs coming from multiple virtual machines for processing?
Scenario:
Two virtual machines produce logs. I want these logs to be sent to a central server for processing (as soon as a line of these logs is generated-->streaming ). The condition here is that I ...
3
votes
1answer
249 views
Accepting the UUID collision risk based on number of clients
After reading some questions about the probability of UUID collisions it seems like collisions although unlikely, are still possible and a conflict solution is still needed. Therefore I am wondering ...
4
votes
2answers
1k views
How does Elastic Search approach the problem of distributed membership and consensus?
I already understand that Elastic Search is supposed to be deployed in a distributed topology, in that you can have multiple nodes for a cluster of ES instances.
I like the API, and it looks ...
1
vote
1answer
349 views
Observing progress of a distributed system
For a distributed system, there is a requirement of observing the progress of
smaller applications on distributed computers (runtime 5 - 20 minutes).
There is a web fronted, which right now only ...
5
votes
1answer
2k views
Persistent storage for Apache Mesos
Recently I've discovered such a thing as a Apache Mesos.
It all looks amazingly in all that demos and examples. I could easily imagine how one would run for stateless jobs - that fits to the whole ...
0
votes
1answer
202 views
Interconnect nodes in a Java distributed infrastructure for tweet processing
I'm working in a new version of an old project that I used to download and process user statuses from Twitter. The main problem of that project was its infrastructure. I used multiple instances of a ...
2
votes
0answers
151 views
Managing setTimeouts and setInterval in a distributed environment
I am building a turnbased game on nodejs using socket.io and redis as the datastore. I am planning on hosting the game on AWS opsworks with the ability to scale by adding more nodes.
How should I ...
2
votes
1answer
107 views
Message Passing: How to detect and/or handle “dead ends”
I'm playing with a mental model for distributed actor system that communicates using messages. In this, it is possible for an actor to create another actor. I've encountered one specific situation ...
4
votes
1answer
456 views
Use a custom value object or a Guid as an entity identifier in a distributed system?
tl;dr
I've been told that in domain-driven design, an identifier for an entity could be a custom value object, i.e. something other than Guid, string, int, etc. Can this really be advisable in a ...
1
vote
3answers
530 views
what are the things to take in consideration when designing a database that likely will be distributed after some time? [closed]
We are small start-up now, we don't have a DBA.
if our project succeed, it will have a lot of data that will likely need to be distributed over several machines.
So we want to make that step easier, ...
1
vote
1answer
300 views
Building a serverless p2p CMS/System
I'm thinking about building an Open Source, serverless, offline-replicated p2p CMS, but I'm concerned about it really working in a real environment. How would you go about doing that, reducing the ...
