High-Performance Concurrent Model-View Rendering Architecture [duplicate]

Question

I have a single-threaded file indexing/scanning engine in D I want to extend with a view thread that visualizes the scanned content as a tree (or later force-directed graph). I want this solution to be modern in terms of concurrent performance (high performance with no view thread lockups).

I plan on sending the data as a logical description of a file system tree as a stream of tuples, where each tuple contains a path, file-type and optional-dir-content names. This stream is "sent" by the data thread over some queue-like mechanism to the view thread that does the layout and opengl rendering of this model data.

What kind of communication should we use for this in terms of

• blockingness (lock-free)
• data-coarseness (should I buffer up as much as possible and send it all in a block synched with framerate or as a stream of smaller packages in a lock-free queue)

What are the typical wait-times for mutexes-locks nowadays?

Answer 1

Mutexes are pretty fast (probably faster by now): http://axisofeval.blogspot.sk/2010/11/numbers-everybody-should-know.html

The approach I use is to 'double buffer' all data that changes between frames. I.e. there are 2 copies of all data. At any given moment, 'back'/'future' copy is being updated from 'past' and modified by new input, while 'front'/'past' copy is const can be 'viewed' - read by other threads. Between frames these copies are swapped.

This requires very little locking (only between the frames), if you don't mind the memory overhead.