I have a piece of code written in node and I wanted to know if I am in the right direction. Basically I have a site where the homepage lists 6 vehicle cards. Every time the page is requested I do the following:
- Take 16 vehicles off neo4j (skip x amount if not first page)
- For each of the 16th vehicle objects, check if there exist a directory, if yes then read its content (i.e. vehicle images, max 5 per directory)
- Return jade template with all data to client
Initially I was using fs.readdirSync
and I have just updated the code to use fs.readdir
async version.
My worry is how would the following piece of code that read directory and lists its images including finding out its dimension perform in real life for hundreds of concurrent requests?
vehicles.forEach(function(vehicle) {
var images = [];
fs.readdir(__dirname + '/photos/' + v.vehicle_id, function(err, files) {
if(err && err.code == 'ENOENT') {
logger.info('No images found for vehicle.');
modifiedVehicles.push(v);
} else {
files.forEach(function(file) {
if(file.indexOf("thumbnail") != 0) {
var dimension = sizeOf(__dirname + '/photos/' + v.vehicle_id +'/' + file);
var temp = {
filename: file,
width: dimension.width,
height: dimension.height
};
images.push(temp);
}
});
v.images = images;
modifiedVehicles.push(v);
}
});
});
callback(modifiedVehicles);
As a single request to a page of 16 vehicle cards with only 8 of those 16 vehicles having at least 1 image the time it took is:
benchmark took 5135013 nanoseconds (5.1 milliseconds)
Assuming that I host with VM that runs SSD, is the above the right solution? I am thinking of the following options:
Method 1
I could return the response to the user request without list of images but, then as soon as the page is loaded I send a subsequent req to retrieve list of images for all vehicles
Method 2
Retrieve list of images only when the vehicle card is viewed. Ajax request.
Method 3
When the app is started, for each of most recent vehicle (latest 500) read its images and insert them as key (vehicle_id) and value (array of images) to Redis and then retrieve values from Redis for each http request. Not sure how much of a difference Redis instead of Disk I/O can be.
Method 4
Store filename of of vehicles images in mongodb when the vehicle is created/updated. For each http request query mongodb for all images with vehicle ids IN (...)
Any other method that you think I should consider? please share.