I originally posted this question on StackOverflow but was told to move it here.
I have an application that I have been developing for the last two years, and over the course of time it has become increasingly complex. At it’s core is a tree structure where each node corresponds to a level in my application with certain properties. At the leaf nodes there are more properties than in the intermediate nodes. When I started developing this app I made the tree structure as an object with a list of children objects of same type, with the possibility of extending the leaf nodes.
Gradually more and more features and thus properties have been implemented, and I have found that creating objects for all my features and properties is taking quite a bit memory. From the beginning I did not consider using a database, and such the application is in no way structured to use a database. The application reads data from files, and can thus vary in memory usage according to the size of the file loaded. For a small file loaded the app uses roughly 300K objects and has allocated about 16 MB. For larger files it can use upwards of 500K - 600K objects and allocate all it’s availabe memory.
My problem is that I am going to extend the application further in the future, while the files being loaded becomes bigger and bigger. What is the way to go? Should I completely restructure the app to use databases and will it be quick? Should I continue using objects and just assume that devices are becoming increasingly potent and can handle all my objects? Can I do some sort of crossover, keeping my tree structure and basic objects and moving other stuff to databases?
