current community

your communities

more stack exchange communities

Stack Exchange
tour help
Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.
up vote 1 down vote favorite

I am coding a program which has persistent data (the MELT monitor, related to GCC MELT). The data is persistent because it is expected to be read and overwritten by most executions. (That data is in particular made of abstract syntax trees)

Currently, this persistent data is an Sqlite database. Of course, I am backing up it in textual format, obtained by sqlite dump, and I want to manage this textual dump with a version control system (probably git, but perhaps also subversion ....).

Unless I take special precautions, these SQL dumps will probably have quite long lines (e.g. several dozens of kilobytes). Wide SQL columns would probably contain JSON text of many kilobytes.

Would git (and svn) be more happy with shorter lines (in particular would they perform slower with long lines, or have repositories using much more disk space)?

I'm probably not mostly interested in the diff commands (e.g. git diff or svn diff), because I expect that using them for such textual dump is not very interesting for the human developers.

I am coding the persistency routines, so I am able to change slightly the format (e.g. to add newlines in some JSON text, which sqlite3 dump is dumping verbatim).

If you are curious, my current code snapshot is on http://starynkevitch.net/Basile/monimelt-bgc-21apr2014.tar.bz2 and contains a state-monimelt.sql dump file (which will of course become much larger, perhaps near or above a megabyte)

share|improve this question
1  
If you do not care about the human-readability or performing meaningful diffs, you can compress the text. It will it take less space and does not require newline substitutions across platforms, so it should perform better. –  Snowman Apr 21 at 19:41
    
I sometimes care about the human readability. The persistent data is somehow abstract syntax tree in a domain specific language.... –  Basile Starynkevitch Apr 21 at 19:44

1 Answer 1

up vote 3 down vote accepted

Svn basically stores the diffs between subsequent versions, so breaking the lines as to minimize the size of the diff should minimize the amount of storage you use. Git compresses its revisions somewhat differently, so breaking the lines may not matter as much. Whether it's significant in practice you can only tell by measuring. My recommendation is to try it manually both ways and see if it makes enough difference to bother automating. Just make sure to run a git gc to get the full effect of git's compression.

share|improve this answer
1  
Subversion does not use a line-based diff in its storage, it uses xdelta (a binary diff algorithm). It is not necessarily true that longer lines will result in larger xdeltas. –  John Bartholomew Apr 21 at 20:37
    
I did not know that, thank you. Still, that validates my admonition to test it first. –  Karl Bielefeldt Apr 21 at 20:39

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.