current community

your communities

more stack exchange communities

Stack Exchange
tour help
Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.
up vote 5 down vote favorite
2

Suppose I have a sparse file F on a Linux ext4 filesystem, and process P1 is writing to a disjoint 50% subset of F while P2 writes to the other 50% of F. I would like to minimize fragmentation while the file "grows". (I put "grows" in quotes because the file is pre-allocated as a sparse file, but as the blocks get written they fill in phantom blocks with actual data.)

I realize since P1 and P2 are running in parallel that one may get ahead of the other, but barring this, is it best to have P1 write blocks 1,3,5,7,... while P2 writes 2,4,6,...? Or better to have P1 write 1,2,3,4,...n/2 and P2 write n/2+1, ...., n?

share|improve this question
2  
If file fragmentation matters (i.e. it causes performance problems for you), then you should pre-allocate it. Really. Otherwise, it's going to depend on the size of the blocks your application writes compared to the filesystem's block size, and how much free space you have - if low on space/fragmented partition, it won't matter much. –  Mat Oct 21 '11 at 5:33
    
@Mat I'm worried that fallocate (or posix_fallocate) will take too long on some OSes, but I'll look into this; cheers. –  Fixee Oct 21 '11 at 18:04

1 Answer 1

up vote 1 down vote accepted

The kernel caches the writes and lazily flushes them to disk in the background, allocating disk space as it does so in such a way that it minimizes fragmentation. In other words, you're over thinking things -- don't worry about it.

More specifically when it does to to flush some dirty cache buffers, ext4 goes to allocate enough disk space to hold all of the dirty buffers in the cache, as well as reserving additional space for further growth.

The load you are describing sounds a lot like bit torrent. I recently downloaded the Ubuntu 11.10 iso via bit torrent and checking it with filefrag shows that it is only broken into 3 fragments, which is not bad at all for a 700mb file.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.