> if the last directory entry remains, the directory size won’t shrink
So if you keep appending data the directory will never be shrinked? If so i would say "the BSD world" is also affected by this problem, in practice. I'm assuming it all depends on the specifics of your filesystem, though.
If you keep appending the directory obviously won’t shrink because those entries need to fit somewhere - to shrink you’d need to remove stuff.
Let me rephrase, though. In UFS/FFS, the kernel will try to truncate the directory size every time you add a new entry. However, it will only cut off the free space at the end, it won’t punch a hole inside. This means that if you remove a bunch of files/subdirectories, but leave the one occupying the last (highest numbered) directory entry, that directory won’t shrink. It will shrink after you remove that last one.
Yeah that's what i mean, that for a sort of append-only cache where a garbage collector kicks in to remove old entries (<-- the part i forgot to mention), the directory size will in fact grow forever.
I must say that came as a surprise to me (i've used this pattern more than once, but never with millions of files), so thanks for telling me!
So if you keep appending data the directory will never be shrinked? If so i would say "the BSD world" is also affected by this problem, in practice. I'm assuming it all depends on the specifics of your filesystem, though.