Remove empty directories when checking out a commit with fewer submodules