free_ref_entry(): do not trigger reading of loose refs