refs: do not invalidate the packed-refs cache unnecessarily