Remove the symlink, not the directory it points to.