Use handle based xattr set if available.