Fix issue where togglerefs might be incorrectly pinned. (#12815)