Polyhedron_Factor: make sure group information gets updated