isl_*_to_*_set: fix error handling