isl_bernstein.c: extract_coefficients: improve error handling