target/i386: fix fisttpl, fisttpll handling of out-of-range values