fix signed range problems with hex conversions