math: Improve fmod