Make sure PulseAudio sets an update size that's a multiple of 4 samples with SSE