Properly update the delay line offsets when fading is done