Manually track the source decoder sample pos