demux: ogg: only invalid pts on no granule interpolation