Don't refill the buffer queue after detecting a stopped stream