server: Keep polling sockets after we got a 0 recv, but not for POLLIN.