server: Use a less hackish approach to not POLLIN when we have a hung up socket.