tevent_poll: Decouple poll_ev->fds handling from adding/removing fds