poll: avoid calling select with empty sets which hangs the process