Limit the number of allocated color pairs