Decrement context count when it's removed from the list