Only remove something that clashes with something we are really going to record ...