Reset USB devices before turning them off or else presumably the PHY is still enabled...