USB: ti_usb: fix module removal