serial: mfd: fix bug in serial_hsu_remove()