usb-serial: Fail instead of crash when chardev is missing