tty: serial: altera_uart: Handle pdev->id == -1 in altera_uart_remove