lt2p: Fix possible WARN_ON from socket code when UDP socket is closed