rpc: Always supply both the remote and local address to the auth subsystem