| blob | 991bc05b666769ac3187c21f36674769b9bfcbae |
1 Filename: 141-jit-sd-downloads.txt
2 Title: Download server descriptors on demand
3 Version: $Revision$
4 Last-Modified: $Date$
5 Author: Peter Palfrader
6 Created: 15-Jun-2008
7 Status: Draft
9 1. Overview
11 Downloading all server descriptors is the most expensive part
12 of bootstrapping a Tor client. These server descriptors currently
13 amount to about 1.5 Megabytes of data, and this size will grow
14 linearly with network size.
16 Fetching all these server descriptors takes a long while for people
17 behind slow network connections. It is also a considerable load on
18 our network of directory mirrors.
20 This document describes proposed changes to the Tor network and
21 directory protocol so that clients will no longer need to download
22 all server descriptors.
24 These changes consist of moving load balancing information into
25 network status documents, implementing a means to download server
26 descriptors on demand in an anonymity-preserving way, and dealing
27 with exit node selection.
29 2. What is in a server descriptor
31 When a Tor client starts the first thing it will try to get is a
32 current network status document: a consensus signed by a majority
33 of directory authorities. This document is currently about 100
34 Kilobytes in size, tho it will grow linearly with network size.
35 This document lists all servers currently running on the network.
36 The Tor client will then try to get a server descriptor for each
37 of the running servers. All server descriptors currently amount
38 to about 1.5 Megabytes of downloads.
40 A Tor client learns several things about a server from its descriptor.
41 Some of these it already learned from the network status document
42 published by the authorities, but the server descriptor contains it
43 again in a single statement signed by the server itself, not just by
44 the directory authorities.
46 Tor clients use the information from server descriptors for
47 different purposes, which are considered in the following sections.
49 #three ways: One, to determine if a server will be able to handle
50 #this client's request; two, to actually communicate or use the server;
51 #three, for load balancing decisions.
52 #
53 #These three points are considered in the following subsections.
55 2.1 Load balancing
57 The Tor load balancing mechanism is quite complex in its details, but
58 it has a simple goal: The more traffic a server can handle the more
59 traffic it should get. That means the more traffic a server can
60 handle the more likely a client will use it.
62 For this purpose each server descriptor has bandwidth information
63 which tries to convey a server's capacity to clients.
65 Currently we weigh servers differently for different purposes. There
66 is a weigh for when we use a server as a guard node (our entry to the
67 Tor network), there is one weigh we assign servers for exit duties,
68 and a third for when we need intermediate (middle) nodes.
70 2.2 Exit information
72 When a Tor wants to exit to some resource on the internet it will
73 build a circuit to an exit node that allows access to that resource's
74 IP address and TCP Port.
76 When building that circuit the client can make sure that the circuit
77 ends at a server that will be able to fulfill the request because the
78 client already learned of all the servers' exit policies from their
79 descriptors.
81 2.3 Capability information
83 Server descriptors contain information about the specific version or
84 the Tor protocol they understand [proposal 105].
86 Furthermore the server descriptor also contains the exact version of
87 the Tor software that the server is running and some decisions are
88 made based on the server version number (for instance a Tor client
89 will only make conditional consensus requests [proposal 139] when
90 talking to Tor servers version 0.2.1.1-alpha or later).
92 2.4 Contact/key information
94 A server descriptor lists a server's IP address and TCP ports on which
95 it accepts onion and directory connections. Furthermore it contains
96 the onion key (a short lived RSA key to which clients encrypt CREATE
97 cells).
99 2.5 Identity information
101 A Tor client learns the digest of a server's key from the network
102 status document. Once it has a server descriptor this descriptor
103 contains the full RSA identity key of the server. Clients verify
104 that 1) the digest of the identity key matches the expected digest
105 it got from the consensus, and 2) that the signature on the descriptor
106 from that key is valid.
109 3. No longer require clients to have copies of all SDs
111 3.1 Load balancing info in consensus documents
113 One of the reasons why clients download all server descriptors is for
114 doing load proper load balancing as described in 2.1. In order for
115 clients to not require all server descriptors this information will
116 have to move into the network status document.
118 Consensus documents will have a new line per router similar
119 to the "r", "s", and "v" lines that already exist. This line
120 will convey weight information to clients.
122 "w Bandwidth=193671"
124 The bandwidth number is the lesser of observed bandwidth and bandwidth
125 rate limit from the server descriptor that the "r" line referenced by
126 digest (1st and 3rd field of the bandwidth line in the descriptor).
128 The bandwidth item is added as another item in the router tuple
129 described in dir-spec section 3.4:
130 | * Two router entries are "the same" if they have the same
131 | <descriptor digest, published time, nickname, IP, ports> tuple.
132 | We choose the tuple for a given router as whichever tuple appears
133 | for that router in the most votes. We break ties in favor of
134 | the more recently published.
136 3.2 Fetching descriptors on demand
138 As described in 2.4 a descriptor lists IP address, OR- and Dir-Port,
139 and the onion key for a server.
141 A client already knows the IP address and the ports from the consensus
142 documents, but without the onion key it will not be able to send
143 CREATE/EXTEND cells for that server. Since the client needs the onion
144 key it needs the descriptor.
146 If a client only downloaded a few descriptors in an observable manner
147 then that would leak which nodes it was going to use.
149 This proposal suggests the following:
151 1) when connecting to a guard node for which the client does not
152 yet have a cached descriptor it requests the descriptor it
153 expects by hash. (The consensus document that the client holds
154 has a hash for the descriptor of this server. We want exactly
155 that descriptor, not a different one.)
157 It does that by sending a RELAY_REQUEST_SD cell.
159 A client MAY cache the descriptor of the guard node so that it does
160 not need to request it every single time it contacts the guard.
162 2) when a client wants to extend a circuit that currently ends in
163 server B to a new next server C, the client will send a
164 RELAY_REQUEST_SD cell to server B. This cell contains in its
165 payload the hash of a server descriptor the client would like
166 to obtain (C's server descriptor). The server sends back the
167 descriptor and the client can now form a valid EXTEND/CREATE cell
168 encrypted to C's onion key.
170 Clients MUST NOT cache such descriptors. If they did they might
171 leak that they already extended to that server at least once
172 before.
174 Replies to RELAY_REQUEST_SD requests need to be padded to some
175 constant upper limit in order to conceal a client's destination
176 from anybody who might be counting cells/bytes.
178 RELAY_REQUEST_SD cells contain the following information:
179 - hash of the server descriptor requested
180 - hash of the identity digest of the server for which we want the SD
181 - IP address and OR-port or the server for which we want the SD
182 - padding factor - the number of cells we want the answer
183 padded to.
184 [XXX this just occured to me and it might be smart. or it might
185 be stupid. clients would learn the padding factor they want
186 to use from the consensus document. This allows us to grow
187 the replies later on should SDs become larger.]
188 [XXX: figure out a decent padding size]
190 3.3 Protocol versions
192 Server descriptors contain optional information of supported
193 link-level and circuit-level protocols in the form of
194 "opt protocols Link 1 2 Circuit 1". These are not currently needed
195 and will probably eventually move into the "v" (version) line in
196 the consensus. This proposal does not deal with them.
198 Similarly a server descriptor contains the version number of
199 a Tor node. This information is already present in the consensus
200 and is thus available to all clients immediately.
202 3.4 Exit selection
204 Currently finding an appropriate exit node for a user's request is
205 easy for a client because it has complete knowledge of all the exit
206 policies of all servers on the network.
208 The consensus document will once again be extended to contain the
209 information required by clients. This information will be a summary
210 of each node's exit policy. The exit policy summary will only contain
211 the list of ports to which a node exits to most destination IP
212 addresses.
214 A summary should claim a router exits to a specific TCP port if,
215 ignoring private IP addresses (link and site local per RFC3300), the
216 exit policy indicates that the router would exit to this port to any
217 IP address with the exception of at most 2^25 single addresses (That's
218 either two /8 netblocks, or one /8 and a couple of /12s or any other
219 combination).
221 An exit policy summary will be included in votes and consensus as a
222 new line attached to each exit node. A lack of policy should indicate
223 a non-exit policy. The line will have the format
224 "p" <space> "accept"|"reject" <portlist>
225 where portlist is a comma seperated list of single port numbers or
226 portranges (e.g. "22,80-88,1024-6000,6667"). Whether the summary
227 shows the list of accepted ports or the list of rejected ports depends
228 on which list is shorter (has less elements). In case of ties we
229 choose the list of accepted ports.
231 Similarly to IP address, ports, timestamp, and bandwidth a consensus
232 should list the exit policy matching the descriptor digest referenced
233 in the consensus document.
235 4. Migration
237 4.1 Consensus document changes.
239 The consensus will need to include
240 - bandwidth information (see 3.1)
241 - exit policy summaries (3.4)
243 A new consensus method (number TBD) will be chosen for this.
245 5. Future possibilities
247 This proposal still requires that all servers have the descriptors of
248 every other node in the network in order to answer RELAY_REQUEST_SD
249 cells. These cells are sent when a circuit is extended from ending at
250 node B to a new node C. In that case B would have to answer a
251 RELAY_REQUEST_SD cell that asks for C's server descriptor (by SD digest).
253 In order to answer that request B obviously needs a copy of C's server
254 descriptor. The RELAY_REQUEST_SD cell already has all the info that
255 B needs to contact C so it can ask about the descriptor before passing it
256 back to the client.