From e383dd57af51c2ad1bf62620f7993dcb7a16fb72 Mon Sep 17 00:00:00 2001 From: Bartosz Taudul Date: Sun, 15 Feb 2009 18:00:39 +0100 Subject: [PATCH] Add relevant RFCs. --- doc/rfc0959.txt | 3933 +++++++++++++++++++++++++++++++++++++ doc/rfc1123.txt | 5782 +++++++++++++++++++++++++++++++++++++++++++++++++++++++ doc/rfc2228.txt | 1515 +++++++++++++++ doc/rfc2640.txt | 1515 +++++++++++++++ doc/rfc2773.txt | 507 +++++ doc/rfc3659.txt | 3419 ++++++++++++++++++++++++++++++++ 6 files changed, 16671 insertions(+) create mode 100644 doc/rfc0959.txt create mode 100644 doc/rfc1123.txt create mode 100644 doc/rfc2228.txt create mode 100644 doc/rfc2640.txt create mode 100644 doc/rfc2773.txt create mode 100644 doc/rfc3659.txt diff --git a/doc/rfc0959.txt b/doc/rfc0959.txt new file mode 100644 index 0000000..5c9f11a --- /dev/null +++ b/doc/rfc0959.txt @@ -0,0 +1,3933 @@ + + +Network Working Group J. Postel +Request for Comments: 959 J. Reynolds + ISI +Obsoletes RFC: 765 (IEN 149) October 1985 + + FILE TRANSFER PROTOCOL (FTP) + + +Status of this Memo + + This memo is the official specification of the File Transfer + Protocol (FTP). Distribution of this memo is unlimited. + + The following new optional commands are included in this edition of + the specification: + + CDUP (Change to Parent Directory), SMNT (Structure Mount), STOU + (Store Unique), RMD (Remove Directory), MKD (Make Directory), PWD + (Print Directory), and SYST (System). + + Note that this specification is compatible with the previous edition. + +1. INTRODUCTION + + The objectives of FTP are 1) to promote sharing of files (computer + programs and/or data), 2) to encourage indirect or implicit (via + programs) use of remote computers, 3) to shield a user from + variations in file storage systems among hosts, and 4) to transfer + data reliably and efficiently. FTP, though usable directly by a user + at a terminal, is designed mainly for use by programs. + + The attempt in this specification is to satisfy the diverse needs of + users of maxi-hosts, mini-hosts, personal workstations, and TACs, + with a simple, and easily implemented protocol design. + + This paper assumes knowledge of the Transmission Control Protocol + (TCP) [2] and the Telnet Protocol [3]. These documents are contained + in the ARPA-Internet protocol handbook [1]. + +2. OVERVIEW + + In this section, the history, the terminology, and the FTP model are + discussed. The terms defined in this section are only those that + have special significance in FTP. Some of the terminology is very + specific to the FTP model; some readers may wish to turn to the + section on the FTP model while reviewing the terminology. + + + + + + + +Postel & Reynolds [Page 1] + + + +RFC 959 October 1985 +File Transfer Protocol + + + 2.1. HISTORY + + FTP has had a long evolution over the years. Appendix III is a + chronological compilation of Request for Comments documents + relating to FTP. These include the first proposed file transfer + mechanisms in 1971 that were developed for implementation on hosts + at M.I.T. (RFC 114), plus comments and discussion in RFC 141. + + RFC 172 provided a user-level oriented protocol for file transfer + between host computers (including terminal IMPs). A revision of + this as RFC 265, restated FTP for additional review, while RFC 281 + suggested further changes. The use of a "Set Data Type" + transaction was proposed in RFC 294 in January 1982. + + RFC 354 obsoleted RFCs 264 and 265. The File Transfer Protocol + was now defined as a protocol for file transfer between HOSTs on + the ARPANET, with the primary function of FTP defined as + transfering files efficiently and reliably among hosts and + allowing the convenient use of remote file storage capabilities. + RFC 385 further commented on errors, emphasis points, and + additions to the protocol, while RFC 414 provided a status report + on the working server and user FTPs. RFC 430, issued in 1973, + (among other RFCs too numerous to mention) presented further + comments on FTP. Finally, an "official" FTP document was + published as RFC 454. + + By July 1973, considerable changes from the last versions of FTP + were made, but the general structure remained the same. RFC 542 + was published as a new "official" specification to reflect these + changes. However, many implementations based on the older + specification were not updated. + + In 1974, RFCs 607 and 614 continued comments on FTP. RFC 624 + proposed further design changes and minor modifications. In 1975, + RFC 686 entitled, "Leaving Well Enough Alone", discussed the + differences between all of the early and later versions of FTP. + RFC 691 presented a minor revision of RFC 686, regarding the + subject of print files. + + Motivated by the transition from the NCP to the TCP as the + underlying protocol, a phoenix was born out of all of the above + efforts in RFC 765 as the specification of FTP for use on TCP. + + This current edition of the FTP specification is intended to + correct some minor documentation errors, to improve the + explanation of some protocol features, and to add some new + optional commands. + + +Postel & Reynolds [Page 2] + + + +RFC 959 October 1985 +File Transfer Protocol + + + In particular, the following new optional commands are included in + this edition of the specification: + + CDUP - Change to Parent Directory + + SMNT - Structure Mount + + STOU - Store Unique + + RMD - Remove Directory + + MKD - Make Directory + + PWD - Print Directory + + SYST - System + + This specification is compatible with the previous edition. A + program implemented in conformance to the previous specification + should automatically be in conformance to this specification. + + 2.2. TERMINOLOGY + + ASCII + + The ASCII character set is as defined in the ARPA-Internet + Protocol Handbook. In FTP, ASCII characters are defined to be + the lower half of an eight-bit code set (i.e., the most + significant bit is zero). + + access controls + + Access controls define users' access privileges to the use of a + system, and to the files in that system. Access controls are + necessary to prevent unauthorized or accidental use of files. + It is the prerogative of a server-FTP process to invoke access + controls. + + byte size + + There are two byte sizes of interest in FTP: the logical byte + size of the file, and the transfer byte size used for the + transmission of the data. The transfer byte size is always 8 + bits. The transfer byte size is not necessarily the byte size + in which data is to be stored in a system, nor the logical byte + size for interpretation of the structure of the data. + + + +Postel & Reynolds [Page 3] + + + +RFC 959 October 1985 +File Transfer Protocol + + + control connection + + The communication path between the USER-PI and SERVER-PI for + the exchange of commands and replies. This connection follows + the Telnet Protocol. + + data connection + + A full duplex connection over which data is transferred, in a + specified mode and type. The data transferred may be a part of + a file, an entire file or a number of files. The path may be + between a server-DTP and a user-DTP, or between two + server-DTPs. + + data port + + The passive data transfer process "listens" on the data port + for a connection from the active transfer process in order to + open the data connection. + + DTP + + The data transfer process establishes and manages the data + connection. The DTP can be passive or active. + + End-of-Line + + The end-of-line sequence defines the separation of printing + lines. The sequence is Carriage Return, followed by Line Feed. + + EOF + + The end-of-file condition that defines the end of a file being + transferred. + + EOR + + The end-of-record condition that defines the end of a record + being transferred. + + error recovery + + A procedure that allows a user to recover from certain errors + such as failure of either host system or transfer process. In + FTP, error recovery may involve restarting a file transfer at a + given checkpoint. + + + +Postel & Reynolds [Page 4] + + + +RFC 959 October 1985 +File Transfer Protocol + + + FTP commands + + A set of commands that comprise the control information flowing + from the user-FTP to the server-FTP process. + + file + + An ordered set of computer data (including programs), of + arbitrary length, uniquely identified by a pathname. + + mode + + The mode in which data is to be transferred via the data + connection. The mode defines the data format during transfer + including EOR and EOF. The transfer modes defined in FTP are + described in the Section on Transmission Modes. + + NVT + + The Network Virtual Terminal as defined in the Telnet Protocol. + + NVFS + + The Network Virtual File System. A concept which defines a + standard network file system with standard commands and + pathname conventions. + + page + + A file may be structured as a set of independent parts called + pages. FTP supports the transmission of discontinuous files as + independent indexed pages. + + pathname + + Pathname is defined to be the character string which must be + input to a file system by a user in order to identify a file. + Pathname normally contains device and/or directory names, and + file name specification. FTP does not yet specify a standard + pathname convention. Each user must follow the file naming + conventions of the file systems involved in the transfer. + + PI + + The protocol interpreter. The user and server sides of the + protocol have distinct roles implemented in a user-PI and a + server-PI. + + +Postel & Reynolds [Page 5] + + + +RFC 959 October 1985 +File Transfer Protocol + + + record + + A sequential file may be structured as a number of contiguous + parts called records. Record structures are supported by FTP + but a file need not have record structure. + + reply + + A reply is an acknowledgment (positive or negative) sent from + server to user via the control connection in response to FTP + commands. The general form of a reply is a completion code + (including error codes) followed by a text string. The codes + are for use by programs and the text is usually intended for + human users. + + server-DTP + + The data transfer process, in its normal "active" state, + establishes the data connection with the "listening" data port. + It sets up parameters for transfer and storage, and transfers + data on command from its PI. The DTP can be placed in a + "passive" state to listen for, rather than initiate a + connection on the data port. + + server-FTP process + + A process or set of processes which perform the function of + file transfer in cooperation with a user-FTP process and, + possibly, another server. The functions consist of a protocol + interpreter (PI) and a data transfer process (DTP). + + server-PI + + The server protocol interpreter "listens" on Port L for a + connection from a user-PI and establishes a control + communication connection. It receives standard FTP commands + from the user-PI, sends replies, and governs the server-DTP. + + type + + The data representation type used for data transfer and + storage. Type implies certain transformations between the time + of data storage and data transfer. The representation types + defined in FTP are described in the Section on Establishing + Data Connections. + + + + +Postel & Reynolds [Page 6] + + + +RFC 959 October 1985 +File Transfer Protocol + + + user + + A person or a process on behalf of a person wishing to obtain + file transfer service. The human user may interact directly + with a server-FTP process, but use of a user-FTP process is + preferred since the protocol design is weighted towards + automata. + + user-DTP + + The data transfer process "listens" on the data port for a + connection from a server-FTP process. If two servers are + transferring data between them, the user-DTP is inactive. + + user-FTP process + + A set of functions including a protocol interpreter, a data + transfer process and a user interface which together perform + the function of file transfer in cooperation with one or more + server-FTP processes. The user interface allows a local + language to be used in the command-reply dialogue with the + user. + + user-PI + + The user protocol interpreter initiates the control connection + from its port U to the server-FTP process, initiates FTP + commands, and governs the user-DTP if that process is part of + the file transfer. + + + + + + + + + + + + + + + + + + + + +Postel & Reynolds [Page 7] + + + +RFC 959 October 1985 +File Transfer Protocol + + + 2.3. THE FTP MODEL + + With the above definitions in mind, the following model (shown in + Figure 1) may be diagrammed for an FTP service. + + ------------- + |/---------\| + || User || -------- + ||Interface|<--->| User | + |\----^----/| -------- + ---------- | | | + |/------\| FTP Commands |/----V----\| + ||Server|<---------------->| User || + || PI || FTP Replies || PI || + |\--^---/| |\----^----/| + | | | | | | + -------- |/--V---\| Data |/----V----\| -------- + | File |<--->|Server|<---------------->| User |<--->| File | + |System| || DTP || Connection || DTP || |System| + -------- |\------/| |\---------/| -------- + ---------- ------------- + + Server-FTP USER-FTP + + NOTES: 1. The data connection may be used in either direction. + 2. The data connection need not exist all of the time. + + Figure 1 Model for FTP Use + + In the model described in Figure 1, the user-protocol interpreter + initiates the control connection. The control connection follows + the Telnet protocol. At the initiation of the user, standard FTP + commands are generated by the user-PI and transmitted to the + server process via the control connection. (The user may + establish a direct control connection to the server-FTP, from a + TAC terminal for example, and generate standard FTP commands + independently, bypassing the user-FTP process.) Standard replies + are sent from the server-PI to the user-PI over the control + connection in response to the commands. + + The FTP commands specify the parameters for the data connection + (data port, transfer mode, representation type, and structure) and + the nature of file system operation (store, retrieve, append, + delete, etc.). The user-DTP or its designate should "listen" on + the specified data port, and the server initiate the data + connection and data transfer in accordance with the specified + parameters. It should be noted that the data port need not be in + + +Postel & Reynolds [Page 8] + + + +RFC 959 October 1985 +File Transfer Protocol + + + the same host that initiates the FTP commands via the control + connection, but the user or the user-FTP process must ensure a + "listen" on the specified data port. It ought to also be noted + that the data connection may be used for simultaneous sending and + receiving. + + In another situation a user might wish to transfer files between + two hosts, neither of which is a local host. The user sets up + control connections to the two servers and then arranges for a + data connection between them. In this manner, control information + is passed to the user-PI but data is transferred between the + server data transfer processes. Following is a model of this + server-server interaction. + + + Control ------------ Control + ---------->| User-FTP |<----------- + | | User-PI | | + | | "C" | | + V ------------ V + -------------- -------------- + | Server-FTP | Data Connection | Server-FTP | + | "A" |<---------------------->| "B" | + -------------- Port (A) Port (B) -------------- + + + Figure 2 + + The protocol requires that the control connections be open while + data transfer is in progress. It is the responsibility of the + user to request the closing of the control connections when + finished using the FTP service, while it is the server who takes + the action. The server may abort data transfer if the control + connections are closed without command. + + The Relationship between FTP and Telnet: + + The FTP uses the Telnet protocol on the control connection. + This can be achieved in two ways: first, the user-PI or the + server-PI may implement the rules of the Telnet Protocol + directly in their own procedures; or, second, the user-PI or + the server-PI may make use of the existing Telnet module in the + system. + + Ease of implementaion, sharing code, and modular programming + argue for the second approach. Efficiency and independence + + + +Postel & Reynolds [Page 9] + + + +RFC 959 October 1985 +File Transfer Protocol + + + argue for the first approach. In practice, FTP relies on very + little of the Telnet Protocol, so the first approach does not + necessarily involve a large amount of code. + +3. DATA TRANSFER FUNCTIONS + + Files are transferred only via the data connection. The control + connection is used for the transfer of commands, which describe the + functions to be performed, and the replies to these commands (see the + Section on FTP Replies). Several commands are concerned with the + transfer of data between hosts. These data transfer commands include + the MODE command which specify how the bits of the data are to be + transmitted, and the STRUcture and TYPE commands, which are used to + define the way in which the data are to be represented. The + transmission and representation are basically independent but the + "Stream" transmission mode is dependent on the file structure + attribute and if "Compressed" transmission mode is used, the nature + of the filler byte depends on the representation type. + + 3.1. DATA REPRESENTATION AND STORAGE + + Data is transferred from a storage device in the sending host to a + storage device in the receiving host. Often it is necessary to + perform certain transformations on the data because data storage + representations in the two systems are different. For example, + NVT-ASCII has different data storage representations in different + systems. DEC TOPS-20s's generally store NVT-ASCII as five 7-bit + ASCII characters, left-justified in a 36-bit word. IBM Mainframe's + store NVT-ASCII as 8-bit EBCDIC codes. Multics stores NVT-ASCII + as four 9-bit characters in a 36-bit word. It is desirable to + convert characters into the standard NVT-ASCII representation when + transmitting text between dissimilar systems. The sending and + receiving sites would have to perform the necessary + transformations between the standard representation and their + internal representations. + + A different problem in representation arises when transmitting + binary data (not character codes) between host systems with + different word lengths. It is not always clear how the sender + should send data, and the receiver store it. For example, when + transmitting 32-bit bytes from a 32-bit word-length system to a + 36-bit word-length system, it may be desirable (for reasons of + efficiency and usefulness) to store the 32-bit bytes + right-justified in a 36-bit word in the latter system. In any + case, the user should have the option of specifying data + representation and transformation functions. It should be noted + + + +Postel & Reynolds [Page 10] + + + +RFC 959 October 1985 +File Transfer Protocol + + + that FTP provides for very limited data type representations. + Transformations desired beyond this limited capability should be + performed by the user directly. + + 3.1.1. DATA TYPES + + Data representations are handled in FTP by a user specifying a + representation type. This type may implicitly (as in ASCII or + EBCDIC) or explicitly (as in Local byte) define a byte size for + interpretation which is referred to as the "logical byte size." + Note that this has nothing to do with the byte size used for + transmission over the data connection, called the "transfer + byte size", and the two should not be confused. For example, + NVT-ASCII has a logical byte size of 8 bits. If the type is + Local byte, then the TYPE command has an obligatory second + parameter specifying the logical byte size. The transfer byte + size is always 8 bits. + + 3.1.1.1. ASCII TYPE + + This is the default type and must be accepted by all FTP + implementations. It is intended primarily for the transfer + of text files, except when both hosts would find the EBCDIC + type more convenient. + + The sender converts the data from an internal character + representation to the standard 8-bit NVT-ASCII + representation (see the Telnet specification). The receiver + will convert the data from the standard form to his own + internal form. + + In accordance with the NVT standard, the sequence + should be used where necessary to denote the end of a line + of text. (See the discussion of file structure at the end + of the Section on Data Representation and Storage.) + + Using the standard NVT-ASCII representation means that data + must be interpreted as 8-bit bytes. + + The Format parameter for ASCII and EBCDIC types is discussed + below. + + + + + + + + +Postel & Reynolds [Page 11] + + + +RFC 959 October 1985 +File Transfer Protocol + + + 3.1.1.2. EBCDIC TYPE + + This type is intended for efficient transfer between hosts + which use EBCDIC for their internal character + representation. + + For transmission, the data are represented as 8-bit EBCDIC + characters. The character code is the only difference + between the functional specifications of EBCDIC and ASCII + types. + + End-of-line (as opposed to end-of-record--see the discussion + of structure) will probably be rarely used with EBCDIC type + for purposes of denoting structure, but where it is + necessary the character should be used. + + 3.1.1.3. IMAGE TYPE + + The data are sent as contiguous bits which, for transfer, + are packed into the 8-bit transfer bytes. The receiving + site must store the data as contiguous bits. The structure + of the storage system might necessitate the padding of the + file (or of each record, for a record-structured file) to + some convenient boundary (byte, word or block). This + padding, which must be all zeros, may occur only at the end + of the file (or at the end of each record) and there must be + a way of identifying the padding bits so that they may be + stripped off if the file is retrieved. The padding + transformation should be well publicized to enable a user to + process a file at the storage site. + + Image type is intended for the efficient storage and + retrieval of files and for the transfer of binary data. It + is recommended that this type be accepted by all FTP + implementations. + + 3.1.1.4. LOCAL TYPE + + The data is transferred in logical bytes of the size + specified by the obligatory second parameter, Byte size. + The value of Byte size must be a decimal integer; there is + no default value. The logical byte size is not necessarily + the same as the transfer byte size. If there is a + difference in byte sizes, then the logical bytes should be + packed contiguously, disregarding transfer byte boundaries + and with any necessary padding at the end. + + + +Postel & Reynolds [Page 12] + + + +RFC 959 October 1985 +File Transfer Protocol + + + When the data reaches the receiving host, it will be + transformed in a manner dependent on the logical byte size + and the particular host. This transformation must be + invertible (i.e., an identical file can be retrieved if the + same parameters are used) and should be well publicized by + the FTP implementors. + + For example, a user sending 36-bit floating-point numbers to + a host with a 32-bit word could send that data as Local byte + with a logical byte size of 36. The receiving host would + then be expected to store the logical bytes so that they + could be easily manipulated; in this example putting the + 36-bit logical bytes into 64-bit double words should + suffice. + + In another example, a pair of hosts with a 36-bit word size + may send data to one another in words by using TYPE L 36. + The data would be sent in the 8-bit transmission bytes + packed so that 9 transmission bytes carried two host words. + + 3.1.1.5. FORMAT CONTROL + + The types ASCII and EBCDIC also take a second (optional) + parameter; this is to indicate what kind of vertical format + control, if any, is associated with a file. The following + data representation types are defined in FTP: + + A character file may be transferred to a host for one of + three purposes: for printing, for storage and later + retrieval, or for processing. If a file is sent for + printing, the receiving host must know how the vertical + format control is represented. In the second case, it must + be possible to store a file at a host and then retrieve it + later in exactly the same form. Finally, it should be + possible to move a file from one host to another and process + the file at the second host without undue trouble. A single + ASCII or EBCDIC format does not satisfy all these + conditions. Therefore, these types have a second parameter + specifying one of the following three formats: + + 3.1.1.5.1. NON PRINT + + This is the default format to be used if the second + (format) parameter is omitted. Non-print format must be + accepted by all FTP implementations. + + + + +Postel & Reynolds [Page 13] + + + +RFC 959 October 1985 +File Transfer Protocol + + + The file need contain no vertical format information. If + it is passed to a printer process, this process may + assume standard values for spacing and margins. + + Normally, this format will be used with files destined + for processing or just storage. + + 3.1.1.5.2. TELNET FORMAT CONTROLS + + The file contains ASCII/EBCDIC vertical format controls + (i.e., , , , , ) which the printer + process will interpret appropriately. , in exactly + this sequence, also denotes end-of-line. + + 3.1.1.5.2. CARRIAGE CONTROL (ASA) + + The file contains ASA (FORTRAN) vertical format control + characters. (See RFC 740 Appendix C; and Communications + of the ACM, Vol. 7, No. 10, p. 606, October 1964.) In a + line or a record formatted according to the ASA Standard, + the first character is not to be printed. Instead, it + should be used to determine the vertical movement of the + paper which should take place before the rest of the + record is printed. + + The ASA Standard specifies the following control + characters: + + Character Vertical Spacing + + blank Move paper up one line + 0 Move paper up two lines + 1 Move paper to top of next page + + No movement, i.e., overprint + + Clearly there must be some way for a printer process to + distinguish the end of the structural entity. If a file + has record structure (see below) this is no problem; + records will be explicitly marked during transfer and + storage. If the file has no record structure, the + end-of-line sequence is used to separate printing lines, + but these format effectors are overridden by the ASA + controls. + + + + + + +Postel & Reynolds [Page 14] + + + +RFC 959 October 1985 +File Transfer Protocol + + + 3.1.2. DATA STRUCTURES + + In addition to different representation types, FTP allows the + structure of a file to be specified. Three file structures are + defined in FTP: + + file-structure, where there is no internal structure and + the file is considered to be a + continuous sequence of data bytes, + + record-structure, where the file is made up of sequential + records, + + and page-structure, where the file is made up of independent + indexed pages. + + File-structure is the default to be assumed if the STRUcture + command has not been used but both file and record structures + must be accepted for "text" files (i.e., files with TYPE ASCII + or EBCDIC) by all FTP implementations. The structure of a file + will affect both the transfer mode of a file (see the Section + on Transmission Modes) and the interpretation and storage of + the file. + + The "natural" structure of a file will depend on which host + stores the file. A source-code file will usually be stored on + an IBM Mainframe in fixed length records but on a DEC TOPS-20 + as a stream of characters partitioned into lines, for example + by . If the transfer of files between such disparate + sites is to be useful, there must be some way for one site to + recognize the other's assumptions about the file. + + With some sites being naturally file-oriented and others + naturally record-oriented there may be problems if a file with + one structure is sent to a host oriented to the other. If a + text file is sent with record-structure to a host which is file + oriented, then that host should apply an internal + transformation to the file based on the record structure. + Obviously, this transformation should be useful, but it must + also be invertible so that an identical file may be retrieved + using record structure. + + In the case of a file being sent with file-structure to a + record-oriented host, there exists the question of what + criteria the host should use to divide the file into records + which can be processed locally. If this division is necessary, + the FTP implementation should use the end-of-line sequence, + + +Postel & Reynolds [Page 15] + + + +RFC 959 October 1985 +File Transfer Protocol + + + for ASCII, or for EBCDIC text files, as the + delimiter. If an FTP implementation adopts this technique, it + must be prepared to reverse the transformation if the file is + retrieved with file-structure. + + 3.1.2.1. FILE STRUCTURE + + File structure is the default to be assumed if the STRUcture + command has not been used. + + In file-structure there is no internal structure and the + file is considered to be a continuous sequence of data + bytes. + + 3.1.2.2. RECORD STRUCTURE + + Record structures must be accepted for "text" files (i.e., + files with TYPE ASCII or EBCDIC) by all FTP implementations. + + In record-structure the file is made up of sequential + records. + + 3.1.2.3. PAGE STRUCTURE + + To transmit files that are discontinuous, FTP defines a page + structure. Files of this type are sometimes known as + "random access files" or even as "holey files". In these + files there is sometimes other information associated with + the file as a whole (e.g., a file descriptor), or with a + section of the file (e.g., page access controls), or both. + In FTP, the sections of the file are called pages. + + To provide for various page sizes and associated + information, each page is sent with a page header. The page + header has the following defined fields: + + Header Length + + The number of logical bytes in the page header + including this byte. The minimum header length is 4. + + Page Index + + The logical page number of this section of the file. + This is not the transmission sequence number of this + page, but the index used to identify this page of the + file. + + +Postel & Reynolds [Page 16] + + + +RFC 959 October 1985 +File Transfer Protocol + + + Data Length + + The number of logical bytes in the page data. The + minimum data length is 0. + + Page Type + + The type of page this is. The following page types + are defined: + + 0 = Last Page + + This is used to indicate the end of a paged + structured transmission. The header length must + be 4, and the data length must be 0. + + 1 = Simple Page + + This is the normal type for simple paged files + with no page level associated control + information. The header length must be 4. + + 2 = Descriptor Page + + This type is used to transmit the descriptive + information for the file as a whole. + + 3 = Access Controlled Page + + This type includes an additional header field + for paged files with page level access control + information. The header length must be 5. + + Optional Fields + + Further header fields may be used to supply per page + control information, for example, per page access + control. + + All fields are one logical byte in length. The logical byte + size is specified by the TYPE command. See Appendix I for + further details and a specific case at the page structure. + + A note of caution about parameters: a file must be stored and + retrieved with the same parameters if the retrieved version is to + + + + +Postel & Reynolds [Page 17] + + + +RFC 959 October 1985 +File Transfer Protocol + + + be identical to the version originally transmitted. Conversely, + FTP implementations must return a file identical to the original + if the parameters used to store and retrieve a file are the same. + + 3.2. ESTABLISHING DATA CONNECTIONS + + The mechanics of transferring data consists of setting up the data + connection to the appropriate ports and choosing the parameters + for transfer. Both the user and the server-DTPs have a default + data port. The user-process default data port is the same as the + control connection port (i.e., U). The server-process default + data port is the port adjacent to the control connection port + (i.e., L-1). + + The transfer byte size is 8-bit bytes. This byte size is relevant + only for the actual transfer of the data; it has no bearing on + representation of the data within a host's file system. + + The passive data transfer process (this may be a user-DTP or a + second server-DTP) shall "listen" on the data port prior to + sending a transfer request command. The FTP request command + determines the direction of the data transfer. The server, upon + receiving the transfer request, will initiate the data connection + to the port. When the connection is established, the data + transfer begins between DTP's, and the server-PI sends a + confirming reply to the user-PI. + + Every FTP implementation must support the use of the default data + ports, and only the USER-PI can initiate a change to non-default + ports. + + It is possible for the user to specify an alternate data port by + use of the PORT command. The user may want a file dumped on a TAC + line printer or retrieved from a third party host. In the latter + case, the user-PI sets up control connections with both + server-PI's. One server is then told (by an FTP command) to + "listen" for a connection which the other will initiate. The + user-PI sends one server-PI a PORT command indicating the data + port of the other. Finally, both are sent the appropriate + transfer commands. The exact sequence of commands and replies + sent between the user-controller and the servers is defined in the + Section on FTP Replies. + + In general, it is the server's responsibility to maintain the data + connection--to initiate it and to close it. The exception to this + + + + +Postel & Reynolds [Page 18] + + + +RFC 959 October 1985 +File Transfer Protocol + + + is when the user-DTP is sending the data in a transfer mode that + requires the connection to be closed to indicate EOF. The server + MUST close the data connection under the following conditions: + + 1. The server has completed sending data in a transfer mode + that requires a close to indicate EOF. + + 2. The server receives an ABORT command from the user. + + 3. The port specification is changed by a command from the + user. + + 4. The control connection is closed legally or otherwise. + + 5. An irrecoverable error condition occurs. + + Otherwise the close is a server option, the exercise of which the + server must indicate to the user-process by either a 250 or 226 + reply only. + + 3.3. DATA CONNECTION MANAGEMENT + + Default Data Connection Ports: All FTP implementations must + support use of the default data connection ports, and only the + User-PI may initiate the use of non-default ports. + + Negotiating Non-Default Data Ports: The User-PI may specify a + non-default user side data port with the PORT command. The + User-PI may request the server side to identify a non-default + server side data port with the PASV command. Since a connection + is defined by the pair of addresses, either of these actions is + enough to get a different data connection, still it is permitted + to do both commands to use new ports on both ends of the data + connection. + + Reuse of the Data Connection: When using the stream mode of data + transfer the end of the file must be indicated by closing the + connection. This causes a problem if multiple files are to be + transfered in the session, due to need for TCP to hold the + connection record for a time out period to guarantee the reliable + communication. Thus the connection can not be reopened at once. + + There are two solutions to this problem. The first is to + negotiate a non-default port. The second is to use another + transfer mode. + + A comment on transfer modes. The stream transfer mode is + + +Postel & Reynolds [Page 19] + + + +RFC 959 October 1985 +File Transfer Protocol + + + inherently unreliable, since one can not determine if the + connection closed prematurely or not. The other transfer modes + (Block, Compressed) do not close the connection to indicate the + end of file. They have enough FTP encoding that the data + connection can be parsed to determine the end of the file. + Thus using these modes one can leave the data connection open + for multiple file transfers. + + 3.4. TRANSMISSION MODES + + The next consideration in transferring data is choosing the + appropriate transmission mode. There are three modes: one which + formats the data and allows for restart procedures; one which also + compresses the data for efficient transfer; and one which passes + the data with little or no processing. In this last case the mode + interacts with the structure attribute to determine the type of + processing. In the compressed mode, the representation type + determines the filler byte. + + All data transfers must be completed with an end-of-file (EOF) + which may be explicitly stated or implied by the closing of the + data connection. For files with record structure, all the + end-of-record markers (EOR) are explicit, including the final one. + For files transmitted in page structure a "last-page" page type is + used. + + NOTE: In the rest of this section, byte means "transfer byte" + except where explicitly stated otherwise. + + For the purpose of standardized transfer, the sending host will + translate its internal end of line or end of record denotation + into the representation prescribed by the transfer mode and file + structure, and the receiving host will perform the inverse + translation to its internal denotation. An IBM Mainframe record + count field may not be recognized at another host, so the + end-of-record information may be transferred as a two byte control + code in Stream mode or as a flagged bit in a Block or Compressed + mode descriptor. End-of-line in an ASCII or EBCDIC file with no + record structure should be indicated by or , + respectively. Since these transformations imply extra work for + some systems, identical systems transferring non-record structured + text files might wish to use a binary representation and stream + mode for the transfer. + + + + + + +Postel & Reynolds [Page 20] + + + +RFC 959 October 1985 +File Transfer Protocol + + + The following transmission modes are defined in FTP: + + 3.4.1. STREAM MODE + + The data is transmitted as a stream of bytes. There is no + restriction on the representation type used; record structures + are allowed. + + In a record structured file EOR and EOF will each be indicated + by a two-byte control code. The first byte of the control code + will be all ones, the escape character. The second byte will + have the low order bit on and zeros elsewhere for EOR and the + second low order bit on for EOF; that is, the byte will have + value 1 for EOR and value 2 for EOF. EOR and EOF may be + indicated together on the last byte transmitted by turning both + low order bits on (i.e., the value 3). If a byte of all ones + was intended to be sent as data, it should be repeated in the + second byte of the control code. + + If the structure is a file structure, the EOF is indicated by + the sending host closing the data connection and all bytes are + data bytes. + + 3.4.2. BLOCK MODE + + The file is transmitted as a series of data blocks preceded by + one or more header bytes. The header bytes contain a count + field, and descriptor code. The count field indicates the + total length of the data block in bytes, thus marking the + beginning of the next data block (there are no filler bits). + The descriptor code defines: last block in the file (EOF) last + block in the record (EOR), restart marker (see the Section on + Error Recovery and Restart) or suspect data (i.e., the data + being transferred is suspected of errors and is not reliable). + This last code is NOT intended for error control within FTP. + It is motivated by the desire of sites exchanging certain types + of data (e.g., seismic or weather data) to send and receive all + the data despite local errors (such as "magnetic tape read + errors"), but to indicate in the transmission that certain + portions are suspect). Record structures are allowed in this + mode, and any representation type may be used. + + The header consists of the three bytes. Of the 24 bits of + header information, the 16 low order bits shall represent byte + count, and the 8 high order bits shall represent descriptor + codes as shown below. + + + +Postel & Reynolds [Page 21] + + + +RFC 959 October 1985 +File Transfer Protocol + + + Block Header + + +----------------+----------------+----------------+ + | Descriptor | Byte Count | + | 8 bits | 16 bits | + +----------------+----------------+----------------+ + + + The descriptor codes are indicated by bit flags in the + descriptor byte. Four codes have been assigned, where each + code number is the decimal value of the corresponding bit in + the byte. + + Code Meaning + + 128 End of data block is EOR + 64 End of data block is EOF + 32 Suspected errors in data block + 16 Data block is a restart marker + + With this encoding, more than one descriptor coded condition + may exist for a particular block. As many bits as necessary + may be flagged. + + The restart marker is embedded in the data stream as an + integral number of 8-bit bytes representing printable + characters in the language being used over the control + connection (e.g., default--NVT-ASCII). (Space, in the + appropriate language) must not be used WITHIN a restart marker. + + For example, to transmit a six-character marker, the following + would be sent: + + +--------+--------+--------+ + |Descrptr| Byte count | + |code= 16| = 6 | + +--------+--------+--------+ + + +--------+--------+--------+ + | Marker | Marker | Marker | + | 8 bits | 8 bits | 8 bits | + +--------+--------+--------+ + + +--------+--------+--------+ + | Marker | Marker | Marker | + | 8 bits | 8 bits | 8 bits | + +--------+--------+--------+ + + +Postel & Reynolds [Page 22] + + + +RFC 959 October 1985 +File Transfer Protocol + + + 3.4.3. COMPRESSED MODE + + There are three kinds of information to be sent: regular data, + sent in a byte string; compressed data, consisting of + replications or filler; and control information, sent in a + two-byte escape sequence. If n>0 bytes (up to 127) of regular + data are sent, these n bytes are preceded by a byte with the + left-most bit set to 0 and the right-most 7 bits containing the + number n. + + Byte string: + + 1 7 8 8 + +-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+ + |0| n | | d(1) | ... | d(n) | + +-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+ + ^ ^ + |---n bytes---| + of data + + String of n data bytes d(1),..., d(n) + Count n must be positive. + + To compress a string of n replications of the data byte d, the + following 2 bytes are sent: + + Replicated Byte: + + 2 6 8 + +-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+ + |1 0| n | | d | + +-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+ + + A string of n filler bytes can be compressed into a single + byte, where the filler byte varies with the representation + type. If the type is ASCII or EBCDIC the filler byte is + (Space, ASCII code 32, EBCDIC code 64). If the type is Image + or Local byte the filler is a zero byte. + + Filler String: + + 2 6 + +-+-+-+-+-+-+-+-+ + |1 1| n | + +-+-+-+-+-+-+-+-+ + + The escape sequence is a double byte, the first of which is the + + +Postel & Reynolds [Page 23] + + + +RFC 959 October 1985 +File Transfer Protocol + + + escape byte (all zeros) and the second of which contains + descriptor codes as defined in Block mode. The descriptor + codes have the same meaning as in Block mode and apply to the + succeeding string of bytes. + + Compressed mode is useful for obtaining increased bandwidth on + very large network transmissions at a little extra CPU cost. + It can be most effectively used to reduce the size of printer + files such as those generated by RJE hosts. + + 3.5. ERROR RECOVERY AND RESTART + + There is no provision for detecting bits lost or scrambled in data + transfer; this level of error control is handled by the TCP. + However, a restart procedure is provided to protect users from + gross system failures (including failures of a host, an + FTP-process, or the underlying network). + + The restart procedure is defined only for the block and compressed + modes of data transfer. It requires the sender of data to insert + a special marker code in the data stream with some marker + information. The marker information has meaning only to the + sender, but must consist of printable characters in the default or + negotiated language of the control connection (ASCII or EBCDIC). + The marker could represent a bit-count, a record-count, or any + other information by which a system may identify a data + checkpoint. The receiver of data, if it implements the restart + procedure, would then mark the corresponding position of this + marker in the receiving system, and return this information to the + user. + + In the event of a system failure, the user can restart the data + transfer by identifying the marker point with the FTP restart + procedure. The following example illustrates the use of the + restart procedure. + + The sender of the data inserts an appropriate marker block in the + data stream at a convenient point. The receiving host marks the + corresponding data point in its file system and conveys the last + known sender and receiver marker information to the user, either + directly or over the control connection in a 110 reply (depending + on who is the sender). In the event of a system failure, the user + or controller process restarts the server at the last server + marker by sending a restart command with server's marker code as + its argument. The restart command is transmitted over the control + + + + +Postel & Reynolds [Page 24] + + + +RFC 959 October 1985 +File Transfer Protocol + + + connection and is immediately followed by the command (such as + RETR, STOR or LIST) which was being executed when the system + failure occurred. + +4. FILE TRANSFER FUNCTIONS + + The communication channel from the user-PI to the server-PI is + established as a TCP connection from the user to the standard server + port. The user protocol interpreter is responsible for sending FTP + commands and interpreting the replies received; the server-PI + interprets commands, sends replies and directs its DTP to set up the + data connection and transfer the data. If the second party to the + data transfer (the passive transfer process) is the user-DTP, then it + is governed through the internal protocol of the user-FTP host; if it + is a second server-DTP, then it is governed by its PI on command from + the user-PI. The FTP replies are discussed in the next section. In + the description of a few of the commands in this section, it is + helpful to be explicit about the possible replies. + + 4.1. FTP COMMANDS + + 4.1.1. ACCESS CONTROL COMMANDS + + The following commands specify access control identifiers + (command codes are shown in parentheses). + + USER NAME (USER) + + The argument field is a Telnet string identifying the user. + The user identification is that which is required by the + server for access to its file system. This command will + normally be the first command transmitted by the user after + the control connections are made (some servers may require + this). Additional identification information in the form of + a password and/or an account command may also be required by + some servers. Servers may allow a new USER command to be + entered at any point in order to change the access control + and/or accounting information. This has the effect of + flushing any user, password, and account information already + supplied and beginning the login sequence again. All + transfer parameters are unchanged and any file transfer in + progress is completed under the old access control + parameters. + + + + + + +Postel & Reynolds [Page 25] + + + +RFC 959 October 1985 +File Transfer Protocol + + + PASSWORD (PASS) + + The argument field is a Telnet string specifying the user's + password. This command must be immediately preceded by the + user name command, and, for some sites, completes the user's + identification for access control. Since password + information is quite sensitive, it is desirable in general + to "mask" it or suppress typeout. It appears that the + server has no foolproof way to achieve this. It is + therefore the responsibility of the user-FTP process to hide + the sensitive password information. + + ACCOUNT (ACCT) + + The argument field is a Telnet string identifying the user's + account. The command is not necessarily related to the USER + command, as some sites may require an account for login and + others only for specific access, such as storing files. In + the latter case the command may arrive at any time. + + There are reply codes to differentiate these cases for the + automation: when account information is required for login, + the response to a successful PASSword command is reply code + 332. On the other hand, if account information is NOT + required for login, the reply to a successful PASSword + command is 230; and if the account information is needed for + a command issued later in the dialogue, the server should + return a 332 or 532 reply depending on whether it stores + (pending receipt of the ACCounT command) or discards the + command, respectively. + + CHANGE WORKING DIRECTORY (CWD) + + This command allows the user to work with a different + directory or dataset for file storage or retrieval without + altering his login or accounting information. Transfer + parameters are similarly unchanged. The argument is a + pathname specifying a directory or other system dependent + file group designator. + + CHANGE TO PARENT DIRECTORY (CDUP) + + This command is a special case of CWD, and is included to + simplify the implementation of programs for transferring + directory trees between operating systems having different + + + + +Postel & Reynolds [Page 26] + + + +RFC 959 October 1985 +File Transfer Protocol + + + syntaxes for naming the parent directory. The reply codes + shall be identical to the reply codes of CWD. See + Appendix II for further details. + + STRUCTURE MOUNT (SMNT) + + This command allows the user to mount a different file + system data structure without altering his login or + accounting information. Transfer parameters are similarly + unchanged. The argument is a pathname specifying a + directory or other system dependent file group designator. + + REINITIALIZE (REIN) + + This command terminates a USER, flushing all I/O and account + information, except to allow any transfer in progress to be + completed. All parameters are reset to the default settings + and the control connection is left open. This is identical + to the state in which a user finds himself immediately after + the control connection is opened. A USER command may be + expected to follow. + + LOGOUT (QUIT) + + This command terminates a USER and if file transfer is not + in progress, the server closes the control connection. If + file transfer is in progress, the connection will remain + open for result response and the server will then close it. + If the user-process is transferring files for several USERs + but does not wish to close and then reopen connections for + each, then the REIN command should be used instead of QUIT. + + An unexpected close on the control connection will cause the + server to take the effective action of an abort (ABOR) and a + logout (QUIT). + + 4.1.2. TRANSFER PARAMETER COMMANDS + + All data transfer parameters have default values, and the + commands specifying data transfer parameters are required only + if the default parameter values are to be changed. The default + value is the last specified value, or if no value has been + specified, the standard default value is as stated here. This + implies that the server must "remember" the applicable default + values. The commands may be in any order except that they must + precede the FTP service request. The following commands + specify data transfer parameters: + + +Postel & Reynolds [Page 27] + + + +RFC 959 October 1985 +File Transfer Protocol + + + DATA PORT (PORT) + + The argument is a HOST-PORT specification for the data port + to be used in data connection. There are defaults for both + the user and server data ports, and under normal + circumstances this command and its reply are not needed. If + this command is used, the argument is the concatenation of a + 32-bit internet host address and a 16-bit TCP port address. + This address information is broken into 8-bit fields and the + value of each field is transmitted as a decimal number (in + character string representation). The fields are separated + by commas. A port command would be: + + PORT h1,h2,h3,h4,p1,p2 + + where h1 is the high order 8 bits of the internet host + address. + + PASSIVE (PASV) + + This command requests the server-DTP to "listen" on a data + port (which is not its default data port) and to wait for a + connection rather than initiate one upon receipt of a + transfer command. The response to this command includes the + host and port address this server is listening on. + + REPRESENTATION TYPE (TYPE) + + The argument specifies the representation type as described + in the Section on Data Representation and Storage. Several + types take a second parameter. The first parameter is + denoted by a single Telnet character, as is the second + Format parameter for ASCII and EBCDIC; the second parameter + for local byte is a decimal integer to indicate Bytesize. + The parameters are separated by a (Space, ASCII code + 32). + + The following codes are assigned for type: + + \ / + A - ASCII | | N - Non-print + |-><-| T - Telnet format effectors + E - EBCDIC| | C - Carriage Control (ASA) + / \ + I - Image + + L - Local byte Byte size + + +Postel & Reynolds [Page 28] + + + +RFC 959 October 1985 +File Transfer Protocol + + + The default representation type is ASCII Non-print. If the + Format parameter is changed, and later just the first + argument is changed, Format then returns to the Non-print + default. + + FILE STRUCTURE (STRU) + + The argument is a single Telnet character code specifying + file structure described in the Section on Data + Representation and Storage. + + The following codes are assigned for structure: + + F - File (no record structure) + R - Record structure + P - Page structure + + The default structure is File. + + TRANSFER MODE (MODE) + + The argument is a single Telnet character code specifying + the data transfer modes described in the Section on + Transmission Modes. + + The following codes are assigned for transfer modes: + + S - Stream + B - Block + C - Compressed + + The default transfer mode is Stream. + + 4.1.3. FTP SERVICE COMMANDS + + The FTP service commands define the file transfer or the file + system function requested by the user. The argument of an FTP + service command will normally be a pathname. The syntax of + pathnames must conform to server site conventions (with + standard defaults applicable), and the language conventions of + the control connection. The suggested default handling is to + use the last specified device, directory or file name, or the + standard default defined for local users. The commands may be + in any order except that a "rename from" command must be + followed by a "rename to" command and the restart command must + be followed by the interrupted service command (e.g., STOR or + RETR). The data, when transferred in response to FTP service + + +Postel & Reynolds [Page 29] + + + +RFC 959 October 1985 +File Transfer Protocol + + + commands, shall always be sent over the data connection, except + for certain informative replies. The following commands + specify FTP service requests: + + RETRIEVE (RETR) + + This command causes the server-DTP to transfer a copy of the + file, specified in the pathname, to the server- or user-DTP + at the other end of the data connection. The status and + contents of the file at the server site shall be unaffected. + + STORE (STOR) + + This command causes the server-DTP to accept the data + transferred via the data connection and to store the data as + a file at the server site. If the file specified in the + pathname exists at the server site, then its contents shall + be replaced by the data being transferred. A new file is + created at the server site if the file specified in the + pathname does not already exist. + + STORE UNIQUE (STOU) + + This command behaves like STOR except that the resultant + file is to be created in the current directory under a name + unique to that directory. The 250 Transfer Started response + must include the name generated. + + APPEND (with create) (APPE) + + This command causes the server-DTP to accept the data + transferred via the data connection and to store the data in + a file at the server site. If the file specified in the + pathname exists at the server site, then the data shall be + appended to that file; otherwise the file specified in the + pathname shall be created at the server site. + + ALLOCATE (ALLO) + + This command may be required by some servers to reserve + sufficient storage to accommodate the new file to be + transferred. The argument shall be a decimal integer + representing the number of bytes (using the logical byte + size) of storage to be reserved for the file. For files + sent with record or page structure a maximum record or page + size (in logical bytes) might also be necessary; this is + indicated by a decimal integer in a second argument field of + + +Postel & Reynolds [Page 30] + + + +RFC 959 October 1985 +File Transfer Protocol + + + the command. This second argument is optional, but when + present should be separated from the first by the three + Telnet characters R . This command shall be + followed by a STORe or APPEnd command. The ALLO command + should be treated as a NOOP (no operation) by those servers + which do not require that the maximum size of the file be + declared beforehand, and those servers interested in only + the maximum record or page size should accept a dummy value + in the first argument and ignore it. + + RESTART (REST) + + The argument field represents the server marker at which + file transfer is to be restarted. This command does not + cause file transfer but skips over the file to the specified + data checkpoint. This command shall be immediately followed + by the appropriate FTP service command which shall cause + file transfer to resume. + + RENAME FROM (RNFR) + + This command specifies the old pathname of the file which is + to be renamed. This command must be immediately followed by + a "rename to" command specifying the new file pathname. + + RENAME TO (RNTO) + + This command specifies the new pathname of the file + specified in the immediately preceding "rename from" + command. Together the two commands cause a file to be + renamed. + + ABORT (ABOR) + + This command tells the server to abort the previous FTP + service command and any associated transfer of data. The + abort command may require "special action", as discussed in + the Section on FTP Commands, to force recognition by the + server. No action is to be taken if the previous command + has been completed (including data transfer). The control + connection is not to be closed by the server, but the data + connection must be closed. + + There are two cases for the server upon receipt of this + command: (1) the FTP service command was already completed, + or (2) the FTP service command is still in progress. + + + +Postel & Reynolds [Page 31] + + + +RFC 959 October 1985 +File Transfer Protocol + + + In the first case, the server closes the data connection + (if it is open) and responds with a 226 reply, indicating + that the abort command was successfully processed. + + In the second case, the server aborts the FTP service in + progress and closes the data connection, returning a 426 + reply to indicate that the service request terminated + abnormally. The server then sends a 226 reply, + indicating that the abort command was successfully + processed. + + DELETE (DELE) + + This command causes the file specified in the pathname to be + deleted at the server site. If an extra level of protection + is desired (such as the query, "Do you really wish to + delete?"), it should be provided by the user-FTP process. + + REMOVE DIRECTORY (RMD) + + This command causes the directory specified in the pathname + to be removed as a directory (if the pathname is absolute) + or as a subdirectory of the current working directory (if + the pathname is relative). See Appendix II. + + MAKE DIRECTORY (MKD) + + This command causes the directory specified in the pathname + to be created as a directory (if the pathname is absolute) + or as a subdirectory of the current working directory (if + the pathname is relative). See Appendix II. + + PRINT WORKING DIRECTORY (PWD) + + This command causes the name of the current working + directory to be returned in the reply. See Appendix II. + + LIST (LIST) + + This command causes a list to be sent from the server to the + passive DTP. If the pathname specifies a directory or other + group of files, the server should transfer a list of files + in the specified directory. If the pathname specifies a + file then the server should send current information on the + file. A null argument implies the user's current working or + default directory. The data transfer is over the data + connection in type ASCII or type EBCDIC. (The user must + + +Postel & Reynolds [Page 32] + + + +RFC 959 October 1985 +File Transfer Protocol + + + ensure that the TYPE is appropriately ASCII or EBCDIC). + Since the information on a file may vary widely from system + to system, this information may be hard to use automatically + in a program, but may be quite useful to a human user. + + NAME LIST (NLST) + + This command causes a directory listing to be sent from + server to user site. The pathname should specify a + directory or other system-specific file group descriptor; a + null argument implies the current directory. The server + will return a stream of names of files and no other + information. The data will be transferred in ASCII or + EBCDIC type over the data connection as valid pathname + strings separated by or . (Again the user must + ensure that the TYPE is correct.) This command is intended + to return information that can be used by a program to + further process the files automatically. For example, in + the implementation of a "multiple get" function. + + SITE PARAMETERS (SITE) + + This command is used by the server to provide services + specific to his system that are essential to file transfer + but not sufficiently universal to be included as commands in + the protocol. The nature of these services and the + specification of their syntax can be stated in a reply to + the HELP SITE command. + + SYSTEM (SYST) + + This command is used to find out the type of operating + system at the server. The reply shall have as its first + word one of the system names listed in the current version + of the Assigned Numbers document [4]. + + STATUS (STAT) + + This command shall cause a status response to be sent over + the control connection in the form of a reply. The command + may be sent during a file transfer (along with the Telnet IP + and Synch signals--see the Section on FTP Commands) in which + case the server will respond with the status of the + operation in progress, or it may be sent between file + transfers. In the latter case, the command may have an + argument field. If the argument is a pathname, the command + is analogous to the "list" command except that data shall be + + +Postel & Reynolds [Page 33] + + + +RFC 959 October 1985 +File Transfer Protocol + + + transferred over the control connection. If a partial + pathname is given, the server may respond with a list of + file names or attributes associated with that specification. + If no argument is given, the server should return general + status information about the server FTP process. This + should include current values of all transfer parameters and + the status of connections. + + HELP (HELP) + + This command shall cause the server to send helpful + information regarding its implementation status over the + control connection to the user. The command may take an + argument (e.g., any command name) and return more specific + information as a response. The reply is type 211 or 214. + It is suggested that HELP be allowed before entering a USER + command. The server may use this reply to specify + site-dependent parameters, e.g., in response to HELP SITE. + + NOOP (NOOP) + + This command does not affect any parameters or previously + entered commands. It specifies no action other than that the + server send an OK reply. + + The File Transfer Protocol follows the specifications of the Telnet + protocol for all communications over the control connection. Since + the language used for Telnet communication may be a negotiated + option, all references in the next two sections will be to the + "Telnet language" and the corresponding "Telnet end-of-line code". + Currently, one may take these to mean NVT-ASCII and . No other + specifications of the Telnet protocol will be cited. + + FTP commands are "Telnet strings" terminated by the "Telnet end of + line code". The command codes themselves are alphabetic characters + terminated by the character (Space) if parameters follow and + Telnet-EOL otherwise. The command codes and the semantics of + commands are described in this section; the detailed syntax of + commands is specified in the Section on Commands, the reply sequences + are discussed in the Section on Sequencing of Commands and Replies, + and scenarios illustrating the use of commands are provided in the + Section on Typical FTP Scenarios. + + FTP commands may be partitioned as those specifying access-control + identifiers, data transfer parameters, or FTP service requests. + Certain commands (such as ABOR, STAT, QUIT) may be sent over the + control connection while a data transfer is in progress. Some + + +Postel & Reynolds [Page 34] + + + +RFC 959 October 1985 +File Transfer Protocol + + + servers may not be able to monitor the control and data connections + simultaneously, in which case some special action will be necessary + to get the server's attention. The following ordered format is + tentatively recommended: + + 1. User system inserts the Telnet "Interrupt Process" (IP) signal + in the Telnet stream. + + 2. User system sends the Telnet "Synch" signal. + + 3. User system inserts the command (e.g., ABOR) in the Telnet + stream. + + 4. Server PI, after receiving "IP", scans the Telnet stream for + EXACTLY ONE FTP command. + + (For other servers this may not be necessary but the actions listed + above should have no unusual effect.) + + 4.2. FTP REPLIES + + Replies to File Transfer Protocol commands are devised to ensure + the synchronization of requests and actions in the process of file + transfer, and to guarantee that the user process always knows the + state of the Server. Every command must generate at least one + reply, although there may be more than one; in the latter case, + the multiple replies must be easily distinguished. In addition, + some commands occur in sequential groups, such as USER, PASS and + ACCT, or RNFR and RNTO. The replies show the existence of an + intermediate state if all preceding commands have been successful. + A failure at any point in the sequence necessitates the repetition + of the entire sequence from the beginning. + + The details of the command-reply sequence are made explicit in + a set of state diagrams below. + + An FTP reply consists of a three digit number (transmitted as + three alphanumeric characters) followed by some text. The number + is intended for use by automata to determine what state to enter + next; the text is intended for the human user. It is intended + that the three digits contain enough encoded information that the + user-process (the User-PI) will not need to examine the text and + may either discard it or pass it on to the user, as appropriate. + In particular, the text may be server-dependent, so there are + likely to be varying texts for each reply code. + + A reply is defined to contain the 3-digit code, followed by Space + + +Postel & Reynolds [Page 35] + + + +RFC 959 October 1985 +File Transfer Protocol + + + , followed by one line of text (where some maximum line length + has been specified), and terminated by the Telnet end-of-line + code. There will be cases however, where the text is longer than + a single line. In these cases the complete text must be bracketed + so the User-process knows when it may stop reading the reply (i.e. + stop processing input on the control connection) and go do other + things. This requires a special format on the first line to + indicate that more than one line is coming, and another on the + last line to designate it as the last. At least one of these must + contain the appropriate reply code to indicate the state of the + transaction. To satisfy all factions, it was decided that both + the first and last line codes should be the same. + + Thus the format for multi-line replies is that the first line + will begin with the exact required reply code, followed + immediately by a Hyphen, "-" (also known as Minus), followed by + text. The last line will begin with the same code, followed + immediately by Space , optionally some text, and the Telnet + end-of-line code. + + For example: + 123-First line + Second line + 234 A line beginning with numbers + 123 The last line + + The user-process then simply needs to search for the second + occurrence of the same reply code, followed by (Space), at + the beginning of a line, and ignore all intermediary lines. If + an intermediary line begins with a 3-digit number, the Server + must pad the front to avoid confusion. + + This scheme allows standard system routines to be used for + reply information (such as for the STAT reply), with + "artificial" first and last lines tacked on. In rare cases + where these routines are able to generate three digits and a + Space at the beginning of any line, the beginning of each + text line should be offset by some neutral text, like Space. + + This scheme assumes that multi-line replies may not be nested. + + The three digits of the reply each have a special significance. + This is intended to allow a range of very simple to very + sophisticated responses by the user-process. The first digit + denotes whether the response is good, bad or incomplete. + (Referring to the state diagram), an unsophisticated user-process + will be able to determine its next action (proceed as planned, + + +Postel & Reynolds [Page 36] + + + +RFC 959 October 1985 +File Transfer Protocol + + + redo, retrench, etc.) by simply examining this first digit. A + user-process that wants to know approximately what kind of error + occurred (e.g. file system error, command syntax error) may + examine the second digit, reserving the third digit for the finest + gradation of information (e.g., RNTO command without a preceding + RNFR). + + There are five values for the first digit of the reply code: + + 1yz Positive Preliminary reply + + The requested action is being initiated; expect another + reply before proceeding with a new command. (The + user-process sending another command before the + completion reply would be in violation of protocol; but + server-FTP processes should queue any commands that + arrive while a preceding command is in progress.) This + type of reply can be used to indicate that the command + was accepted and the user-process may now pay attention + to the data connections, for implementations where + simultaneous monitoring is difficult. The server-FTP + process may send at most, one 1yz reply per command. + + 2yz Positive Completion reply + + The requested action has been successfully completed. A + new request may be initiated. + + 3yz Positive Intermediate reply + + The command has been accepted, but the requested action + is being held in abeyance, pending receipt of further + information. The user should send another command + specifying this information. This reply is used in + command sequence groups. + + 4yz Transient Negative Completion reply + + The command was not accepted and the requested action did + not take place, but the error condition is temporary and + the action may be requested again. The user should + return to the beginning of the command sequence, if any. + It is difficult to assign a meaning to "transient", + particularly when two distinct sites (Server- and + User-processes) have to agree on the interpretation. + Each reply in the 4yz category might have a slightly + different time value, but the intent is that the + + +Postel & Reynolds [Page 37] + + + +RFC 959 October 1985 +File Transfer Protocol + + + user-process is encouraged to try again. A rule of thumb + in determining if a reply fits into the 4yz or the 5yz + (Permanent Negative) category is that replies are 4yz if + the commands can be repeated without any change in + command form or in properties of the User or Server + (e.g., the command is spelled the same with the same + arguments used; the user does not change his file access + or user name; the server does not put up a new + implementation.) + + 5yz Permanent Negative Completion reply + + The command was not accepted and the requested action did + not take place. The User-process is discouraged from + repeating the exact request (in the same sequence). Even + some "permanent" error conditions can be corrected, so + the human user may want to direct his User-process to + reinitiate the command sequence by direct action at some + point in the future (e.g., after the spelling has been + changed, or the user has altered his directory status.) + + The following function groupings are encoded in the second + digit: + + x0z Syntax - These replies refer to syntax errors, + syntactically correct commands that don't fit any + functional category, unimplemented or superfluous + commands. + + x1z Information - These are replies to requests for + information, such as status or help. + + x2z Connections - Replies referring to the control and + data connections. + + x3z Authentication and accounting - Replies for the login + process and accounting procedures. + + x4z Unspecified as yet. + + x5z File system - These replies indicate the status of the + Server file system vis-a-vis the requested transfer or + other file system action. + + The third digit gives a finer gradation of meaning in each of + the function categories, specified by the second digit. The + list of replies below will illustrate this. Note that the text + + +Postel & Reynolds [Page 38] + + + +RFC 959 October 1985 +File Transfer Protocol + + + associated with each reply is recommended, rather than + mandatory, and may even change according to the command with + which it is associated. The reply codes, on the other hand, + must strictly follow the specifications in the last section; + that is, Server implementations should not invent new codes for + situations that are only slightly different from the ones + described here, but rather should adapt codes already defined. + + A command such as TYPE or ALLO whose successful execution + does not offer the user-process any new information will + cause a 200 reply to be returned. If the command is not + implemented by a particular Server-FTP process because it + has no relevance to that computer system, for example ALLO + at a TOPS20 site, a Positive Completion reply is still + desired so that the simple User-process knows it can proceed + with its course of action. A 202 reply is used in this case + with, for example, the reply text: "No storage allocation + necessary." If, on the other hand, the command requests a + non-site-specific action and is unimplemented, the response + is 502. A refinement of that is the 504 reply for a command + that is implemented, but that requests an unimplemented + parameter. + + 4.2.1 Reply Codes by Function Groups + + 200 Command okay. + 500 Syntax error, command unrecognized. + This may include errors such as command line too long. + 501 Syntax error in parameters or arguments. + 202 Command not implemented, superfluous at this site. + 502 Command not implemented. + 503 Bad sequence of commands. + 504 Command not implemented for that parameter. + + + + + + + + + + + + + + + + +Postel & Reynolds [Page 39] + + + +RFC 959 October 1985 +File Transfer Protocol + + + 110 Restart marker reply. + In this case, the text is exact and not left to the + particular implementation; it must read: + MARK yyyy = mmmm + Where yyyy is User-process data stream marker, and mmmm + server's equivalent marker (note the spaces between markers + and "="). + 211 System status, or system help reply. + 212 Directory status. + 213 File status. + 214 Help message. + On how to use the server or the meaning of a particular + non-standard command. This reply is useful only to the + human user. + 215 NAME system type. + Where NAME is an official system name from the list in the + Assigned Numbers document. + + 120 Service ready in nnn minutes. + 220 Service ready for new user. + 221 Service closing control connection. + Logged out if appropriate. + 421 Service not available, closing control connection. + This may be a reply to any command if the service knows it + must shut down. + 125 Data connection already open; transfer starting. + 225 Data connection open; no transfer in progress. + 425 Can't open data connection. + 226 Closing data connection. + Requested file action successful (for example, file + transfer or file abort). + 426 Connection closed; transfer aborted. + 227 Entering Passive Mode (h1,h2,h3,h4,p1,p2). + + 230 User logged in, proceed. + 530 Not logged in. + 331 User name okay, need password. + 332 Need account for login. + 532 Need account for storing files. + + + + + + + + + + +Postel & Reynolds [Page 40] + + + +RFC 959 October 1985 +File Transfer Protocol + + + 150 File status okay; about to open data connection. + 250 Requested file action okay, completed. + 257 "PATHNAME" created. + 350 Requested file action pending further information. + 450 Requested file action not taken. + File unavailable (e.g., file busy). + 550 Requested action not taken. + File unavailable (e.g., file not found, no access). + 451 Requested action aborted. Local error in processing. + 551 Requested action aborted. Page type unknown. + 452 Requested action not taken. + Insufficient storage space in system. + 552 Requested file action aborted. + Exceeded storage allocation (for current directory or + dataset). + 553 Requested action not taken. + File name not allowed. + + + 4.2.2 Numeric Order List of Reply Codes + + 110 Restart marker reply. + In this case, the text is exact and not left to the + particular implementation; it must read: + MARK yyyy = mmmm + Where yyyy is User-process data stream marker, and mmmm + server's equivalent marker (note the spaces between markers + and "="). + 120 Service ready in nnn minutes. + 125 Data connection already open; transfer starting. + 150 File status okay; about to open data connection. + + + + + + + + + + + + + + + + + + +Postel & Reynolds [Page 41] + + + +RFC 959 October 1985 +File Transfer Protocol + + + 200 Command okay. + 202 Command not implemented, superfluous at this site. + 211 System status, or system help reply. + 212 Directory status. + 213 File status. + 214 Help message. + On how to use the server or the meaning of a particular + non-standard command. This reply is useful only to the + human user. + 215 NAME system type. + Where NAME is an official system name from the list in the + Assigned Numbers document. + 220 Service ready for new user. + 221 Service closing control connection. + Logged out if appropriate. + 225 Data connection open; no transfer in progress. + 226 Closing data connection. + Requested file action successful (for example, file + transfer or file abort). + 227 Entering Passive Mode (h1,h2,h3,h4,p1,p2). + 230 User logged in, proceed. + 250 Requested file action okay, completed. + 257 "PATHNAME" created. + + 331 User name okay, need password. + 332 Need account for login. + 350 Requested file action pending further information. + + 421 Service not available, closing control connection. + This may be a reply to any command if the service knows it + must shut down. + 425 Can't open data connection. + 426 Connection closed; transfer aborted. + 450 Requested file action not taken. + File unavailable (e.g., file busy). + 451 Requested action aborted: local error in processing. + 452 Requested action not taken. + Insufficient storage space in system. + + + + + + + + + + + +Postel & Reynolds [Page 42] + + + +RFC 959 October 1985 +File Transfer Protocol + + + 500 Syntax error, command unrecognized. + This may include errors such as command line too long. + 501 Syntax error in parameters or arguments. + 502 Command not implemented. + 503 Bad sequence of commands. + 504 Command not implemented for that parameter. + 530 Not logged in. + 532 Need account for storing files. + 550 Requested action not taken. + File unavailable (e.g., file not found, no access). + 551 Requested action aborted: page type unknown. + 552 Requested file action aborted. + Exceeded storage allocation (for current directory or + dataset). + 553 Requested action not taken. + File name not allowed. + + +5. DECLARATIVE SPECIFICATIONS + + 5.1. MINIMUM IMPLEMENTATION + + In order to make FTP workable without needless error messages, the + following minimum implementation is required for all servers: + + TYPE - ASCII Non-print + MODE - Stream + STRUCTURE - File, Record + COMMANDS - USER, QUIT, PORT, + TYPE, MODE, STRU, + for the default values + RETR, STOR, + NOOP. + + The default values for transfer parameters are: + + TYPE - ASCII Non-print + MODE - Stream + STRU - File + + All hosts must accept the above as the standard defaults. + + + + + + + + +Postel & Reynolds [Page 43] + + + +RFC 959 October 1985 +File Transfer Protocol + + + 5.2. CONNECTIONS + + The server protocol interpreter shall "listen" on Port L. The + user or user protocol interpreter shall initiate the full-duplex + control connection. Server- and user- processes should follow the + conventions of the Telnet protocol as specified in the + ARPA-Internet Protocol Handbook [1]. Servers are under no + obligation to provide for editing of command lines and may require + that it be done in the user host. The control connection shall be + closed by the server at the user's request after all transfers and + replies are completed. + + The user-DTP must "listen" on the specified data port; this may be + the default user port (U) or a port specified in the PORT command. + The server shall initiate the data connection from his own default + data port (L-1) using the specified user data port. The direction + of the transfer and the port used will be determined by the FTP + service command. + + Note that all FTP implementation must support data transfer using + the default port, and that only the USER-PI may initiate the use + of non-default ports. + + When data is to be transferred between two servers, A and B (refer + to Figure 2), the user-PI, C, sets up control connections with + both server-PI's. One of the servers, say A, is then sent a PASV + command telling him to "listen" on his data port rather than + initiate a connection when he receives a transfer service command. + When the user-PI receives an acknowledgment to the PASV command, + which includes the identity of the host and port being listened + on, the user-PI then sends A's port, a, to B in a PORT command; a + reply is returned. The user-PI may then send the corresponding + service commands to A and B. Server B initiates the connection + and the transfer proceeds. The command-reply sequence is listed + below where the messages are vertically synchronous but + horizontally asynchronous: + + + + + + + + + + + + + +Postel & Reynolds [Page 44] + + + +RFC 959 October 1985 +File Transfer Protocol + + + User-PI - Server A User-PI - Server B + ------------------ ------------------ + + C->A : Connect C->B : Connect + C->A : PASV + A->C : 227 Entering Passive Mode. A1,A2,A3,A4,a1,a2 + C->B : PORT A1,A2,A3,A4,a1,a2 + B->C : 200 Okay + C->A : STOR C->B : RETR + B->A : Connect to HOST-A, PORT-a + + Figure 3 + + The data connection shall be closed by the server under the + conditions described in the Section on Establishing Data + Connections. If the data connection is to be closed following a + data transfer where closing the connection is not required to + indicate the end-of-file, the server must do so immediately. + Waiting until after a new transfer command is not permitted + because the user-process will have already tested the data + connection to see if it needs to do a "listen"; (remember that the + user must "listen" on a closed data port BEFORE sending the + transfer request). To prevent a race condition here, the server + sends a reply (226) after closing the data connection (or if the + connection is left open, a "file transfer completed" reply (250) + and the user-PI should wait for one of these replies before + issuing a new transfer command). + + Any time either the user or server see that the connection is + being closed by the other side, it should promptly read any + remaining data queued on the connection and issue the close on its + own side. + + 5.3. COMMANDS + + The commands are Telnet character strings transmitted over the + control connections as described in the Section on FTP Commands. + The command functions and semantics are described in the Section + on Access Control Commands, Transfer Parameter Commands, FTP + Service Commands, and Miscellaneous Commands. The command syntax + is specified here. + + The commands begin with a command code followed by an argument + field. The command codes are four or fewer alphabetic characters. + Upper and lower case alphabetic characters are to be treated + identically. Thus, any of the following may represent the + retrieve command: + + +Postel & Reynolds [Page 45] + + + +RFC 959 October 1985 +File Transfer Protocol + + + RETR Retr retr ReTr rETr + + This also applies to any symbols representing parameter values, + such as A or a for ASCII TYPE. The command codes and the argument + fields are separated by one or more spaces. + + The argument field consists of a variable length character string + ending with the character sequence (Carriage Return, Line + Feed) for NVT-ASCII representation; for other negotiated languages + a different end of line character might be used. It should be + noted that the server is to take no action until the end of line + code is received. + + The syntax is specified below in NVT-ASCII. All characters in the + argument field are ASCII characters including any ASCII + represented decimal integers. Square brackets denote an optional + argument field. If the option is not taken, the appropriate + default is implied. + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +Postel & Reynolds [Page 46] + + + +RFC 959 October 1985 +File Transfer Protocol + + + 5.3.1. FTP COMMANDS + + The following are the FTP commands: + + USER + PASS + ACCT + CWD + CDUP + SMNT + QUIT + REIN + PORT + PASV + TYPE + STRU + MODE + RETR + STOR + STOU + APPE + ALLO + [ R ] + REST + RNFR + RNTO + ABOR + DELE + RMD + MKD + PWD + LIST [ ] + NLST [ ] + SITE + SYST + STAT [ ] + HELP [ ] + NOOP + + + + + + + + + + + +Postel & Reynolds [Page 47] + + + +RFC 959 October 1985 +File Transfer Protocol + + + 5.3.2. FTP COMMAND ARGUMENTS + + The syntax of the above argument fields (using BNF notation + where applicable) is: + + ::= + ::= + ::= + ::= | + ::= any of the 128 ASCII characters except and + + ::= + ::= | + ::= printable characters, any + ASCII code 33 through 126 + ::= + ::= , + ::= ,,, + ::= , + ::= any decimal integer 1 through 255 + ::= N | T | C + ::= A [ ] + | E [ ] + | I + | L + ::= F | R | P + ::= S | B | C + ::= + ::= any decimal integer + + + + + + + + + + + + + + + + + + + + +Postel & Reynolds [Page 48] + + + +RFC 959 October 1985 +File Transfer Protocol + + + 5.4. SEQUENCING OF COMMANDS AND REPLIES + + The communication between the user and server is intended to be an + alternating dialogue. As such, the user issues an FTP command and + the server responds with a prompt primary reply. The user should + wait for this initial primary success or failure response before + sending further commands. + + Certain commands require a second reply for which the user should + also wait. These replies may, for example, report on the progress + or completion of file transfer or the closing of the data + connection. They are secondary replies to file transfer commands. + + One important group of informational replies is the connection + greetings. Under normal circumstances, a server will send a 220 + reply, "awaiting input", when the connection is completed. The + user should wait for this greeting message before sending any + commands. If the server is unable to accept input right away, a + 120 "expected delay" reply should be sent immediately and a 220 + reply when ready. The user will then know not to hang up if there + is a delay. + + Spontaneous Replies + + Sometimes "the system" spontaneously has a message to be sent + to a user (usually all users). For example, "System going down + in 15 minutes". There is no provision in FTP for such + spontaneous information to be sent from the server to the user. + It is recommended that such information be queued in the + server-PI and delivered to the user-PI in the next reply + (possibly making it a multi-line reply). + + The table below lists alternative success and failure replies for + each command. These must be strictly adhered to; a server may + substitute text in the replies, but the meaning and action implied + by the code numbers and by the specific command reply sequence + cannot be altered. + + Command-Reply Sequences + + In this section, the command-reply sequence is presented. Each + command is listed with its possible replies; command groups are + listed together. Preliminary replies are listed first (with + their succeeding replies indented and under them), then + positive and negative completion, and finally intermediary + + + + +Postel & Reynolds [Page 49] + + + +RFC 959 October 1985 +File Transfer Protocol + + + replies with the remaining commands from the sequence + following. This listing forms the basis for the state + diagrams, which will be presented separately. + + Connection Establishment + 120 + 220 + 220 + 421 + Login + USER + 230 + 530 + 500, 501, 421 + 331, 332 + PASS + 230 + 202 + 530 + 500, 501, 503, 421 + 332 + ACCT + 230 + 202 + 530 + 500, 501, 503, 421 + CWD + 250 + 500, 501, 502, 421, 530, 550 + CDUP + 200 + 500, 501, 502, 421, 530, 550 + SMNT + 202, 250 + 500, 501, 502, 421, 530, 550 + Logout + REIN + 120 + 220 + 220 + 421 + 500, 502 + QUIT + 221 + 500 + + + + +Postel & Reynolds [Page 50] + + + +RFC 959 October 1985 +File Transfer Protocol + + + Transfer parameters + PORT + 200 + 500, 501, 421, 530 + PASV + 227 + 500, 501, 502, 421, 530 + MODE + 200 + 500, 501, 504, 421, 530 + TYPE + 200 + 500, 501, 504, 421, 530 + STRU + 200 + 500, 501, 504, 421, 530 + File action commands + ALLO + 200 + 202 + 500, 501, 504, 421, 530 + REST + 500, 501, 502, 421, 530 + 350 + STOR + 125, 150 + (110) + 226, 250 + 425, 426, 451, 551, 552 + 532, 450, 452, 553 + 500, 501, 421, 530 + STOU + 125, 150 + (110) + 226, 250 + 425, 426, 451, 551, 552 + 532, 450, 452, 553 + 500, 501, 421, 530 + RETR + 125, 150 + (110) + 226, 250 + 425, 426, 451 + 450, 550 + 500, 501, 421, 530 + + + + +Postel & Reynolds [Page 51] + + + +RFC 959 October 1985 +File Transfer Protocol + + + LIST + 125, 150 + 226, 250 + 425, 426, 451 + 450 + 500, 501, 502, 421, 530 + NLST + 125, 150 + 226, 250 + 425, 426, 451 + 450 + 500, 501, 502, 421, 530 + APPE + 125, 150 + (110) + 226, 250 + 425, 426, 451, 551, 552 + 532, 450, 550, 452, 553 + 500, 501, 502, 421, 530 + RNFR + 450, 550 + 500, 501, 502, 421, 530 + 350 + RNTO + 250 + 532, 553 + 500, 501, 502, 503, 421, 530 + DELE + 250 + 450, 550 + 500, 501, 502, 421, 530 + RMD + 250 + 500, 501, 502, 421, 530, 550 + MKD + 257 + 500, 501, 502, 421, 530, 550 + PWD + 257 + 500, 501, 502, 421, 550 + ABOR + 225, 226 + 500, 501, 502, 421 + + + + + + +Postel & Reynolds [Page 52] + + + +RFC 959 October 1985 +File Transfer Protocol + + + Informational commands + SYST + 215 + 500, 501, 502, 421 + STAT + 211, 212, 213 + 450 + 500, 501, 502, 421, 530 + HELP + 211, 214 + 500, 501, 502, 421 + Miscellaneous commands + SITE + 200 + 202 + 500, 501, 530 + NOOP + 200 + 500 421 + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +Postel & Reynolds [Page 53] + + + +RFC 959 October 1985 +File Transfer Protocol + + +6. STATE DIAGRAMS + + Here we present state diagrams for a very simple minded FTP + implementation. Only the first digit of the reply codes is used. + There is one state diagram for each group of FTP commands or command + sequences. + + The command groupings were determined by constructing a model for + each command then collecting together the commands with structurally + identical models. + + For each command or command sequence there are three possible + outcomes: success (S), failure (F), and error (E). In the state + diagrams below we use the symbol B for "begin", and the symbol W for + "wait for reply". + + We first present the diagram that represents the largest group of FTP + commands: + + + 1,3 +---+ + ----------->| E | + | +---+ + | + +---+ cmd +---+ 2 +---+ + | B |---------->| W |---------->| S | + +---+ +---+ +---+ + | + | 4,5 +---+ + ----------->| F | + +---+ + + + This diagram models the commands: + + ABOR, ALLO, DELE, CWD, CDUP, SMNT, HELP, MODE, NOOP, PASV, + QUIT, SITE, PORT, SYST, STAT, RMD, MKD, PWD, STRU, and TYPE. + + + + + + + + + + + + +Postel & Reynolds [Page 54] + + + +RFC 959 October 1985 +File Transfer Protocol + + + The other large group of commands is represented by a very similar + diagram: + + + 3 +---+ + ----------->| E | + | +---+ + | + +---+ cmd +---+ 2 +---+ + | B |---------->| W |---------->| S | + +---+ --->+---+ +---+ + | | | + | | | 4,5 +---+ + | 1 | ----------->| F | + ----- +---+ + + + This diagram models the commands: + + APPE, LIST, NLST, REIN, RETR, STOR, and STOU. + + Note that this second model could also be used to represent the first + group of commands, the only difference being that in the first group + the 100 series replies are unexpected and therefore treated as error, + while the second group expects (some may require) 100 series replies. + Remember that at most, one 100 series reply is allowed per command. + + The remaining diagrams model command sequences, perhaps the simplest + of these is the rename sequence: + + + +---+ RNFR +---+ 1,2 +---+ + | B |---------->| W |---------->| E | + +---+ +---+ -->+---+ + | | | + 3 | | 4,5 | + -------------- ------ | + | | | +---+ + | ------------->| S | + | | 1,3 | | +---+ + | 2| -------- + | | | | + V | | | + +---+ RNTO +---+ 4,5 ----->+---+ + | |---------->| W |---------->| F | + +---+ +---+ +---+ + + + +Postel & Reynolds [Page 55] + + + +RFC 959 October 1985 +File Transfer Protocol + + + The next diagram is a simple model of the Restart command: + + + +---+ REST +---+ 1,2 +---+ + | B |---------->| W |---------->| E | + +---+ +---+ -->+---+ + | | | + 3 | | 4,5 | + -------------- ------ | + | | | +---+ + | ------------->| S | + | | 3 | | +---+ + | 2| -------- + | | | | + V | | | + +---+ cmd +---+ 4,5 ----->+---+ + | |---------->| W |---------->| F | + +---+ -->+---+ +---+ + | | + | 1 | + ------ + + + Where "cmd" is APPE, STOR, or RETR. + + We note that the above three models are similar. The Restart differs + from the Rename two only in the treatment of 100 series replies at + the second stage, while the second group expects (some may require) + 100 series replies. Remember that at most, one 100 series reply is + allowed per command. + + + + + + + + + + + + + + + + + + + +Postel & Reynolds [Page 56] + + + +RFC 959 October 1985 +File Transfer Protocol + + + The most complicated diagram is for the Login sequence: + + + 1 + +---+ USER +---+------------->+---+ + | B |---------->| W | 2 ---->| E | + +---+ +---+------ | -->+---+ + | | | | | + 3 | | 4,5 | | | + -------------- ----- | | | + | | | | | + | | | | | + | --------- | + | 1| | | | + V | | | | + +---+ PASS +---+ 2 | ------>+---+ + | |---------->| W |------------->| S | + +---+ +---+ ---------->+---+ + | | | | | + 3 | |4,5| | | + -------------- -------- | + | | | | | + | | | | | + | ----------- + | 1,3| | | | + V | 2| | | + +---+ ACCT +---+-- | ----->+---+ + | |---------->| W | 4,5 -------->| F | + +---+ +---+------------->+---+ + + + + + + + + + + + + + + + + + + + + +Postel & Reynolds [Page 57] + + + +RFC 959 October 1985 +File Transfer Protocol + + + Finally, we present a generalized diagram that could be used to model + the command and reply interchange: + + + ------------------------------------ + | | + Begin | | + | V | + | +---+ cmd +---+ 2 +---+ | + -->| |------->| |---------->| | | + | | | W | | S |-----| + -->| | -->| |----- | | | + | +---+ | +---+ 4,5 | +---+ | + | | | | | | | + | | | 1| |3 | +---+ | + | | | | | | | | | + | | ---- | ---->| F |----- + | | | | | + | | | +---+ + ------------------- + | + | + V + End + + + + + + + + + + + + + + + + + + + + + + + + + +Postel & Reynolds [Page 58] + + + +RFC 959 October 1985 +File Transfer Protocol + + +7. TYPICAL FTP SCENARIO + + User at host U wanting to transfer files to/from host S: + + In general, the user will communicate to the server via a mediating + user-FTP process. The following may be a typical scenario. The + user-FTP prompts are shown in parentheses, '---->' represents + commands from host U to host S, and '<----' represents replies from + host S to host U. + + LOCAL COMMANDS BY USER ACTION INVOLVED + + ftp (host) multics Connect to host S, port L, + establishing control connections. + <---- 220 Service ready . + username Doe USER Doe----> + <---- 331 User name ok, + need password. + password mumble PASS mumble----> + <---- 230 User logged in. + retrieve (local type) ASCII + (local pathname) test 1 User-FTP opens local file in ASCII. + (for. pathname) test.pl1 RETR test.pl1 ----> + <---- 150 File status okay; + about to open data + connection. + Server makes data connection + to port U. + + <---- 226 Closing data connection, + file transfer successful. + type Image TYPE I ----> + <---- 200 Command OK + store (local type) image + (local pathname) file dump User-FTP opens local file in Image. + (for.pathname) >udd>cn>fd STOR >udd>cn>fd ----> + <---- 550 Access denied + terminate QUIT ----> + Server closes all + connections. + +8. CONNECTION ESTABLISHMENT + + The FTP control connection is established via TCP between the user + process port U and the server process port L. This protocol is + assigned the service port 21 (25 octal), that is L=21. + + + +Postel & Reynolds [Page 59] + + + +RFC 959 October 1985 +File Transfer Protocol + + +APPENDIX I - PAGE STRUCTURE + + The need for FTP to support page structure derives principally from + the need to support efficient transmission of files between TOPS-20 + systems, particularly the files used by NLS. + + The file system of TOPS-20 is based on the concept of pages. The + operating system is most efficient at manipulating files as pages. + The operating system provides an interface to the file system so that + many applications view files as sequential streams of characters. + However, a few applications use the underlying page structures + directly, and some of these create holey files. + + A TOPS-20 disk file consists of four things: a pathname, a page + table, a (possibly empty) set of pages, and a set of attributes. + + The pathname is specified in the RETR or STOR command. It includes + the directory name, file name, file name extension, and generation + number. + + The page table contains up to 2**18 entries. Each entry may be + EMPTY, or may point to a page. If it is not empty, there are also + some page-specific access bits; not all pages of a file need have the + same access protection. + + A page is a contiguous set of 512 words of 36 bits each. + + The attributes of the file, in the File Descriptor Block (FDB), + contain such things as creation time, write time, read time, writer's + byte-size, end-of-file pointer, count of reads and writes, backup + system tape numbers, etc. + + Note that there is NO requirement that entries in the page table be + contiguous. There may be empty page table slots between occupied + ones. Also, the end of file pointer is simply a number. There is no + requirement that it in fact point at the "last" datum in the file. + Ordinary sequential I/O calls in TOPS-20 will cause the end of file + pointer to be left after the last datum written, but other operations + may cause it not to be so, if a particular programming system so + requires. + + In fact, in both of these special cases, "holey" files and + end-of-file pointers NOT at the end of the file, occur with NLS data + files. + + + + + +Postel & Reynolds [Page 60] + + + +RFC 959 October 1985 +File Transfer Protocol + + + The TOPS-20 paged files can be sent with the FTP transfer parameters: + TYPE L 36, STRU P, and MODE S (in fact, any mode could be used). + + Each page of information has a header. Each header field, which is a + logical byte, is a TOPS-20 word, since the TYPE is L 36. + + The header fields are: + + Word 0: Header Length. + + The header length is 5. + + Word 1: Page Index. + + If the data is a disk file page, this is the number of that + page in the file's page map. Empty pages (holes) in the file + are simply not sent. Note that a hole is NOT the same as a + page of zeros. + + Word 2: Data Length. + + The number of data words in this page, following the header. + Thus, the total length of the transmission unit is the Header + Length plus the Data Length. + + Word 3: Page Type. + + A code for what type of chunk this is. A data page is type 3, + the FDB page is type 2. + + Word 4: Page Access Control. + + The access bits associated with the page in the file's page + map. (This full word quantity is put into AC2 of an SPACS by + the program reading from net to disk.) + + After the header are Data Length data words. Data Length is + currently either 512 for a data page or 31 for an FDB. Trailing + zeros in a disk file page may be discarded, making Data Length less + than 512 in that case. + + + + + + + + + +Postel & Reynolds [Page 61] + + + +RFC 959 October 1985 +File Transfer Protocol + + +APPENDIX II - DIRECTORY COMMANDS + + Since UNIX has a tree-like directory structure in which directories + are as easy to manipulate as ordinary files, it is useful to expand + the FTP servers on these machines to include commands which deal with + the creation of directories. Since there are other hosts on the + ARPA-Internet which have tree-like directories (including TOPS-20 and + Multics), these commands are as general as possible. + + Four directory commands have been added to FTP: + + MKD pathname + + Make a directory with the name "pathname". + + RMD pathname + + Remove the directory with the name "pathname". + + PWD + + Print the current working directory name. + + CDUP + + Change to the parent of the current working directory. + + The "pathname" argument should be created (removed) as a + subdirectory of the current working directory, unless the "pathname" + string contains sufficient information to specify otherwise to the + server, e.g., "pathname" is an absolute pathname (in UNIX and + Multics), or pathname is something like "" to + TOPS-20. + + REPLY CODES + + The CDUP command is a special case of CWD, and is included to + simplify the implementation of programs for transferring directory + trees between operating systems having different syntaxes for + naming the parent directory. The reply codes for CDUP be + identical to the reply codes of CWD. + + The reply codes for RMD be identical to the reply codes for its + file analogue, DELE. + + The reply codes for MKD, however, are a bit more complicated. A + freshly created directory will probably be the object of a future + + +Postel & Reynolds [Page 62] + + + +RFC 959 October 1985 +File Transfer Protocol + + + CWD command. Unfortunately, the argument to MKD may not always be + a suitable argument for CWD. This is the case, for example, when + a TOPS-20 subdirectory is created by giving just the subdirectory + name. That is, with a TOPS-20 server FTP, the command sequence + + MKD MYDIR + CWD MYDIR + + will fail. The new directory may only be referred to by its + "absolute" name; e.g., if the MKD command above were issued while + connected to the directory , the new subdirectory + could only be referred to by the name . + + Even on UNIX and Multics, however, the argument given to MKD may + not be suitable. If it is a "relative" pathname (i.e., a pathname + which is interpreted relative to the current directory), the user + would need to be in the same current directory in order to reach + the subdirectory. Depending on the application, this may be + inconvenient. It is not very robust in any case. + + To solve these problems, upon successful completion of an MKD + command, the server should return a line of the form: + + 257"" + + That is, the server will tell the user what string to use when + referring to the created directory. The directory name can + contain any character; embedded double-quotes should be escaped by + double-quotes (the "quote-doubling" convention). + + For example, a user connects to the directory /usr/dm, and creates + a subdirectory, named pathname: + + CWD /usr/dm + 200 directory changed to /usr/dm + MKD pathname + 257 "/usr/dm/pathname" directory created + + An example with an embedded double quote: + + MKD foo"bar + 257 "/usr/dm/foo""bar" directory created + CWD /usr/dm/foo"bar + 200 directory changed to /usr/dm/foo"bar + + + + + +Postel & Reynolds [Page 63] + + + +RFC 959 October 1985 +File Transfer Protocol + + + The prior existence of a subdirectory with the same name is an + error, and the server must return an "access denied" error reply + in that case. + + CWD /usr/dm + 200 directory changed to /usr/dm + MKD pathname + 521-"/usr/dm/pathname" directory already exists; + 521 taking no action. + + The failure replies for MKD are analogous to its file creating + cousin, STOR. Also, an "access denied" return is given if a file + name with the same name as the subdirectory will conflict with the + creation of the subdirectory (this is a problem on UNIX, but + shouldn't be one on TOPS-20). + + Essentially because the PWD command returns the same type of + information as the successful MKD command, the successful PWD + command uses the 257 reply code as well. + + SUBTLETIES + + Because these commands will be most useful in transferring + subtrees from one machine to another, carefully observe that the + argument to MKD is to be interpreted as a sub-directory of the + current working directory, unless it contains enough information + for the destination host to tell otherwise. A hypothetical + example of its use in the TOPS-20 world: + + CWD + 200 Working directory changed + MKD overrainbow + 257 "" directory created + CWD overrainbow + 431 No such directory + CWD + 200 Working directory changed + + CWD + 200 Working directory changed to + MKD + 257 "" directory created + CWD + + Note that the first example results in a subdirectory of the + connected directory. In contrast, the argument in the second + example contains enough information for TOPS-20 to tell that the + + +Postel & Reynolds [Page 64] + + + +RFC 959 October 1985 +File Transfer Protocol + + + directory is a top-level directory. Note also that + in the first example the user "violated" the protocol by + attempting to access the freshly created directory with a name + other than the one returned by TOPS-20. Problems could have + resulted in this case had there been an directory; + this is an ambiguity inherent in some TOPS-20 implementations. + Similar considerations apply to the RMD command. The point is + this: except where to do so would violate a host's conventions for + denoting relative versus absolute pathnames, the host should treat + the operands of the MKD and RMD commands as subdirectories. The + 257 reply to the MKD command must always contain the absolute + pathname of the created directory. + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +Postel & Reynolds [Page 65] + + + +RFC 959 October 1985 +File Transfer Protocol + + +APPENDIX III - RFCs on FTP + + Bhushan, Abhay, "A File Transfer Protocol", RFC 114 (NIC 5823), + MIT-Project MAC, 16 April 1971. + + Harslem, Eric, and John Heafner, "Comments on RFC 114 (A File + Transfer Protocol)", RFC 141 (NIC 6726), RAND, 29 April 1971. + + Bhushan, Abhay, et al, "The File Transfer Protocol", RFC 172 + (NIC 6794), MIT-Project MAC, 23 June 1971. + + Braden, Bob, "Comments on DTP and FTP Proposals", RFC 238 (NIC 7663), + UCLA/CCN, 29 September 1971. + + Bhushan, Abhay, et al, "The File Transfer Protocol", RFC 265 + (NIC 7813), MIT-Project MAC, 17 November 1971. + + McKenzie, Alex, "A Suggested Addition to File Transfer Protocol", + RFC 281 (NIC 8163), BBN, 8 December 1971. + + Bhushan, Abhay, "The Use of "Set Data Type" Transaction in File + Transfer Protocol", RFC 294 (NIC 8304), MIT-Project MAC, + 25 January 1972. + + Bhushan, Abhay, "The File Transfer Protocol", RFC 354 (NIC 10596), + MIT-Project MAC, 8 July 1972. + + Bhushan, Abhay, "Comments on the File Transfer Protocol (RFC 354)", + RFC 385 (NIC 11357), MIT-Project MAC, 18 August 1972. + + Hicks, Greg, "User FTP Documentation", RFC 412 (NIC 12404), Utah, + 27 November 1972. + + Bhushan, Abhay, "File Transfer Protocol (FTP) Status and Further + Comments", RFC 414 (NIC 12406), MIT-Project MAC, 20 November 1972. + + Braden, Bob, "Comments on File Transfer Protocol", RFC 430 + (NIC 13299), UCLA/CCN, 7 February 1973. + + Thomas, Bob, and Bob Clements, "FTP Server-Server Interaction", + RFC 438 (NIC 13770), BBN, 15 January 1973. + + Braden, Bob, "Print Files in FTP", RFC 448 (NIC 13299), UCLA/CCN, + 27 February 1973. + + McKenzie, Alex, "File Transfer Protocol", RFC 454 (NIC 14333), BBN, + 16 February 1973. + + +Postel & Reynolds [Page 66] + + + +RFC 959 October 1985 +File Transfer Protocol + + + Bressler, Bob, and Bob Thomas, "Mail Retrieval via FTP", RFC 458 + (NIC 14378), BBN-NET and BBN-TENEX, 20 February 1973. + + Neigus, Nancy, "File Transfer Protocol", RFC 542 (NIC 17759), BBN, + 12 July 1973. + + Krilanovich, Mark, and George Gregg, "Comments on the File Transfer + Protocol", RFC 607 (NIC 21255), UCSB, 7 January 1974. + + Pogran, Ken, and Nancy Neigus, "Response to RFC 607 - Comments on the + File Transfer Protocol", RFC 614 (NIC 21530), BBN, 28 January 1974. + + Krilanovich, Mark, George Gregg, Wayne Hathaway, and Jim White, + "Comments on the File Transfer Protocol", RFC 624 (NIC 22054), UCSB, + Ames Research Center, SRI-ARC, 28 February 1974. + + Bhushan, Abhay, "FTP Comments and Response to RFC 430", RFC 463 + (NIC 14573), MIT-DMCG, 21 February 1973. + + Braden, Bob, "FTP Data Compression", RFC 468 (NIC 14742), UCLA/CCN, + 8 March 1973. + + Bhushan, Abhay, "FTP and Network Mail System", RFC 475 (NIC 14919), + MIT-DMCG, 6 March 1973. + + Bressler, Bob, and Bob Thomas "FTP Server-Server Interaction - II", + RFC 478 (NIC 14947), BBN-NET and BBN-TENEX, 26 March 1973. + + White, Jim, "Use of FTP by the NIC Journal", RFC 479 (NIC 14948), + SRI-ARC, 8 March 1973. + + White, Jim, "Host-Dependent FTP Parameters", RFC 480 (NIC 14949), + SRI-ARC, 8 March 1973. + + Padlipsky, Mike, "An FTP Command-Naming Problem", RFC 506 + (NIC 16157), MIT-Multics, 26 June 1973. + + Day, John, "Memo to FTP Group (Proposal for File Access Protocol)", + RFC 520 (NIC 16819), Illinois, 25 June 1973. + + Merryman, Robert, "The UCSD-CC Server-FTP Facility", RFC 532 + (NIC 17451), UCSD-CC, 22 June 1973. + + Braden, Bob, "TENEX FTP Problem", RFC 571 (NIC 18974), UCLA/CCN, + 15 November 1973. + + + + +Postel & Reynolds [Page 67] + + + +RFC 959 October 1985 +File Transfer Protocol + + + McKenzie, Alex, and Jon Postel, "Telnet and FTP Implementation - + Schedule Change", RFC 593 (NIC 20615), BBN and MITRE, + 29 November 1973. + + Sussman, Julie, "FTP Error Code Usage for More Reliable Mail + Service", RFC 630 (NIC 30237), BBN, 10 April 1974. + + Postel, Jon, "Revised FTP Reply Codes", RFC 640 (NIC 30843), + UCLA/NMC, 5 June 1974. + + Harvey, Brian, "Leaving Well Enough Alone", RFC 686 (NIC 32481), + SU-AI, 10 May 1975. + + Harvey, Brian, "One More Try on the FTP", RFC 691 (NIC 32700), SU-AI, + 28 May 1975. + + Lieb, J., "CWD Command of FTP", RFC 697 (NIC 32963), 14 July 1975. + + Harrenstien, Ken, "FTP Extension: XSEN", RFC 737 (NIC 42217), SRI-KL, + 31 October 1977. + + Harrenstien, Ken, "FTP Extension: XRSQ/XRCP", RFC 743 (NIC 42758), + SRI-KL, 30 December 1977. + + Lebling, P. David, "Survey of FTP Mail and MLFL", RFC 751, MIT, + 10 December 1978. + + Postel, Jon, "File Transfer Protocol Specification", RFC 765, ISI, + June 1980. + + Mankins, David, Dan Franklin, and Buzz Owen, "Directory Oriented FTP + Commands", RFC 776, BBN, December 1980. + + Padlipsky, Michael, "FTP Unique-Named Store Command", RFC 949, MITRE, + July 1985. + + + + + + + + + + + + + + +Postel & Reynolds [Page 68] + + + +RFC 959 October 1985 +File Transfer Protocol + + +REFERENCES + + [1] Feinler, Elizabeth, "Internet Protocol Transition Workbook", + Network Information Center, SRI International, March 1982. + + [2] Postel, Jon, "Transmission Control Protocol - DARPA Internet + Program Protocol Specification", RFC 793, DARPA, September 1981. + + [3] Postel, Jon, and Joyce Reynolds, "Telnet Protocol + Specification", RFC 854, ISI, May 1983. + + [4] Reynolds, Joyce, and Jon Postel, "Assigned Numbers", RFC 943, + ISI, April 1985. + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +Postel & Reynolds [Page 69] + diff --git a/doc/rfc1123.txt b/doc/rfc1123.txt new file mode 100644 index 0000000..51cdf83 --- /dev/null +++ b/doc/rfc1123.txt @@ -0,0 +1,5782 @@ + + + + + + +Network Working Group Internet Engineering Task Force +Request for Comments: 1123 R. Braden, Editor + October 1989 + + + Requirements for Internet Hosts -- Application and Support + +Status of This Memo + + This RFC is an official specification for the Internet community. It + incorporates by reference, amends, corrects, and supplements the + primary protocol standards documents relating to hosts. Distribution + of this document is unlimited. + +Summary + + This RFC is one of a pair that defines and discusses the requirements + for Internet host software. This RFC covers the application and + support protocols; its companion RFC-1122 covers the communication + protocol layers: link layer, IP layer, and transport layer. + + + + Table of Contents + + + + + 1. INTRODUCTION ............................................... 5 + 1.1 The Internet Architecture .............................. 6 + 1.2 General Considerations ................................. 6 + 1.2.1 Continuing Internet Evolution ..................... 6 + 1.2.2 Robustness Principle .............................. 7 + 1.2.3 Error Logging ..................................... 8 + 1.2.4 Configuration ..................................... 8 + 1.3 Reading this Document .................................. 10 + 1.3.1 Organization ...................................... 10 + 1.3.2 Requirements ...................................... 10 + 1.3.3 Terminology ....................................... 11 + 1.4 Acknowledgments ........................................ 12 + + 2. GENERAL ISSUES ............................................. 13 + 2.1 Host Names and Numbers ................................. 13 + 2.2 Using Domain Name Service .............................. 13 + 2.3 Applications on Multihomed hosts ....................... 14 + 2.4 Type-of-Service ........................................ 14 + 2.5 GENERAL APPLICATION REQUIREMENTS SUMMARY ............... 15 + + + + +Internet Engineering Task Force [Page 1] + + + + +RFC1123 INTRODUCTION October 1989 + + + 3. REMOTE LOGIN -- TELNET PROTOCOL ............................ 16 + 3.1 INTRODUCTION ........................................... 16 + 3.2 PROTOCOL WALK-THROUGH .................................. 16 + 3.2.1 Option Negotiation ................................ 16 + 3.2.2 Telnet Go-Ahead Function .......................... 16 + 3.2.3 Control Functions ................................. 17 + 3.2.4 Telnet "Synch" Signal ............................. 18 + 3.2.5 NVT Printer and Keyboard .......................... 19 + 3.2.6 Telnet Command Structure .......................... 20 + 3.2.7 Telnet Binary Option .............................. 20 + 3.2.8 Telnet Terminal-Type Option ....................... 20 + 3.3 SPECIFIC ISSUES ........................................ 21 + 3.3.1 Telnet End-of-Line Convention ..................... 21 + 3.3.2 Data Entry Terminals .............................. 23 + 3.3.3 Option Requirements ............................... 24 + 3.3.4 Option Initiation ................................. 24 + 3.3.5 Telnet Linemode Option ............................ 25 + 3.4 TELNET/USER INTERFACE .................................. 25 + 3.4.1 Character Set Transparency ........................ 25 + 3.4.2 Telnet Commands ................................... 26 + 3.4.3 TCP Connection Errors ............................. 26 + 3.4.4 Non-Default Telnet Contact Port ................... 26 + 3.4.5 Flushing Output ................................... 26 + 3.5. TELNET REQUIREMENTS SUMMARY ........................... 27 + + 4. FILE TRANSFER .............................................. 29 + 4.1 FILE TRANSFER PROTOCOL -- FTP .......................... 29 + 4.1.1 INTRODUCTION ...................................... 29 + 4.1.2. PROTOCOL WALK-THROUGH ............................ 29 + 4.1.2.1 LOCAL Type ................................... 29 + 4.1.2.2 Telnet Format Control ........................ 30 + 4.1.2.3 Page Structure ............................... 30 + 4.1.2.4 Data Structure Transformations ............... 30 + 4.1.2.5 Data Connection Management ................... 31 + 4.1.2.6 PASV Command ................................. 31 + 4.1.2.7 LIST and NLST Commands ....................... 31 + 4.1.2.8 SITE Command ................................. 32 + 4.1.2.9 STOU Command ................................. 32 + 4.1.2.10 Telnet End-of-line Code ..................... 32 + 4.1.2.11 FTP Replies ................................. 33 + 4.1.2.12 Connections ................................. 34 + 4.1.2.13 Minimum Implementation; RFC-959 Section ..... 34 + 4.1.3 SPECIFIC ISSUES ................................... 35 + 4.1.3.1 Non-standard Command Verbs ................... 35 + 4.1.3.2 Idle Timeout ................................. 36 + 4.1.3.3 Concurrency of Data and Control .............. 36 + 4.1.3.4 FTP Restart Mechanism ........................ 36 + 4.1.4 FTP/USER INTERFACE ................................ 39 + + + +Internet Engineering Task Force [Page 2] + + + + +RFC1123 INTRODUCTION October 1989 + + + 4.1.4.1 Pathname Specification ....................... 39 + 4.1.4.2 "QUOTE" Command .............................. 40 + 4.1.4.3 Displaying Replies to User ................... 40 + 4.1.4.4 Maintaining Synchronization .................. 40 + 4.1.5 FTP REQUIREMENTS SUMMARY ......................... 41 + 4.2 TRIVIAL FILE TRANSFER PROTOCOL -- TFTP ................. 44 + 4.2.1 INTRODUCTION ...................................... 44 + 4.2.2 PROTOCOL WALK-THROUGH ............................. 44 + 4.2.2.1 Transfer Modes ............................... 44 + 4.2.2.2 UDP Header ................................... 44 + 4.2.3 SPECIFIC ISSUES ................................... 44 + 4.2.3.1 Sorcerer's Apprentice Syndrome ............... 44 + 4.2.3.2 Timeout Algorithms ........................... 46 + 4.2.3.3 Extensions ................................... 46 + 4.2.3.4 Access Control ............................... 46 + 4.2.3.5 Broadcast Request ............................ 46 + 4.2.4 TFTP REQUIREMENTS SUMMARY ......................... 47 + + 5. ELECTRONIC MAIL -- SMTP and RFC-822 ........................ 48 + 5.1 INTRODUCTION ........................................... 48 + 5.2 PROTOCOL WALK-THROUGH .................................. 48 + 5.2.1 The SMTP Model .................................... 48 + 5.2.2 Canonicalization .................................. 49 + 5.2.3 VRFY and EXPN Commands ............................ 50 + 5.2.4 SEND, SOML, and SAML Commands ..................... 50 + 5.2.5 HELO Command ...................................... 50 + 5.2.6 Mail Relay ........................................ 51 + 5.2.7 RCPT Command ...................................... 52 + 5.2.8 DATA Command ...................................... 53 + 5.2.9 Command Syntax .................................... 54 + 5.2.10 SMTP Replies ..................................... 54 + 5.2.11 Transparency ..................................... 55 + 5.2.12 WKS Use in MX Processing ......................... 55 + 5.2.13 RFC-822 Message Specification .................... 55 + 5.2.14 RFC-822 Date and Time Specification .............. 55 + 5.2.15 RFC-822 Syntax Change ............................ 56 + 5.2.16 RFC-822 Local-part .............................. 56 + 5.2.17 Domain Literals .................................. 57 + 5.2.18 Common Address Formatting Errors ................. 58 + 5.2.19 Explicit Source Routes ........................... 58 + 5.3 SPECIFIC ISSUES ........................................ 59 + 5.3.1 SMTP Queueing Strategies .......................... 59 + 5.3.1.1 Sending Strategy .............................. 59 + 5.3.1.2 Receiving strategy ........................... 61 + 5.3.2 Timeouts in SMTP .................................. 61 + 5.3.3 Reliable Mail Receipt ............................. 63 + 5.3.4 Reliable Mail Transmission ........................ 63 + 5.3.5 Domain Name Support ............................... 65 + + + +Internet Engineering Task Force [Page 3] + + + + +RFC1123 INTRODUCTION October 1989 + + + 5.3.6 Mailing Lists and Aliases ......................... 65 + 5.3.7 Mail Gatewaying ................................... 66 + 5.3.8 Maximum Message Size .............................. 68 + 5.4 SMTP REQUIREMENTS SUMMARY .............................. 69 + + 6. SUPPORT SERVICES ............................................ 72 + 6.1 DOMAIN NAME TRANSLATION ................................. 72 + 6.1.1 INTRODUCTION ....................................... 72 + 6.1.2 PROTOCOL WALK-THROUGH ............................. 72 + 6.1.2.1 Resource Records with Zero TTL ............... 73 + 6.1.2.2 QCLASS Values ................................ 73 + 6.1.2.3 Unused Fields ................................ 73 + 6.1.2.4 Compression .................................. 73 + 6.1.2.5 Misusing Configuration Info .................. 73 + 6.1.3 SPECIFIC ISSUES ................................... 74 + 6.1.3.1 Resolver Implementation ...................... 74 + 6.1.3.2 Transport Protocols .......................... 75 + 6.1.3.3 Efficient Resource Usage ..................... 77 + 6.1.3.4 Multihomed Hosts ............................. 78 + 6.1.3.5 Extensibility ................................ 79 + 6.1.3.6 Status of RR Types ........................... 79 + 6.1.3.7 Robustness ................................... 80 + 6.1.3.8 Local Host Table ............................. 80 + 6.1.4 DNS USER INTERFACE ................................ 81 + 6.1.4.1 DNS Administration ........................... 81 + 6.1.4.2 DNS User Interface ........................... 81 + 6.1.4.3 Interface Abbreviation Facilities ............. 82 + 6.1.5 DOMAIN NAME SYSTEM REQUIREMENTS SUMMARY ........... 84 + 6.2 HOST INITIALIZATION .................................... 87 + 6.2.1 INTRODUCTION ...................................... 87 + 6.2.2 REQUIREMENTS ...................................... 87 + 6.2.2.1 Dynamic Configuration ........................ 87 + 6.2.2.2 Loading Phase ................................ 89 + 6.3 REMOTE MANAGEMENT ...................................... 90 + 6.3.1 INTRODUCTION ...................................... 90 + 6.3.2 PROTOCOL WALK-THROUGH ............................. 90 + 6.3.3 MANAGEMENT REQUIREMENTS SUMMARY ................... 92 + + 7. REFERENCES ................................................. 93 + + + + + + + + + + + + +Internet Engineering Task Force [Page 4] + + + + +RFC1123 INTRODUCTION October 1989 + + +1. INTRODUCTION + + This document is one of a pair that defines and discusses the + requirements for host system implementations of the Internet protocol + suite. This RFC covers the applications layer and support protocols. + Its companion RFC, "Requirements for Internet Hosts -- Communications + Layers" [INTRO:1] covers the lower layer protocols: transport layer, + IP layer, and link layer. + + These documents are intended to provide guidance for vendors, + implementors, and users of Internet communication software. They + represent the consensus of a large body of technical experience and + wisdom, contributed by members of the Internet research and vendor + communities. + + This RFC enumerates standard protocols that a host connected to the + Internet must use, and it incorporates by reference the RFCs and + other documents describing the current specifications for these + protocols. It corrects errors in the referenced documents and adds + additional discussion and guidance for an implementor. + + For each protocol, this document also contains an explicit set of + requirements, recommendations, and options. The reader must + understand that the list of requirements in this document is + incomplete by itself; the complete set of requirements for an + Internet host is primarily defined in the standard protocol + specification documents, with the corrections, amendments, and + supplements contained in this RFC. + + A good-faith implementation of the protocols that was produced after + careful reading of the RFC's and with some interaction with the + Internet technical community, and that followed good communications + software engineering practices, should differ from the requirements + of this document in only minor ways. Thus, in many cases, the + "requirements" in this RFC are already stated or implied in the + standard protocol documents, so that their inclusion here is, in a + sense, redundant. However, they were included because some past + implementation has made the wrong choice, causing problems of + interoperability, performance, and/or robustness. + + This document includes discussion and explanation of many of the + requirements and recommendations. A simple list of requirements + would be dangerous, because: + + o Some required features are more important than others, and some + features are optional. + + o There may be valid reasons why particular vendor products that + + + +Internet Engineering Task Force [Page 5] + + + + +RFC1123 INTRODUCTION October 1989 + + + are designed for restricted contexts might choose to use + different specifications. + + However, the specifications of this document must be followed to meet + the general goal of arbitrary host interoperation across the + diversity and complexity of the Internet system. Although most + current implementations fail to meet these requirements in various + ways, some minor and some major, this specification is the ideal + towards which we need to move. + + These requirements are based on the current level of Internet + architecture. This document will be updated as required to provide + additional clarifications or to include additional information in + those areas in which specifications are still evolving. + + This introductory section begins with general advice to host software + vendors, and then gives some guidance on reading the rest of the + document. Section 2 contains general requirements that may be + applicable to all application and support protocols. Sections 3, 4, + and 5 contain the requirements on protocols for the three major + applications: Telnet, file transfer, and electronic mail, + respectively. Section 6 covers the support applications: the domain + name system, system initialization, and management. Finally, all + references will be found in Section 7. + + 1.1 The Internet Architecture + + For a brief introduction to the Internet architecture from a host + viewpoint, see Section 1.1 of [INTRO:1]. That section also + contains recommended references for general background on the + Internet architecture. + + 1.2 General Considerations + + There are two important lessons that vendors of Internet host + software have learned and which a new vendor should consider + seriously. + + 1.2.1 Continuing Internet Evolution + + The enormous growth of the Internet has revealed problems of + management and scaling in a large datagram-based packet + communication system. These problems are being addressed, and + as a result there will be continuing evolution of the + specifications described in this document. These changes will + be carefully planned and controlled, since there is extensive + participation in this planning by the vendors and by the + organizations responsible for operations of the networks. + + + +Internet Engineering Task Force [Page 6] + + + + +RFC1123 INTRODUCTION October 1989 + + + Development, evolution, and revision are characteristic of + computer network protocols today, and this situation will + persist for some years. A vendor who develops computer + communication software for the Internet protocol suite (or any + other protocol suite!) and then fails to maintain and update + that software for changing specifications is going to leave a + trail of unhappy customers. The Internet is a large + communication network, and the users are in constant contact + through it. Experience has shown that knowledge of + deficiencies in vendor software propagates quickly through the + Internet technical community. + + 1.2.2 Robustness Principle + + At every layer of the protocols, there is a general rule whose + application can lead to enormous benefits in robustness and + interoperability: + + "Be liberal in what you accept, and + conservative in what you send" + + Software should be written to deal with every conceivable + error, no matter how unlikely; sooner or later a packet will + come in with that particular combination of errors and + attributes, and unless the software is prepared, chaos can + ensue. In general, it is best to assume that the network is + filled with malevolent entities that will send in packets + designed to have the worst possible effect. This assumption + will lead to suitable protective design, although the most + serious problems in the Internet have been caused by + unenvisaged mechanisms triggered by low-probability events; + mere human malice would never have taken so devious a course! + + Adaptability to change must be designed into all levels of + Internet host software. As a simple example, consider a + protocol specification that contains an enumeration of values + for a particular header field -- e.g., a type field, a port + number, or an error code; this enumeration must be assumed to + be incomplete. Thus, if a protocol specification defines four + possible error codes, the software must not break when a fifth + code shows up. An undefined code might be logged (see below), + but it must not cause a failure. + + The second part of the principle is almost as important: + software on other hosts may contain deficiencies that make it + unwise to exploit legal but obscure protocol features. It is + unwise to stray far from the obvious and simple, lest untoward + effects result elsewhere. A corollary of this is "watch out + + + +Internet Engineering Task Force [Page 7] + + + + +RFC1123 INTRODUCTION October 1989 + + + for misbehaving hosts"; host software should be prepared, not + just to survive other misbehaving hosts, but also to cooperate + to limit the amount of disruption such hosts can cause to the + shared communication facility. + + 1.2.3 Error Logging + + The Internet includes a great variety of host and gateway + systems, each implementing many protocols and protocol layers, + and some of these contain bugs and mis-features in their + Internet protocol software. As a result of complexity, + diversity, and distribution of function, the diagnosis of user + problems is often very difficult. + + Problem diagnosis will be aided if host implementations include + a carefully designed facility for logging erroneous or + "strange" protocol events. It is important to include as much + diagnostic information as possible when an error is logged. In + particular, it is often useful to record the header(s) of a + packet that caused an error. However, care must be taken to + ensure that error logging does not consume prohibitive amounts + of resources or otherwise interfere with the operation of the + host. + + There is a tendency for abnormal but harmless protocol events + to overflow error logging files; this can be avoided by using a + "circular" log, or by enabling logging only while diagnosing a + known failure. It may be useful to filter and count duplicate + successive messages. One strategy that seems to work well is: + (1) always count abnormalities and make such counts accessible + through the management protocol (see Section 6.3); and (2) + allow the logging of a great variety of events to be + selectively enabled. For example, it might useful to be able + to "log everything" or to "log everything for host X". + + Note that different managements may have differing policies + about the amount of error logging that they want normally + enabled in a host. Some will say, "if it doesn't hurt me, I + don't want to know about it", while others will want to take a + more watchful and aggressive attitude about detecting and + removing protocol abnormalities. + + 1.2.4 Configuration + + It would be ideal if a host implementation of the Internet + protocol suite could be entirely self-configuring. This would + allow the whole suite to be implemented in ROM or cast into + silicon, it would simplify diskless workstations, and it would + + + +Internet Engineering Task Force [Page 8] + + + + +RFC1123 INTRODUCTION October 1989 + + + be an immense boon to harried LAN administrators as well as + system vendors. We have not reached this ideal; in fact, we + are not even close. + + At many points in this document, you will find a requirement + that a parameter be a configurable option. There are several + different reasons behind such requirements. In a few cases, + there is current uncertainty or disagreement about the best + value, and it may be necessary to update the recommended value + in the future. In other cases, the value really depends on + external factors -- e.g., the size of the host and the + distribution of its communication load, or the speeds and + topology of nearby networks -- and self-tuning algorithms are + unavailable and may be insufficient. In some cases, + configurability is needed because of administrative + requirements. + + Finally, some configuration options are required to communicate + with obsolete or incorrect implementations of the protocols, + distributed without sources, that unfortunately persist in many + parts of the Internet. To make correct systems coexist with + these faulty systems, administrators often have to "mis- + configure" the correct systems. This problem will correct + itself gradually as the faulty systems are retired, but it + cannot be ignored by vendors. + + When we say that a parameter must be configurable, we do not + intend to require that its value be explicitly read from a + configuration file at every boot time. We recommend that + implementors set up a default for each parameter, so a + configuration file is only necessary to override those defaults + that are inappropriate in a particular installation. Thus, the + configurability requirement is an assurance that it will be + POSSIBLE to override the default when necessary, even in a + binary-only or ROM-based product. + + This document requires a particular value for such defaults in + some cases. The choice of default is a sensitive issue when + the configuration item controls the accommodation to existing + faulty systems. If the Internet is to converge successfully to + complete interoperability, the default values built into + implementations must implement the official protocol, not + "mis-configurations" to accommodate faulty implementations. + Although marketing considerations have led some vendors to + choose mis-configuration defaults, we urge vendors to choose + defaults that will conform to the standard. + + Finally, we note that a vendor needs to provide adequate + + + +Internet Engineering Task Force [Page 9] + + + + +RFC1123 INTRODUCTION October 1989 + + + documentation on all configuration parameters, their limits and + effects. + + + 1.3 Reading this Document + + 1.3.1 Organization + + In general, each major section is organized into the following + subsections: + + (1) Introduction + + (2) Protocol Walk-Through -- considers the protocol + specification documents section-by-section, correcting + errors, stating requirements that may be ambiguous or + ill-defined, and providing further clarification or + explanation. + + (3) Specific Issues -- discusses protocol design and + implementation issues that were not included in the walk- + through. + + (4) Interfaces -- discusses the service interface to the next + higher layer. + + (5) Summary -- contains a summary of the requirements of the + section. + + Under many of the individual topics in this document, there is + parenthetical material labeled "DISCUSSION" or + "IMPLEMENTATION". This material is intended to give + clarification and explanation of the preceding requirements + text. It also includes some suggestions on possible future + directions or developments. The implementation material + contains suggested approaches that an implementor may want to + consider. + + The summary sections are intended to be guides and indexes to + the text, but are necessarily cryptic and incomplete. The + summaries should never be used or referenced separately from + the complete RFC. + + 1.3.2 Requirements + + In this document, the words that are used to define the + significance of each particular requirement are capitalized. + These words are: + + + +Internet Engineering Task Force [Page 10] + + + + +RFC1123 INTRODUCTION October 1989 + + + * "MUST" + + This word or the adjective "REQUIRED" means that the item + is an absolute requirement of the specification. + + * "SHOULD" + + This word or the adjective "RECOMMENDED" means that there + may exist valid reasons in particular circumstances to + ignore this item, but the full implications should be + understood and the case carefully weighed before choosing + a different course. + + * "MAY" + + This word or the adjective "OPTIONAL" means that this item + is truly optional. One vendor may choose to include the + item because a particular marketplace requires it or + because it enhances the product, for example; another + vendor may omit the same item. + + + An implementation is not compliant if it fails to satisfy one + or more of the MUST requirements for the protocols it + implements. An implementation that satisfies all the MUST and + all the SHOULD requirements for its protocols is said to be + "unconditionally compliant"; one that satisfies all the MUST + requirements but not all the SHOULD requirements for its + protocols is said to be "conditionally compliant". + + 1.3.3 Terminology + + This document uses the following technical terms: + + Segment + A segment is the unit of end-to-end transmission in the + TCP protocol. A segment consists of a TCP header followed + by application data. A segment is transmitted by + encapsulation in an IP datagram. + + Message + This term is used by some application layer protocols + (particularly SMTP) for an application data unit. + + Datagram + A [UDP] datagram is the unit of end-to-end transmission in + the UDP protocol. + + + + +Internet Engineering Task Force [Page 11] + + + + +RFC1123 INTRODUCTION October 1989 + + + Multihomed + A host is said to be multihomed if it has multiple IP + addresses to connected networks. + + + + 1.4 Acknowledgments + + This document incorporates contributions and comments from a large + group of Internet protocol experts, including representatives of + university and research labs, vendors, and government agencies. + It was assembled primarily by the Host Requirements Working Group + of the Internet Engineering Task Force (IETF). + + The Editor would especially like to acknowledge the tireless + dedication of the following people, who attended many long + meetings and generated 3 million bytes of electronic mail over the + past 18 months in pursuit of this document: Philip Almquist, Dave + Borman (Cray Research), Noel Chiappa, Dave Crocker (DEC), Steve + Deering (Stanford), Mike Karels (Berkeley), Phil Karn (Bellcore), + John Lekashman (NASA), Charles Lynn (BBN), Keith McCloghrie (TWG), + Paul Mockapetris (ISI), Thomas Narten (Purdue), Craig Partridge + (BBN), Drew Perkins (CMU), and James Van Bokkelen (FTP Software). + + In addition, the following people made major contributions to the + effort: Bill Barns (Mitre), Steve Bellovin (AT&T), Mike Brescia + (BBN), Ed Cain (DCA), Annette DeSchon (ISI), Martin Gross (DCA), + Phill Gross (NRI), Charles Hedrick (Rutgers), Van Jacobson (LBL), + John Klensin (MIT), Mark Lottor (SRI), Milo Medin (NASA), Bill + Melohn (Sun Microsystems), Greg Minshall (Kinetics), Jeff Mogul + (DEC), John Mullen (CMC), Jon Postel (ISI), John Romkey (Epilogue + Technology), and Mike StJohns (DCA). The following also made + significant contributions to particular areas: Eric Allman + (Berkeley), Rob Austein (MIT), Art Berggreen (ACC), Keith Bostic + (Berkeley), Vint Cerf (NRI), Wayne Hathaway (NASA), Matt Korn + (IBM), Erik Naggum (Naggum Software, Norway), Robert Ullmann + (Prime Computer), David Waitzman (BBN), Frank Wancho (USA), Arun + Welch (Ohio State), Bill Westfield (Cisco), and Rayan Zachariassen + (Toronto). + + We are grateful to all, including any contributors who may have + been inadvertently omitted from this list. + + + + + + + + + +Internet Engineering Task Force [Page 12] + + + + +RFC1123 APPLICATIONS LAYER -- GENERAL October 1989 + + +2. GENERAL ISSUES + + This section contains general requirements that may be applicable to + all application-layer protocols. + + 2.1 Host Names and Numbers + + The syntax of a legal Internet host name was specified in RFC-952 + [DNS:4]. One aspect of host name syntax is hereby changed: the + restriction on the first character is relaxed to allow either a + letter or a digit. Host software MUST support this more liberal + syntax. + + Host software MUST handle host names of up to 63 characters and + SHOULD handle host names of up to 255 characters. + + Whenever a user inputs the identity of an Internet host, it SHOULD + be possible to enter either (1) a host domain name or (2) an IP + address in dotted-decimal ("#.#.#.#") form. The host SHOULD check + the string syntactically for a dotted-decimal number before + looking it up in the Domain Name System. + + DISCUSSION: + This last requirement is not intended to specify the complete + syntactic form for entering a dotted-decimal host number; + that is considered to be a user-interface issue. For + example, a dotted-decimal number must be enclosed within + "[ ]" brackets for SMTP mail (see Section 5.2.17). This + notation could be made universal within a host system, + simplifying the syntactic checking for a dotted-decimal + number. + + If a dotted-decimal number can be entered without such + identifying delimiters, then a full syntactic check must be + made, because a segment of a host domain name is now allowed + to begin with a digit and could legally be entirely numeric + (see Section 6.1.2.4). However, a valid host name can never + have the dotted-decimal form #.#.#.#, since at least the + highest-level component label will be alphabetic. + + 2.2 Using Domain Name Service + + Host domain names MUST be translated to IP addresses as described + in Section 6.1. + + Applications using domain name services MUST be able to cope with + soft error conditions. Applications MUST wait a reasonable + interval between successive retries due to a soft error, and MUST + + + +Internet Engineering Task Force [Page 13] + + + + +RFC1123 APPLICATIONS LAYER -- GENERAL October 1989 + + + allow for the possibility that network problems may deny service + for hours or even days. + + An application SHOULD NOT rely on the ability to locate a WKS + record containing an accurate listing of all services at a + particular host address, since the WKS RR type is not often used + by Internet sites. To confirm that a service is present, simply + attempt to use it. + + 2.3 Applications on Multihomed hosts + + When the remote host is multihomed, the name-to-address + translation will return a list of alternative IP addresses. As + specified in Section 6.1.3.4, this list should be in order of + decreasing preference. Application protocol implementations + SHOULD be prepared to try multiple addresses from the list until + success is obtained. More specific requirements for SMTP are + given in Section 5.3.4. + + When the local host is multihomed, a UDP-based request/response + application SHOULD send the response with an IP source address + that is the same as the specific destination address of the UDP + request datagram. The "specific destination address" is defined + in the "IP Addressing" section of the companion RFC [INTRO:1]. + + Similarly, a server application that opens multiple TCP + connections to the same client SHOULD use the same local IP + address for all. + + 2.4 Type-of-Service + + Applications MUST select appropriate TOS values when they invoke + transport layer services, and these values MUST be configurable. + Note that a TOS value contains 5 bits, of which only the most- + significant 3 bits are currently defined; the other two bits MUST + be zero. + + DISCUSSION: + As gateway algorithms are developed to implement Type-of- + Service, the recommended values for various application + protocols may change. In addition, it is likely that + particular combinations of users and Internet paths will want + non-standard TOS values. For these reasons, the TOS values + must be configurable. + + See the latest version of the "Assigned Numbers" RFC + [INTRO:5] for the recommended TOS values for the major + application protocols. + + + +Internet Engineering Task Force [Page 14] + + + + +RFC1123 APPLICATIONS LAYER -- GENERAL October 1989 + + + 2.5 GENERAL APPLICATION REQUIREMENTS SUMMARY + + | | | | |S| | + | | | | |H| |F + | | | | |O|M|o + | | |S| |U|U|o + | | |H| |L|S|t + | |M|O| |D|T|n + | |U|U|M| | |o + | |S|L|A|N|N|t + | |T|D|Y|O|O|t +FEATURE |SECTION | | | |T|T|e +-----------------------------------------------|----------|-|-|-|-|-|-- + | | | | | | | +User interfaces: | | | | | | | + Allow host name to begin with digit |2.1 |x| | | | | + Host names of up to 635 characters |2.1 |x| | | | | + Host names of up to 255 characters |2.1 | |x| | | | + Support dotted-decimal host numbers |2.1 | |x| | | | + Check syntactically for dotted-dec first |2.1 | |x| | | | + | | | | | | | +Map domain names per Section 6.1 |2.2 |x| | | | | +Cope with soft DNS errors |2.2 |x| | | | | + Reasonable interval between retries |2.2 |x| | | | | + Allow for long outages |2.2 |x| | | | | +Expect WKS records to be available |2.2 | | | |x| | + | | | | | | | +Try multiple addr's for remote multihomed host |2.3 | |x| | | | +UDP reply src addr is specific dest of request |2.3 | |x| | | | +Use same IP addr for related TCP connections |2.3 | |x| | | | +Specify appropriate TOS values |2.4 |x| | | | | + TOS values configurable |2.4 |x| | | | | + Unused TOS bits zero |2.4 |x| | | | | + | | | | | | | + | | | | | | | + + + + + + + + + + + + + + + + +Internet Engineering Task Force [Page 15] + + + + +RFC1123 REMOTE LOGIN -- TELNET October 1989 + + +3. REMOTE LOGIN -- TELNET PROTOCOL + + 3.1 INTRODUCTION + + Telnet is the standard Internet application protocol for remote + login. It provides the encoding rules to link a user's + keyboard/display on a client ("user") system with a command + interpreter on a remote server system. A subset of the Telnet + protocol is also incorporated within other application protocols, + e.g., FTP and SMTP. + + Telnet uses a single TCP connection, and its normal data stream + ("Network Virtual Terminal" or "NVT" mode) is 7-bit ASCII with + escape sequences to embed control functions. Telnet also allows + the negotiation of many optional modes and functions. + + The primary Telnet specification is to be found in RFC-854 + [TELNET:1], while the options are defined in many other RFCs; see + Section 7 for references. + + 3.2 PROTOCOL WALK-THROUGH + + 3.2.1 Option Negotiation: RFC-854, pp. 2-3 + + Every Telnet implementation MUST include option negotiation and + subnegotiation machinery [TELNET:2]. + + A host MUST carefully follow the rules of RFC-854 to avoid + option-negotiation loops. A host MUST refuse (i.e, reply + WONT/DONT to a DO/WILL) an unsupported option. Option + negotiation SHOULD continue to function (even if all requests + are refused) throughout the lifetime of a Telnet connection. + + If all option negotiations fail, a Telnet implementation MUST + default to, and support, an NVT. + + DISCUSSION: + Even though more sophisticated "terminals" and supporting + option negotiations are becoming the norm, all + implementations must be prepared to support an NVT for any + user-server communication. + + 3.2.2 Telnet Go-Ahead Function: RFC-854, p. 5, and RFC-858 + + On a host that never sends the Telnet command Go Ahead (GA), + the Telnet Server MUST attempt to negotiate the Suppress Go + Ahead option (i.e., send "WILL Suppress Go Ahead"). A User or + Server Telnet MUST always accept negotiation of the Suppress Go + + + +Internet Engineering Task Force [Page 16] + + + + +RFC1123 REMOTE LOGIN -- TELNET October 1989 + + + Ahead option. + + When it is driving a full-duplex terminal for which GA has no + meaning, a User Telnet implementation MAY ignore GA commands. + + DISCUSSION: + Half-duplex ("locked-keyboard") line-at-a-time terminals + for which the Go-Ahead mechanism was designed have largely + disappeared from the scene. It turned out to be difficult + to implement sending the Go-Ahead signal in many operating + systems, even some systems that support native half-duplex + terminals. The difficulty is typically that the Telnet + server code does not have access to information about + whether the user process is blocked awaiting input from + the Telnet connection, i.e., it cannot reliably determine + when to send a GA command. Therefore, most Telnet Server + hosts do not send GA commands. + + The effect of the rules in this section is to allow either + end of a Telnet connection to veto the use of GA commands. + + There is a class of half-duplex terminals that is still + commercially important: "data entry terminals," which + interact in a full-screen manner. However, supporting + data entry terminals using the Telnet protocol does not + require the Go Ahead signal; see Section 3.3.2. + + 3.2.3 Control Functions: RFC-854, pp. 7-8 + + The list of Telnet commands has been extended to include EOR + (End-of-Record), with code 239 [TELNET:9]. + + Both User and Server Telnets MAY support the control functions + EOR, EC, EL, and Break, and MUST support AO, AYT, DM, IP, NOP, + SB, and SE. + + A host MUST be able to receive and ignore any Telnet control + functions that it does not support. + + DISCUSSION: + Note that a Server Telnet is required to support the + Telnet IP (Interrupt Process) function, even if the server + host has an equivalent in-stream function (e.g., Control-C + in many systems). The Telnet IP function may be stronger + than an in-stream interrupt command, because of the out- + of-band effect of TCP urgent data. + + The EOR control function may be used to delimit the + + + +Internet Engineering Task Force [Page 17] + + + + +RFC1123 REMOTE LOGIN -- TELNET October 1989 + + + stream. An important application is data entry terminal + support (see Section 3.3.2). There was concern that since + EOR had not been defined in RFC-854, a host that was not + prepared to correctly ignore unknown Telnet commands might + crash if it received an EOR. To protect such hosts, the + End-of-Record option [TELNET:9] was introduced; however, a + properly implemented Telnet program will not require this + protection. + + 3.2.4 Telnet "Synch" Signal: RFC-854, pp. 8-10 + + When it receives "urgent" TCP data, a User or Server Telnet + MUST discard all data except Telnet commands until the DM (and + end of urgent) is reached. + + When it sends Telnet IP (Interrupt Process), a User Telnet + SHOULD follow it by the Telnet "Synch" sequence, i.e., send as + TCP urgent data the sequence "IAC IP IAC DM". The TCP urgent + pointer points to the DM octet. + + When it receives a Telnet IP command, a Server Telnet MAY send + a Telnet "Synch" sequence back to the user, to flush the output + stream. The choice ought to be consistent with the way the + server operating system behaves when a local user interrupts a + process. + + When it receives a Telnet AO command, a Server Telnet MUST send + a Telnet "Synch" sequence back to the user, to flush the output + stream. + + A User Telnet SHOULD have the capability of flushing output + when it sends a Telnet IP; see also Section 3.4.5. + + DISCUSSION: + There are three possible ways for a User Telnet to flush + the stream of server output data: + + (1) Send AO after IP. + + This will cause the server host to send a "flush- + buffered-output" signal to its operating system. + However, the AO may not take effect locally, i.e., + stop terminal output at the User Telnet end, until + the Server Telnet has received and processed the AO + and has sent back a "Synch". + + (2) Send DO TIMING-MARK [TELNET:7] after IP, and discard + all output locally until a WILL/WONT TIMING-MARK is + + + +Internet Engineering Task Force [Page 18] + + + + +RFC1123 REMOTE LOGIN -- TELNET October 1989 + + + received from the Server Telnet. + + Since the DO TIMING-MARK will be processed after the + IP at the server, the reply to it should be in the + right place in the output data stream. However, the + TIMING-MARK will not send a "flush buffered output" + signal to the server operating system. Whether or + not this is needed is dependent upon the server + system. + + (3) Do both. + + The best method is not entirely clear, since it must + accommodate a number of existing server hosts that do not + follow the Telnet standards in various ways. The safest + approach is probably to provide a user-controllable option + to select (1), (2), or (3). + + 3.2.5 NVT Printer and Keyboard: RFC-854, p. 11 + + In NVT mode, a Telnet SHOULD NOT send characters with the + high-order bit 1, and MUST NOT send it as a parity bit. + Implementations that pass the high-order bit to applications + SHOULD negotiate binary mode (see Section 3.2.6). + + + DISCUSSION: + Implementors should be aware that a strict reading of + RFC-854 allows a client or server expecting NVT ASCII to + ignore characters with the high-order bit set. In + general, binary mode is expected to be used for + transmission of an extended (beyond 7-bit) character set + with Telnet. + + However, there exist applications that really need an 8- + bit NVT mode, which is currently not defined, and these + existing applications do set the high-order bit during + part or all of the life of a Telnet connection. Note that + binary mode is not the same as 8-bit NVT mode, since + binary mode turns off end-of-line processing. For this + reason, the requirements on the high-order bit are stated + as SHOULD, not MUST. + + RFC-854 defines a minimal set of properties of a "network + virtual terminal" or NVT; this is not meant to preclude + additional features in a real terminal. A Telnet + connection is fully transparent to all 7-bit ASCII + characters, including arbitrary ASCII control characters. + + + +Internet Engineering Task Force [Page 19] + + + + +RFC1123 REMOTE LOGIN -- TELNET October 1989 + + + For example, a terminal might support full-screen commands + coded as ASCII escape sequences; a Telnet implementation + would pass these sequences as uninterpreted data. Thus, + an NVT should not be conceived as a terminal type of a + highly-restricted device. + + 3.2.6 Telnet Command Structure: RFC-854, p. 13 + + Since options may appear at any point in the data stream, a + Telnet escape character (known as IAC, with the value 255) to + be sent as data MUST be doubled. + + 3.2.7 Telnet Binary Option: RFC-856 + + When the Binary option has been successfully negotiated, + arbitrary 8-bit characters are allowed. However, the data + stream MUST still be scanned for IAC characters, any embedded + Telnet commands MUST be obeyed, and data bytes equal to IAC + MUST be doubled. Other character processing (e.g., replacing + CR by CR NUL or by CR LF) MUST NOT be done. In particular, + there is no end-of-line convention (see Section 3.3.1) in + binary mode. + + DISCUSSION: + The Binary option is normally negotiated in both + directions, to change the Telnet connection from NVT mode + to "binary mode". + + The sequence IAC EOR can be used to delimit blocks of data + within a binary-mode Telnet stream. + + 3.2.8 Telnet Terminal-Type Option: RFC-1091 + + The Terminal-Type option MUST use the terminal type names + officially defined in the Assigned Numbers RFC [INTRO:5], when + they are available for the particular terminal. However, the + receiver of a Terminal-Type option MUST accept any name. + + DISCUSSION: + RFC-1091 [TELNET:10] updates an earlier version of the + Terminal-Type option defined in RFC-930. The earlier + version allowed a server host capable of supporting + multiple terminal types to learn the type of a particular + client's terminal, assuming that each physical terminal + had an intrinsic type. However, today a "terminal" is + often really a terminal emulator program running in a PC, + perhaps capable of emulating a range of terminal types. + Therefore, RFC-1091 extends the specification to allow a + + + +Internet Engineering Task Force [Page 20] + + + + +RFC1123 REMOTE LOGIN -- TELNET October 1989 + + + more general terminal-type negotiation between User and + Server Telnets. + + 3.3 SPECIFIC ISSUES + + 3.3.1 Telnet End-of-Line Convention + + The Telnet protocol defines the sequence CR LF to mean "end- + of-line". For terminal input, this corresponds to a command- + completion or "end-of-line" key being pressed on a user + terminal; on an ASCII terminal, this is the CR key, but it may + also be labelled "Return" or "Enter". + + When a Server Telnet receives the Telnet end-of-line sequence + CR LF as input from a remote terminal, the effect MUST be the + same as if the user had pressed the "end-of-line" key on a + local terminal. On server hosts that use ASCII, in particular, + receipt of the Telnet sequence CR LF must cause the same effect + as a local user pressing the CR key on a local terminal. Thus, + CR LF and CR NUL MUST have the same effect on an ASCII server + host when received as input over a Telnet connection. + + A User Telnet MUST be able to send any of the forms: CR LF, CR + NUL, and LF. A User Telnet on an ASCII host SHOULD have a + user-controllable mode to send either CR LF or CR NUL when the + user presses the "end-of-line" key, and CR LF SHOULD be the + default. + + The Telnet end-of-line sequence CR LF MUST be used to send + Telnet data that is not terminal-to-computer (e.g., for Server + Telnet sending output, or the Telnet protocol incorporated + another application protocol). + + DISCUSSION: + To allow interoperability between arbitrary Telnet clients + and servers, the Telnet protocol defined a standard + representation for a line terminator. Since the ASCII + character set includes no explicit end-of-line character, + systems have chosen various representations, e.g., CR, LF, + and the sequence CR LF. The Telnet protocol chose the CR + LF sequence as the standard for network transmission. + + Unfortunately, the Telnet protocol specification in RFC- + 854 [TELNET:1] has turned out to be somewhat ambiguous on + what character(s) should be sent from client to server for + the "end-of-line" key. The result has been a massive and + continuing interoperability headache, made worse by + various faulty implementations of both User and Server + + + +Internet Engineering Task Force [Page 21] + + + + +RFC1123 REMOTE LOGIN -- TELNET October 1989 + + + Telnets. + + Although the Telnet protocol is based on a perfectly + symmetric model, in a remote login session the role of the + user at a terminal differs from the role of the server + host. For example, RFC-854 defines the meaning of CR, LF, + and CR LF as output from the server, but does not specify + what the User Telnet should send when the user presses the + "end-of-line" key on the terminal; this turns out to be + the point at issue. + + When a user presses the "end-of-line" key, some User + Telnet implementations send CR LF, while others send CR + NUL (based on a different interpretation of the same + sentence in RFC-854). These will be equivalent for a + correctly-implemented ASCII server host, as discussed + above. For other servers, a mode in the User Telnet is + needed. + + The existence of User Telnets that send only CR NUL when + CR is pressed creates a dilemma for non-ASCII hosts: they + can either treat CR NUL as equivalent to CR LF in input, + thus precluding the possibility of entering a "bare" CR, + or else lose complete interworking. + + Suppose a user on host A uses Telnet to log into a server + host B, and then execute B's User Telnet program to log + into server host C. It is desirable for the Server/User + Telnet combination on B to be as transparent as possible, + i.e., to appear as if A were connected directly to C. In + particular, correct implementation will make B transparent + to Telnet end-of-line sequences, except that CR LF may be + translated to CR NUL or vice versa. + + IMPLEMENTATION: + To understand Telnet end-of-line issues, one must have at + least a general model of the relationship of Telnet to the + local operating system. The Server Telnet process is + typically coupled into the terminal driver software of the + operating system as a pseudo-terminal. A Telnet end-of- + line sequence received by the Server Telnet must have the + same effect as pressing the end-of-line key on a real + locally-connected terminal. + + Operating systems that support interactive character-at- + a-time applications (e.g., editors) typically have two + internal modes for their terminal I/O: a formatted mode, + in which local conventions for end-of-line and other + + + +Internet Engineering Task Force [Page 22] + + + + +RFC1123 REMOTE LOGIN -- TELNET October 1989 + + + formatting rules have been applied to the data stream, and + a "raw" mode, in which the application has direct access + to every character as it was entered. A Server Telnet + must be implemented in such a way that these modes have + the same effect for remote as for local terminals. For + example, suppose a CR LF or CR NUL is received by the + Server Telnet on an ASCII host. In raw mode, a CR + character is passed to the application; in formatted mode, + the local system's end-of-line convention is used. + + 3.3.2 Data Entry Terminals + + DISCUSSION: + In addition to the line-oriented and character-oriented + ASCII terminals for which Telnet was designed, there are + several families of video display terminals that are + sometimes known as "data entry terminals" or DETs. The + IBM 3270 family is a well-known example. + + Two Internet protocols have been designed to support + generic DETs: SUPDUP [TELNET:16, TELNET:17], and the DET + option [TELNET:18, TELNET:19]. The DET option drives a + data entry terminal over a Telnet connection using (sub-) + negotiation. SUPDUP is a completely separate terminal + protocol, which can be entered from Telnet by negotiation. + Although both SUPDUP and the DET option have been used + successfully in particular environments, neither has + gained general acceptance or wide implementation. + + A different approach to DET interaction has been developed + for supporting the IBM 3270 family through Telnet, + although the same approach would be applicable to any DET. + The idea is to enter a "native DET" mode, in which the + native DET input/output stream is sent as binary data. + The Telnet EOR command is used to delimit logical records + (e.g., "screens") within this binary stream. + + IMPLEMENTATION: + The rules for entering and leaving native DET mode are as + follows: + + o The Server uses the Terminal-Type option [TELNET:10] + to learn that the client is a DET. + + o It is conventional, but not required, that both ends + negotiate the EOR option [TELNET:9]. + + o Both ends negotiate the Binary option [TELNET:3] to + + + +Internet Engineering Task Force [Page 23] + + + + +RFC1123 REMOTE LOGIN -- TELNET October 1989 + + + enter native DET mode. + + o When either end negotiates out of binary mode, the + other end does too, and the mode then reverts to + normal NVT. + + + 3.3.3 Option Requirements + + Every Telnet implementation MUST support the Binary option + [TELNET:3] and the Suppress Go Ahead option [TELNET:5], and + SHOULD support the Echo [TELNET:4], Status [TELNET:6], End-of- + Record [TELNET:9], and Extended Options List [TELNET:8] + options. + + A User or Server Telnet SHOULD support the Window Size Option + [TELNET:12] if the local operating system provides the + corresponding capability. + + DISCUSSION: + Note that the End-of-Record option only signifies that a + Telnet can receive a Telnet EOR without crashing; + therefore, every Telnet ought to be willing to accept + negotiation of the End-of-Record option. See also the + discussion in Section 3.2.3. + + 3.3.4 Option Initiation + + When the Telnet protocol is used in a client/server situation, + the server SHOULD initiate negotiation of the terminal + interaction mode it expects. + + DISCUSSION: + The Telnet protocol was defined to be perfectly + symmetrical, but its application is generally asymmetric. + Remote login has been known to fail because NEITHER side + initiated negotiation of the required non-default terminal + modes. It is generally the server that determines the + preferred mode, so the server needs to initiate the + negotiation; since the negotiation is symmetric, the user + can also initiate it. + + A client (User Telnet) SHOULD provide a means for users to + enable and disable the initiation of option negotiation. + + DISCUSSION: + A user sometimes needs to connect to an application + service (e.g., FTP or SMTP) that uses Telnet for its + + + +Internet Engineering Task Force [Page 24] + + + + +RFC1123 REMOTE LOGIN -- TELNET October 1989 + + + control stream but does not support Telnet options. User + Telnet may be used for this purpose if initiation of + option negotiation is disabled. + + 3.3.5 Telnet Linemode Option + + DISCUSSION: + An important new Telnet option, LINEMODE [TELNET:12], has + been proposed. The LINEMODE option provides a standard + way for a User Telnet and a Server Telnet to agree that + the client rather than the server will perform terminal + character processing. When the client has prepared a + complete line of text, it will send it to the server in + (usually) one TCP packet. This option will greatly + decrease the packet cost of Telnet sessions and will also + give much better user response over congested or long- + delay networks. + + The LINEMODE option allows dynamic switching between local + and remote character processing. For example, the Telnet + connection will automatically negotiate into single- + character mode while a full screen editor is running, and + then return to linemode when the editor is finished. + + We expect that when this RFC is released, hosts should + implement the client side of this option, and may + implement the server side of this option. To properly + implement the server side, the server needs to be able to + tell the local system not to do any input character + processing, but to remember its current terminal state and + notify the Server Telnet process whenever the state + changes. This will allow password echoing and full screen + editors to be handled properly, for example. + + 3.4 TELNET/USER INTERFACE + + 3.4.1 Character Set Transparency + + User Telnet implementations SHOULD be able to send or receive + any 7-bit ASCII character. Where possible, any special + character interpretations by the user host's operating system + SHOULD be bypassed so that these characters can conveniently be + sent and received on the connection. + + Some character value MUST be reserved as "escape to command + mode"; conventionally, doubling this character allows it to be + entered as data. The specific character used SHOULD be user + selectable. + + + +Internet Engineering Task Force [Page 25] + + + + +RFC1123 REMOTE LOGIN -- TELNET October 1989 + + + On binary-mode connections, a User Telnet program MAY provide + an escape mechanism for entering arbitrary 8-bit values, if the + host operating system doesn't allow them to be entered directly + from the keyboard. + + IMPLEMENTATION: + The transparency issues are less pressing on servers, but + implementors should take care in dealing with issues like: + masking off parity bits (sent by an older, non-conforming + client) before they reach programs that expect only NVT + ASCII, and properly handling programs that request 8-bit + data streams. + + 3.4.2 Telnet Commands + + A User Telnet program MUST provide a user the capability of + entering any of the Telnet control functions IP, AO, or AYT, + and SHOULD provide the capability of entering EC, EL, and + Break. + + 3.4.3 TCP Connection Errors + + A User Telnet program SHOULD report to the user any TCP errors + that are reported by the transport layer (see "TCP/Application + Layer Interface" section in [INTRO:1]). + + 3.4.4 Non-Default Telnet Contact Port + + A User Telnet program SHOULD allow the user to optionally + specify a non-standard contact port number at the Server Telnet + host. + + 3.4.5 Flushing Output + + A User Telnet program SHOULD provide the user the ability to + specify whether or not output should be flushed when an IP is + sent; see Section 3.2.4. + + For any output flushing scheme that causes the User Telnet to + flush output locally until a Telnet signal is received from the + Server, there SHOULD be a way for the user to manually restore + normal output, in case the Server fails to send the expected + signal. + + + + + + + + +Internet Engineering Task Force [Page 26] + + + + +RFC1123 REMOTE LOGIN -- TELNET October 1989 + + + 3.5. TELNET REQUIREMENTS SUMMARY + + + | | | | |S| | + | | | | |H| |F + | | | | |O|M|o + | | |S| |U|U|o + | | |H| |L|S|t + | |M|O| |D|T|n + | |U|U|M| | |o + | |S|L|A|N|N|t + | |T|D|Y|O|O|t +FEATURE |SECTION | | | |T|T|e +-------------------------------------------------|--------|-|-|-|-|-|-- + | | | | | | | +Option Negotiation |3.2.1 |x| | | | | + Avoid negotiation loops |3.2.1 |x| | | | | + Refuse unsupported options |3.2.1 |x| | | | | + Negotiation OK anytime on connection |3.2.1 | |x| | | | + Default to NVT |3.2.1 |x| | | | | + Send official name in Term-Type option |3.2.8 |x| | | | | + Accept any name in Term-Type option |3.2.8 |x| | | | | + Implement Binary, Suppress-GA options |3.3.3 |x| | | | | + Echo, Status, EOL, Ext-Opt-List options |3.3.3 | |x| | | | + Implement Window-Size option if appropriate |3.3.3 | |x| | | | + Server initiate mode negotiations |3.3.4 | |x| | | | + User can enable/disable init negotiations |3.3.4 | |x| | | | + | | | | | | | +Go-Aheads | | | | | | | + Non-GA server negotiate SUPPRESS-GA option |3.2.2 |x| | | | | + User or Server accept SUPPRESS-GA option |3.2.2 |x| | | | | + User Telnet ignore GA's |3.2.2 | | |x| | | + | | | | | | | +Control Functions | | | | | | | + Support SE NOP DM IP AO AYT SB |3.2.3 |x| | | | | + Support EOR EC EL Break |3.2.3 | | |x| | | + Ignore unsupported control functions |3.2.3 |x| | | | | + User, Server discard urgent data up to DM |3.2.4 |x| | | | | + User Telnet send "Synch" after IP, AO, AYT |3.2.4 | |x| | | | + Server Telnet reply Synch to IP |3.2.4 | | |x| | | + Server Telnet reply Synch to AO |3.2.4 |x| | | | | + User Telnet can flush output when send IP |3.2.4 | |x| | | | + | | | | | | | +Encoding | | | | | | | + Send high-order bit in NVT mode |3.2.5 | | | |x| | + Send high-order bit as parity bit |3.2.5 | | | | |x| + Negot. BINARY if pass high-ord. bit to applic |3.2.5 | |x| | | | + Always double IAC data byte |3.2.6 |x| | | | | + + + +Internet Engineering Task Force [Page 27] + + + + +RFC1123 REMOTE LOGIN -- TELNET October 1989 + + + Double IAC data byte in binary mode |3.2.7 |x| | | | | + Obey Telnet cmds in binary mode |3.2.7 |x| | | | | + End-of-line, CR NUL in binary mode |3.2.7 | | | | |x| + | | | | | | | +End-of-Line | | | | | | | + EOL at Server same as local end-of-line |3.3.1 |x| | | | | + ASCII Server accept CR LF or CR NUL for EOL |3.3.1 |x| | | | | + User Telnet able to send CR LF, CR NUL, or LF |3.3.1 |x| | | | | + ASCII user able to select CR LF/CR NUL |3.3.1 | |x| | | | + User Telnet default mode is CR LF |3.3.1 | |x| | | | + Non-interactive uses CR LF for EOL |3.3.1 |x| | | | | + | | | | | | | +User Telnet interface | | | | | | | + Input & output all 7-bit characters |3.4.1 | |x| | | | + Bypass local op sys interpretation |3.4.1 | |x| | | | + Escape character |3.4.1 |x| | | | | + User-settable escape character |3.4.1 | |x| | | | + Escape to enter 8-bit values |3.4.1 | | |x| | | + Can input IP, AO, AYT |3.4.2 |x| | | | | + Can input EC, EL, Break |3.4.2 | |x| | | | + Report TCP connection errors to user |3.4.3 | |x| | | | + Optional non-default contact port |3.4.4 | |x| | | | + Can spec: output flushed when IP sent |3.4.5 | |x| | | | + Can manually restore output mode |3.4.5 | |x| | | | + | | | | | | | + + + + + + + + + + + + + + + + + + + + + + + + + + +Internet Engineering Task Force [Page 28] + + + + +RFC1123 FILE TRANSFER -- FTP October 1989 + + +4. FILE TRANSFER + + 4.1 FILE TRANSFER PROTOCOL -- FTP + + 4.1.1 INTRODUCTION + + The File Transfer Protocol FTP is the primary Internet standard + for file transfer. The current specification is contained in + RFC-959 [FTP:1]. + + FTP uses separate simultaneous TCP connections for control and + for data transfer. The FTP protocol includes many features, + some of which are not commonly implemented. However, for every + feature in FTP, there exists at least one implementation. The + minimum implementation defined in RFC-959 was too small, so a + somewhat larger minimum implementation is defined here. + + Internet users have been unnecessarily burdened for years by + deficient FTP implementations. Protocol implementors have + suffered from the erroneous opinion that implementing FTP ought + to be a small and trivial task. This is wrong, because FTP has + a user interface, because it has to deal (correctly) with the + whole variety of communication and operating system errors that + may occur, and because it has to handle the great diversity of + real file systems in the world. + + 4.1.2. PROTOCOL WALK-THROUGH + + 4.1.2.1 LOCAL Type: RFC-959 Section 3.1.1.4 + + An FTP program MUST support TYPE I ("IMAGE" or binary type) + as well as TYPE L 8 ("LOCAL" type with logical byte size 8). + A machine whose memory is organized into m-bit words, where + m is not a multiple of 8, MAY also support TYPE L m. + + DISCUSSION: + The command "TYPE L 8" is often required to transfer + binary data between a machine whose memory is organized + into (e.g.) 36-bit words and a machine with an 8-bit + byte organization. For an 8-bit byte machine, TYPE L 8 + is equivalent to IMAGE. + + "TYPE L m" is sometimes specified to the FTP programs + on two m-bit word machines to ensure the correct + transfer of a native-mode binary file from one machine + to the other. However, this command should have the + same effect on these machines as "TYPE I". + + + + +Internet Engineering Task Force [Page 29] + + + + +RFC1123 FILE TRANSFER -- FTP October 1989 + + + 4.1.2.2 Telnet Format Control: RFC-959 Section 3.1.1.5.2 + + A host that makes no distinction between TYPE N and TYPE T + SHOULD implement TYPE T to be identical to TYPE N. + + DISCUSSION: + This provision should ease interoperation with hosts + that do make this distinction. + + Many hosts represent text files internally as strings + of ASCII characters, using the embedded ASCII format + effector characters (LF, BS, FF, ...) to control the + format when a file is printed. For such hosts, there + is no distinction between "print" files and other + files. However, systems that use record structured + files typically need a special format for printable + files (e.g., ASA carriage control). For the latter + hosts, FTP allows a choice of TYPE N or TYPE T. + + 4.1.2.3 Page Structure: RFC-959 Section 3.1.2.3 and Appendix I + + Implementation of page structure is NOT RECOMMENDED in + general. However, if a host system does need to implement + FTP for "random access" or "holey" files, it MUST use the + defined page structure format rather than define a new + private FTP format. + + 4.1.2.4 Data Structure Transformations: RFC-959 Section 3.1.2 + + An FTP transformation between record-structure and file- + structure SHOULD be invertible, to the extent possible while + making the result useful on the target host. + + DISCUSSION: + RFC-959 required strict invertibility between record- + structure and file-structure, but in practice, + efficiency and convenience often preclude it. + Therefore, the requirement is being relaxed. There are + two different objectives for transferring a file: + processing it on the target host, or just storage. For + storage, strict invertibility is important. For + processing, the file created on the target host needs + to be in the format expected by application programs on + that host. + + As an example of the conflict, imagine a record- + oriented operating system that requires some data files + to have exactly 80 bytes in each record. While STORing + + + +Internet Engineering Task Force [Page 30] + + + + +RFC1123 FILE TRANSFER -- FTP October 1989 + + + a file on such a host, an FTP Server must be able to + pad each line or record to 80 bytes; a later retrieval + of such a file cannot be strictly invertible. + + 4.1.2.5 Data Connection Management: RFC-959 Section 3.3 + + A User-FTP that uses STREAM mode SHOULD send a PORT command + to assign a non-default data port before each transfer + command is issued. + + DISCUSSION: + This is required because of the long delay after a TCP + connection is closed until its socket pair can be + reused, to allow multiple transfers during a single FTP + session. Sending a port command can avoided if a + transfer mode other than stream is used, by leaving the + data transfer connection open between transfers. + + 4.1.2.6 PASV Command: RFC-959 Section 4.1.2 + + A server-FTP MUST implement the PASV command. + + If multiple third-party transfers are to be executed during + the same session, a new PASV command MUST be issued before + each transfer command, to obtain a unique port pair. + + IMPLEMENTATION: + The format of the 227 reply to a PASV command is not + well standardized. In particular, an FTP client cannot + assume that the parentheses shown on page 40 of RFC-959 + will be present (and in fact, Figure 3 on page 43 omits + them). Therefore, a User-FTP program that interprets + the PASV reply must scan the reply for the first digit + of the host and port numbers. + + Note that the host number h1,h2,h3,h4 is the IP address + of the server host that is sending the reply, and that + p1,p2 is a non-default data transfer port that PASV has + assigned. + + 4.1.2.7 LIST and NLST Commands: RFC-959 Section 4.1.3 + + The data returned by an NLST command MUST contain only a + simple list of legal pathnames, such that the server can use + them directly as the arguments of subsequent data transfer + commands for the individual files. + + The data returned by a LIST or NLST command SHOULD use an + + + +Internet Engineering Task Force [Page 31] + + + + +RFC1123 FILE TRANSFER -- FTP October 1989 + + + implied TYPE AN, unless the current type is EBCDIC, in which + case an implied TYPE EN SHOULD be used. + + DISCUSSION: + Many FTP clients support macro-commands that will get + or put files matching a wildcard specification, using + NLST to obtain a list of pathnames. The expansion of + "multiple-put" is local to the client, but "multiple- + get" requires cooperation by the server. + + The implied type for LIST and NLST is designed to + provide compatibility with existing User-FTPs, and in + particular with multiple-get commands. + + 4.1.2.8 SITE Command: RFC-959 Section 4.1.3 + + A Server-FTP SHOULD use the SITE command for non-standard + features, rather than invent new private commands or + unstandardized extensions to existing commands. + + 4.1.2.9 STOU Command: RFC-959 Section 4.1.3 + + The STOU command stores into a uniquely named file. When it + receives an STOU command, a Server-FTP MUST return the + actual file name in the "125 Transfer Starting" or the "150 + Opening Data Connection" message that precedes the transfer + (the 250 reply code mentioned in RFC-959 is incorrect). The + exact format of these messages is hereby defined to be as + follows: + + 125 FILE: pppp + 150 FILE: pppp + + where pppp represents the unique pathname of the file that + will be written. + + 4.1.2.10 Telnet End-of-line Code: RFC-959, Page 34 + + Implementors MUST NOT assume any correspondence between READ + boundaries on the control connection and the Telnet EOL + sequences (CR LF). + + DISCUSSION: + Thus, a server-FTP (or User-FTP) must continue reading + characters from the control connection until a complete + Telnet EOL sequence is encountered, before processing + the command (or response, respectively). Conversely, a + single READ from the control connection may include + + + +Internet Engineering Task Force [Page 32] + + + + +RFC1123 FILE TRANSFER -- FTP October 1989 + + + more than one FTP command. + + 4.1.2.11 FTP Replies: RFC-959 Section 4.2, Page 35 + + A Server-FTP MUST send only correctly formatted replies on + the control connection. Note that RFC-959 (unlike earlier + versions of the FTP spec) contains no provision for a + "spontaneous" reply message. + + A Server-FTP SHOULD use the reply codes defined in RFC-959 + whenever they apply. However, a server-FTP MAY use a + different reply code when needed, as long as the general + rules of Section 4.2 are followed. When the implementor has + a choice between a 4xx and 5xx reply code, a Server-FTP + SHOULD send a 4xx (temporary failure) code when there is any + reasonable possibility that a failed FTP will succeed a few + hours later. + + A User-FTP SHOULD generally use only the highest-order digit + of a 3-digit reply code for making a procedural decision, to + prevent difficulties when a Server-FTP uses non-standard + reply codes. + + A User-FTP MUST be able to handle multi-line replies. If + the implementation imposes a limit on the number of lines + and if this limit is exceeded, the User-FTP MUST recover, + e.g., by ignoring the excess lines until the end of the + multi-line reply is reached. + + A User-FTP SHOULD NOT interpret a 421 reply code ("Service + not available, closing control connection") specially, but + SHOULD detect closing of the control connection by the + server. + + DISCUSSION: + Server implementations that fail to strictly follow the + reply rules often cause FTP user programs to hang. + Note that RFC-959 resolved ambiguities in the reply + rules found in earlier FTP specifications and must be + followed. + + It is important to choose FTP reply codes that properly + distinguish between temporary and permanent failures, + to allow the successful use of file transfer client + daemons. These programs depend on the reply codes to + decide whether or not to retry a failed transfer; using + a permanent failure code (5xx) for a temporary error + will cause these programs to give up unnecessarily. + + + +Internet Engineering Task Force [Page 33] + + + + +RFC1123 FILE TRANSFER -- FTP October 1989 + + + When the meaning of a reply matches exactly the text + shown in RFC-959, uniformity will be enhanced by using + the RFC-959 text verbatim. However, a Server-FTP + implementor is encouraged to choose reply text that + conveys specific system-dependent information, when + appropriate. + + 4.1.2.12 Connections: RFC-959 Section 5.2 + + The words "and the port used" in the second paragraph of + this section of RFC-959 are erroneous (historical), and they + should be ignored. + + On a multihomed server host, the default data transfer port + (L-1) MUST be associated with the same local IP address as + the corresponding control connection to port L. + + A user-FTP MUST NOT send any Telnet controls other than + SYNCH and IP on an FTP control connection. In particular, it + MUST NOT attempt to negotiate Telnet options on the control + connection. However, a server-FTP MUST be capable of + accepting and refusing Telnet negotiations (i.e., sending + DONT/WONT). + + DISCUSSION: + Although the RFC says: "Server- and User- processes + should follow the conventions for the Telnet + protocol...[on the control connection]", it is not the + intent that Telnet option negotiation is to be + employed. + + 4.1.2.13 Minimum Implementation; RFC-959 Section 5.1 + + The following commands and options MUST be supported by + every server-FTP and user-FTP, except in cases where the + underlying file system or operating system does not allow or + support a particular command. + + Type: ASCII Non-print, IMAGE, LOCAL 8 + Mode: Stream + Structure: File, Record* + Commands: + USER, PASS, ACCT, + PORT, PASV, + TYPE, MODE, STRU, + RETR, STOR, APPE, + RNFR, RNTO, DELE, + CWD, CDUP, RMD, MKD, PWD, + + + +Internet Engineering Task Force [Page 34] + + + + +RFC1123 FILE TRANSFER -- FTP October 1989 + + + LIST, NLST, + SYST, STAT, + HELP, NOOP, QUIT. + + *Record structure is REQUIRED only for hosts whose file + systems support record structure. + + DISCUSSION: + Vendors are encouraged to implement a larger subset of + the protocol. For example, there are important + robustness features in the protocol (e.g., Restart, + ABOR, block mode) that would be an aid to some Internet + users but are not widely implemented. + + A host that does not have record structures in its file + system may still accept files with STRU R, recording + the byte stream literally. + + 4.1.3 SPECIFIC ISSUES + + 4.1.3.1 Non-standard Command Verbs + + FTP allows "experimental" commands, whose names begin with + "X". If these commands are subsequently adopted as + standards, there may still be existing implementations using + the "X" form. At present, this is true for the directory + commands: + + RFC-959 "Experimental" + + MKD XMKD + RMD XRMD + PWD XPWD + CDUP XCUP + CWD XCWD + + All FTP implementations SHOULD recognize both forms of these + commands, by simply equating them with extra entries in the + command lookup table. + + IMPLEMENTATION: + A User-FTP can access a server that supports only the + "X" forms by implementing a mode switch, or + automatically using the following procedure: if the + RFC-959 form of one of the above commands is rejected + with a 500 or 502 response code, then try the + experimental form; any other response would be passed + to the user. + + + +Internet Engineering Task Force [Page 35] + + + + +RFC1123 FILE TRANSFER -- FTP October 1989 + + + 4.1.3.2 Idle Timeout + + A Server-FTP process SHOULD have an idle timeout, which will + terminate the process and close the control connection if + the server is inactive (i.e., no command or data transfer in + progress) for a long period of time. The idle timeout time + SHOULD be configurable, and the default should be at least 5 + minutes. + + A client FTP process ("User-PI" in RFC-959) will need + timeouts on responses only if it is invoked from a program. + + DISCUSSION: + Without a timeout, a Server-FTP process may be left + pending indefinitely if the corresponding client + crashes without closing the control connection. + + 4.1.3.3 Concurrency of Data and Control + + DISCUSSION: + The intent of the designers of FTP was that a user + should be able to send a STAT command at any time while + data transfer was in progress and that the server-FTP + would reply immediately with status -- e.g., the number + of bytes transferred so far. Similarly, an ABOR + command should be possible at any time during a data + transfer. + + Unfortunately, some small-machine operating systems + make such concurrent programming difficult, and some + other implementers seek minimal solutions, so some FTP + implementations do not allow concurrent use of the data + and control connections. Even such a minimal server + must be prepared to accept and defer a STAT or ABOR + command that arrives during data transfer. + + 4.1.3.4 FTP Restart Mechanism + + The description of the 110 reply on pp. 40-41 of RFC-959 is + incorrect; the correct description is as follows. A restart + reply message, sent over the control connection from the + receiving FTP to the User-FTP, has the format: + + 110 MARK ssss = rrrr + + Here: + + * ssss is a text string that appeared in a Restart Marker + + + +Internet Engineering Task Force [Page 36] + + + + +RFC1123 FILE TRANSFER -- FTP October 1989 + + + in the data stream and encodes a position in the + sender's file system; + + * rrrr encodes the corresponding position in the + receiver's file system. + + The encoding, which is specific to a particular file system + and network implementation, is always generated and + interpreted by the same system, either sender or receiver. + + When an FTP that implements restart receives a Restart + Marker in the data stream, it SHOULD force the data to that + point to be written to stable storage before encoding the + corresponding position rrrr. An FTP sending Restart Markers + MUST NOT assume that 110 replies will be returned + synchronously with the data, i.e., it must not await a 110 + reply before sending more data. + + Two new reply codes are hereby defined for errors + encountered in restarting a transfer: + + 554 Requested action not taken: invalid REST parameter. + + A 554 reply may result from a FTP service command that + follows a REST command. The reply indicates that the + existing file at the Server-FTP cannot be repositioned + as specified in the REST. + + 555 Requested action not taken: type or stru mismatch. + + A 555 reply may result from an APPE command or from any + FTP service command following a REST command. The + reply indicates that there is some mismatch between the + current transfer parameters (type and stru) and the + attributes of the existing file. + + DISCUSSION: + Note that the FTP Restart mechanism requires that Block + or Compressed mode be used for data transfer, to allow + the Restart Markers to be included within the data + stream. The frequency of Restart Markers can be low. + + Restart Markers mark a place in the data stream, but + the receiver may be performing some transformation on + the data as it is stored into stable storage. In + general, the receiver's encoding must include any state + information necessary to restart this transformation at + any point of the FTP data stream. For example, in TYPE + + + +Internet Engineering Task Force [Page 37] + + + + +RFC1123 FILE TRANSFER -- FTP October 1989 + + + A transfers, some receiver hosts transform CR LF + sequences into a single LF character on disk. If a + Restart Marker happens to fall between CR and LF, the + receiver must encode in rrrr that the transfer must be + restarted in a "CR has been seen and discarded" state. + + Note that the Restart Marker is required to be encoded + as a string of printable ASCII characters, regardless + of the type of the data. + + RFC-959 says that restart information is to be returned + "to the user". This should not be taken literally. In + general, the User-FTP should save the restart + information (ssss,rrrr) in stable storage, e.g., append + it to a restart control file. An empty restart control + file should be created when the transfer first starts + and deleted automatically when the transfer completes + successfully. It is suggested that this file have a + name derived in an easily-identifiable manner from the + name of the file being transferred and the remote host + name; this is analogous to the means used by many text + editors for naming "backup" files. + + There are three cases for FTP restart. + + (1) User-to-Server Transfer + + The User-FTP puts Restart Markers at + convenient places in the data stream. When the + Server-FTP receives a Marker, it writes all prior + data to disk, encodes its file system position and + transformation state as rrrr, and returns a "110 + MARK ssss = rrrr" reply over the control + connection. The User-FTP appends the pair + (ssss,rrrr) to its restart control file. + + To restart the transfer, the User-FTP fetches the + last (ssss,rrrr) pair from the restart control + file, repositions its local file system and + transformation state using ssss, and sends the + command "REST rrrr" to the Server-FTP. + + (2) Server-to-User Transfer + + The Server-FTP puts Restart Markers at + convenient places in the data stream. When the + User-FTP receives a Marker, it writes all prior + data to disk, encodes its file system position and + + + +Internet Engineering Task Force [Page 38] + + + + +RFC1123 FILE TRANSFER -- FTP October 1989 + + + transformation state as rrrr, and appends the pair + (rrrr,ssss) to its restart control file. + + To restart the transfer, the User-FTP fetches the + last (rrrr,ssss) pair from the restart control + file, repositions its local file system and + transformation state using rrrr, and sends the + command "REST ssss" to the Server-FTP. + + (3) Server-to-Server ("Third-Party") Transfer + + The sending Server-FTP puts Restart Markers + at convenient places in the data stream. When it + receives a Marker, the receiving Server-FTP writes + all prior data to disk, encodes its file system + position and transformation state as rrrr, and + sends a "110 MARK ssss = rrrr" reply over the + control connection to the User. The User-FTP + appends the pair (ssss,rrrr) to its restart + control file. + + To restart the transfer, the User-FTP fetches the + last (ssss,rrrr) pair from the restart control + file, sends "REST ssss" to the sending Server-FTP, + and sends "REST rrrr" to the receiving Server-FTP. + + + 4.1.4 FTP/USER INTERFACE + + This section discusses the user interface for a User-FTP + program. + + 4.1.4.1 Pathname Specification + + Since FTP is intended for use in a heterogeneous + environment, User-FTP implementations MUST support remote + pathnames as arbitrary character strings, so that their form + and content are not limited by the conventions of the local + operating system. + + DISCUSSION: + In particular, remote pathnames can be of arbitrary + length, and all the printing ASCII characters as well + as space (0x20) must be allowed. RFC-959 allows a + pathname to contain any 7-bit ASCII character except CR + or LF. + + + + + +Internet Engineering Task Force [Page 39] + + + + +RFC1123 FILE TRANSFER -- FTP October 1989 + + + 4.1.4.2 "QUOTE" Command + + A User-FTP program MUST implement a "QUOTE" command that + will pass an arbitrary character string to the server and + display all resulting response messages to the user. + + To make the "QUOTE" command useful, a User-FTP SHOULD send + transfer control commands to the server as the user enters + them, rather than saving all the commands and sending them + to the server only when a data transfer is started. + + DISCUSSION: + The "QUOTE" command is essential to allow the user to + access servers that require system-specific commands + (e.g., SITE or ALLO), or to invoke new or optional + features that are not implemented by the User-FTP. For + example, "QUOTE" may be used to specify "TYPE A T" to + send a print file to hosts that require the + distinction, even if the User-FTP does not recognize + that TYPE. + + 4.1.4.3 Displaying Replies to User + + A User-FTP SHOULD display to the user the full text of all + error reply messages it receives. It SHOULD have a + "verbose" mode in which all commands it sends and the full + text and reply codes it receives are displayed, for + diagnosis of problems. + + 4.1.4.4 Maintaining Synchronization + + The state machine in a User-FTP SHOULD be forgiving of + missing and unexpected reply messages, in order to maintain + command synchronization with the server. + + + + + + + + + + + + + + + + + +Internet Engineering Task Force [Page 40] + + + + +RFC1123 FILE TRANSFER -- FTP October 1989 + + + 4.1.5 FTP REQUIREMENTS SUMMARY + + | | | | |S| | + | | | | |H| |F + | | | | |O|M|o + | | |S| |U|U|o + | | |H| |L|S|t + | |M|O| |D|T|n + | |U|U|M| | |o + | |S|L|A|N|N|t + | |T|D|Y|O|O|t +FEATURE |SECTION | | | |T|T|e +-------------------------------------------|---------------|-|-|-|-|-|-- +Implement TYPE T if same as TYPE N |4.1.2.2 | |x| | | | +File/Record transform invertible if poss. |4.1.2.4 | |x| | | | +User-FTP send PORT cmd for stream mode |4.1.2.5 | |x| | | | +Server-FTP implement PASV |4.1.2.6 |x| | | | | + PASV is per-transfer |4.1.2.6 |x| | | | | +NLST reply usable in RETR cmds |4.1.2.7 |x| | | | | +Implied type for LIST and NLST |4.1.2.7 | |x| | | | +SITE cmd for non-standard features |4.1.2.8 | |x| | | | +STOU cmd return pathname as specified |4.1.2.9 |x| | | | | +Use TCP READ boundaries on control conn. |4.1.2.10 | | | | |x| + | | | | | | | +Server-FTP send only correct reply format |4.1.2.11 |x| | | | | +Server-FTP use defined reply code if poss. |4.1.2.11 | |x| | | | + New reply code following Section 4.2 |4.1.2.11 | | |x| | | +User-FTP use only high digit of reply |4.1.2.11 | |x| | | | +User-FTP handle multi-line reply lines |4.1.2.11 |x| | | | | +User-FTP handle 421 reply specially |4.1.2.11 | | | |x| | + | | | | | | | +Default data port same IP addr as ctl conn |4.1.2.12 |x| | | | | +User-FTP send Telnet cmds exc. SYNCH, IP |4.1.2.12 | | | | |x| +User-FTP negotiate Telnet options |4.1.2.12 | | | | |x| +Server-FTP handle Telnet options |4.1.2.12 |x| | | | | +Handle "Experimental" directory cmds |4.1.3.1 | |x| | | | +Idle timeout in server-FTP |4.1.3.2 | |x| | | | + Configurable idle timeout |4.1.3.2 | |x| | | | +Receiver checkpoint data at Restart Marker |4.1.3.4 | |x| | | | +Sender assume 110 replies are synchronous |4.1.3.4 | | | | |x| + | | | | | | | +Support TYPE: | | | | | | | + ASCII - Non-Print (AN) |4.1.2.13 |x| | | | | + ASCII - Telnet (AT) -- if same as AN |4.1.2.2 | |x| | | | + ASCII - Carriage Control (AC) |959 3.1.1.5.2 | | |x| | | + EBCDIC - (any form) |959 3.1.1.2 | | |x| | | + IMAGE |4.1.2.1 |x| | | | | + LOCAL 8 |4.1.2.1 |x| | | | | + + + +Internet Engineering Task Force [Page 41] + + + + +RFC1123 FILE TRANSFER -- FTP October 1989 + + + LOCAL m |4.1.2.1 | | |x| | |2 + | | | | | | | +Support MODE: | | | | | | | + Stream |4.1.2.13 |x| | | | | + Block |959 3.4.2 | | |x| | | + | | | | | | | +Support STRUCTURE: | | | | | | | + File |4.1.2.13 |x| | | | | + Record |4.1.2.13 |x| | | | |3 + Page |4.1.2.3 | | | |x| | + | | | | | | | +Support commands: | | | | | | | + USER |4.1.2.13 |x| | | | | + PASS |4.1.2.13 |x| | | | | + ACCT |4.1.2.13 |x| | | | | + CWD |4.1.2.13 |x| | | | | + CDUP |4.1.2.13 |x| | | | | + SMNT |959 5.3.1 | | |x| | | + REIN |959 5.3.1 | | |x| | | + QUIT |4.1.2.13 |x| | | | | + | | | | | | | + PORT |4.1.2.13 |x| | | | | + PASV |4.1.2.6 |x| | | | | + TYPE |4.1.2.13 |x| | | | |1 + STRU |4.1.2.13 |x| | | | |1 + MODE |4.1.2.13 |x| | | | |1 + | | | | | | | + RETR |4.1.2.13 |x| | | | | + STOR |4.1.2.13 |x| | | | | + STOU |959 5.3.1 | | |x| | | + APPE |4.1.2.13 |x| | | | | + ALLO |959 5.3.1 | | |x| | | + REST |959 5.3.1 | | |x| | | + RNFR |4.1.2.13 |x| | | | | + RNTO |4.1.2.13 |x| | | | | + ABOR |959 5.3.1 | | |x| | | + DELE |4.1.2.13 |x| | | | | + RMD |4.1.2.13 |x| | | | | + MKD |4.1.2.13 |x| | | | | + PWD |4.1.2.13 |x| | | | | + LIST |4.1.2.13 |x| | | | | + NLST |4.1.2.13 |x| | | | | + SITE |4.1.2.8 | | |x| | | + STAT |4.1.2.13 |x| | | | | + SYST |4.1.2.13 |x| | | | | + HELP |4.1.2.13 |x| | | | | + NOOP |4.1.2.13 |x| | | | | + | | | | | | | + + + +Internet Engineering Task Force [Page 42] + + + + +RFC1123 FILE TRANSFER -- FTP October 1989 + + +User Interface: | | | | | | | + Arbitrary pathnames |4.1.4.1 |x| | | | | + Implement "QUOTE" command |4.1.4.2 |x| | | | | + Transfer control commands immediately |4.1.4.2 | |x| | | | + Display error messages to user |4.1.4.3 | |x| | | | + Verbose mode |4.1.4.3 | |x| | | | + Maintain synchronization with server |4.1.4.4 | |x| | | | + +Footnotes: + +(1) For the values shown earlier. + +(2) Here m is number of bits in a memory word. + +(3) Required for host with record-structured file system, optional + otherwise. + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +Internet Engineering Task Force [Page 43] + + + + +RFC1123 FILE TRANSFER -- TFTP October 1989 + + + 4.2 TRIVIAL FILE TRANSFER PROTOCOL -- TFTP + + 4.2.1 INTRODUCTION + + The Trivial File Transfer Protocol TFTP is defined in RFC-783 + [TFTP:1]. + + TFTP provides its own reliable delivery with UDP as its + transport protocol, using a simple stop-and-wait acknowledgment + system. Since TFTP has an effective window of only one 512 + octet segment, it can provide good performance only over paths + that have a small delay*bandwidth product. The TFTP file + interface is very simple, providing no access control or + security. + + TFTP's most important application is bootstrapping a host over + a local network, since it is simple and small enough to be + easily implemented in EPROM [BOOT:1, BOOT:2]. Vendors are + urged to support TFTP for booting. + + 4.2.2 PROTOCOL WALK-THROUGH + + The TFTP specification [TFTP:1] is written in an open style, + and does not fully specify many parts of the protocol. + + 4.2.2.1 Transfer Modes: RFC-783, Page 3 + + The transfer mode "mail" SHOULD NOT be supported. + + 4.2.2.2 UDP Header: RFC-783, Page 17 + + The Length field of a UDP header is incorrectly defined; it + includes the UDP header length (8). + + 4.2.3 SPECIFIC ISSUES + + 4.2.3.1 Sorcerer's Apprentice Syndrome + + There is a serious bug, known as the "Sorcerer's Apprentice + Syndrome," in the protocol specification. While it does not + cause incorrect operation of the transfer (the file will + always be transferred correctly if the transfer completes), + this bug may cause excessive retransmission, which may cause + the transfer to time out. + + Implementations MUST contain the fix for this problem: the + sender (i.e., the side originating the DATA packets) must + never resend the current DATA packet on receipt of a + + + +Internet Engineering Task Force [Page 44] + + + + +RFC1123 FILE TRANSFER -- TFTP October 1989 + + + duplicate ACK. + + DISCUSSION: + The bug is caused by the protocol rule that either + side, on receiving an old duplicate datagram, may + resend the current datagram. If a packet is delayed in + the network but later successfully delivered after + either side has timed out and retransmitted a packet, a + duplicate copy of the response may be generated. If + the other side responds to this duplicate with a + duplicate of its own, then every datagram will be sent + in duplicate for the remainder of the transfer (unless + a datagram is lost, breaking the repetition). Worse + yet, since the delay is often caused by congestion, + this duplicate transmission will usually causes more + congestion, leading to more delayed packets, etc. + + The following example may help to clarify this problem. + + TFTP A TFTP B + + (1) Receive ACK X-1 + Send DATA X + (2) Receive DATA X + Send ACK X + (ACK X is delayed in network, + and A times out): + (3) Retransmit DATA X + + (4) Receive DATA X again + Send ACK X again + (5) Receive (delayed) ACK X + Send DATA X+1 + (6) Receive DATA X+1 + Send ACK X+1 + (7) Receive ACK X again + Send DATA X+1 again + (8) Receive DATA X+1 again + Send ACK X+1 again + (9) Receive ACK X+1 + Send DATA X+2 + (10) Receive DATA X+2 + Send ACK X+3 + (11) Receive ACK X+1 again + Send DATA X+2 again + (12) Receive DATA X+2 again + Send ACK X+3 again + + + + +Internet Engineering Task Force [Page 45] + + + + +RFC1123 FILE TRANSFER -- TFTP October 1989 + + + Notice that once the delayed ACK arrives, the protocol + settles down to duplicate all further packets + (sequences 5-8 and 9-12). The problem is caused not by + either side timing out, but by both sides + retransmitting the current packet when they receive a + duplicate. + + The fix is to break the retransmission loop, as + indicated above. This is analogous to the behavior of + TCP. It is then possible to remove the retransmission + timer on the receiver, since the resent ACK will never + cause any action; this is a useful simplification where + TFTP is used in a bootstrap program. It is OK to allow + the timer to remain, and it may be helpful if the + retransmitted ACK replaces one that was genuinely lost + in the network. The sender still requires a retransmit + timer, of course. + + 4.2.3.2 Timeout Algorithms + + A TFTP implementation MUST use an adaptive timeout. + + IMPLEMENTATION: + TCP retransmission algorithms provide a useful base to + work from. At least an exponential backoff of + retransmission timeout is necessary. + + 4.2.3.3 Extensions + + A variety of non-standard extensions have been made to TFTP, + including additional transfer modes and a secure operation + mode (with passwords). None of these have been + standardized. + + 4.2.3.4 Access Control + + A server TFTP implementation SHOULD include some + configurable access control over what pathnames are allowed + in TFTP operations. + + 4.2.3.5 Broadcast Request + + A TFTP request directed to a broadcast address SHOULD be + silently ignored. + + DISCUSSION: + Due to the weak access control capability of TFTP, + directed broadcasts of TFTP requests to random networks + + + +Internet Engineering Task Force [Page 46] + + + + +RFC1123 FILE TRANSFER -- TFTP October 1989 + + + could create a significant security hole. + + 4.2.4 TFTP REQUIREMENTS SUMMARY + + | | | | |S| | + | | | | |H| |F + | | | | |O|M|o + | | |S| |U|U|o + | | |H| |L|S|t + | |M|O| |D|T|n + | |U|U|M| | |o + | |S|L|A|N|N|t + | |T|D|Y|O|O|t +FEATURE |SECTION | | | |T|T|e +-------------------------------------------------|--------|-|-|-|-|-|-- +Fix Sorcerer's Apprentice Syndrome |4.2.3.1 |x| | | | | +Transfer modes: | | | | | | | + netascii |RFC-783 |x| | | | | + octet |RFC-783 |x| | | | | + mail |4.2.2.1 | | | |x| | + extensions |4.2.3.3 | | |x| | | +Use adaptive timeout |4.2.3.2 |x| | | | | +Configurable access control |4.2.3.4 | |x| | | | +Silently ignore broadcast request |4.2.3.5 | |x| | | | +-------------------------------------------------|--------|-|-|-|-|-|-- +-------------------------------------------------|--------|-|-|-|-|-|-- + + + + + + + + + + + + + + + + + + + + + + + + + +Internet Engineering Task Force [Page 47] + + + + +RFC1123 MAIL -- SMTP & RFC-822 October 1989 + + +5. ELECTRONIC MAIL -- SMTP and RFC-822 + + 5.1 INTRODUCTION + + In the TCP/IP protocol suite, electronic mail in a format + specified in RFC-822 [SMTP:2] is transmitted using the Simple Mail + Transfer Protocol (SMTP) defined in RFC-821 [SMTP:1]. + + While SMTP has remained unchanged over the years, the Internet + community has made several changes in the way SMTP is used. In + particular, the conversion to the Domain Name System (DNS) has + caused changes in address formats and in mail routing. In this + section, we assume familiarity with the concepts and terminology + of the DNS, whose requirements are given in Section 6.1. + + RFC-822 specifies the Internet standard format for electronic mail + messages. RFC-822 supercedes an older standard, RFC-733, that may + still be in use in a few places, although it is obsolete. The two + formats are sometimes referred to simply by number ("822" and + "733"). + + RFC-822 is used in some non-Internet mail environments with + different mail transfer protocols than SMTP, and SMTP has also + been adapted for use in some non-Internet environments. Note that + this document presents the rules for the use of SMTP and RFC-822 + for the Internet environment only; other mail environments that + use these protocols may be expected to have their own rules. + + 5.2 PROTOCOL WALK-THROUGH + + This section covers both RFC-821 and RFC-822. + + The SMTP specification in RFC-821 is clear and contains numerous + examples, so implementors should not find it difficult to + understand. This section simply updates or annotates portions of + RFC-821 to conform with current usage. + + RFC-822 is a long and dense document, defining a rich syntax. + Unfortunately, incomplete or defective implementations of RFC-822 + are common. In fact, nearly all of the many formats of RFC-822 + are actually used, so an implementation generally needs to + recognize and correctly interpret all of the RFC-822 syntax. + + 5.2.1 The SMTP Model: RFC-821 Section 2 + + DISCUSSION: + Mail is sent by a series of request/response transactions + between a client, the "sender-SMTP," and a server, the + + + +Internet Engineering Task Force [Page 48] + + + + +RFC1123 MAIL -- SMTP & RFC-822 October 1989 + + + "receiver-SMTP". These transactions pass (1) the message + proper, which is composed of header and body, and (2) SMTP + source and destination addresses, referred to as the + "envelope". + + The SMTP programs are analogous to Message Transfer Agents + (MTAs) of X.400. There will be another level of protocol + software, closer to the end user, that is responsible for + composing and analyzing RFC-822 message headers; this + component is known as the "User Agent" in X.400, and we + use that term in this document. There is a clear logical + distinction between the User Agent and the SMTP + implementation, since they operate on different levels of + protocol. Note, however, that this distinction is may not + be exactly reflected the structure of typical + implementations of Internet mail. Often there is a + program known as the "mailer" that implements SMTP and + also some of the User Agent functions; the rest of the + User Agent functions are included in a user interface used + for entering and reading mail. + + The SMTP envelope is constructed at the originating site, + typically by the User Agent when the message is first + queued for the Sender-SMTP program. The envelope + addresses may be derived from information in the message + header, supplied by the user interface (e.g., to implement + a bcc: request), or derived from local configuration + information (e.g., expansion of a mailing list). The SMTP + envelope cannot in general be re-derived from the header + at a later stage in message delivery, so the envelope is + transmitted separately from the message itself using the + MAIL and RCPT commands of SMTP. + + The text of RFC-821 suggests that mail is to be delivered + to an individual user at a host. With the advent of the + domain system and of mail routing using mail-exchange (MX) + resource records, implementors should now think of + delivering mail to a user at a domain, which may or may + not be a particular host. This DOES NOT change the fact + that SMTP is a host-to-host mail exchange protocol. + + 5.2.2 Canonicalization: RFC-821 Section 3.1 + + The domain names that a Sender-SMTP sends in MAIL and RCPT + commands MUST have been "canonicalized," i.e., they must be + fully-qualified principal names or domain literals, not + nicknames or domain abbreviations. A canonicalized name either + identifies a host directly or is an MX name; it cannot be a + + + +Internet Engineering Task Force [Page 49] + + + + +RFC1123 MAIL -- SMTP & RFC-822 October 1989 + + + CNAME. + + 5.2.3 VRFY and EXPN Commands: RFC-821 Section 3.3 + + A receiver-SMTP MUST implement VRFY and SHOULD implement EXPN + (this requirement overrides RFC-821). However, there MAY be + configuration information to disable VRFY and EXPN in a + particular installation; this might even allow EXPN to be + disabled for selected lists. + + A new reply code is defined for the VRFY command: + + 252 Cannot VRFY user (e.g., info is not local), but will + take message for this user and attempt delivery. + + DISCUSSION: + SMTP users and administrators make regular use of these + commands for diagnosing mail delivery problems. With the + increasing use of multi-level mailing list expansion + (sometimes more than two levels), EXPN has been + increasingly important for diagnosing inadvertent mail + loops. On the other hand, some feel that EXPN represents + a significant privacy, and perhaps even a security, + exposure. + + 5.2.4 SEND, SOML, and SAML Commands: RFC-821 Section 3.4 + + An SMTP MAY implement the commands to send a message to a + user's terminal: SEND, SOML, and SAML. + + DISCUSSION: + It has been suggested that the use of mail relaying + through an MX record is inconsistent with the intent of + SEND to deliver a message immediately and directly to a + user's terminal. However, an SMTP receiver that is unable + to write directly to the user terminal can return a "251 + User Not Local" reply to the RCPT following a SEND, to + inform the originator of possibly deferred delivery. + + 5.2.5 HELO Command: RFC-821 Section 3.5 + + The sender-SMTP MUST ensure that the parameter in a + HELO command is a valid principal host domain name for the + client host. As a result, the receiver-SMTP will not have to + perform MX resolution on this name in order to validate the + HELO parameter. + + The HELO receiver MAY verify that the HELO parameter really + + + +Internet Engineering Task Force [Page 50] + + + + +RFC1123 MAIL -- SMTP & RFC-822 October 1989 + + + corresponds to the IP address of the sender. However, the + receiver MUST NOT refuse to accept a message, even if the + sender's HELO command fails verification. + + DISCUSSION: + Verifying the HELO parameter requires a domain name lookup + and may therefore take considerable time. An alternative + tool for tracking bogus mail sources is suggested below + (see "DATA Command"). + + Note also that the HELO argument is still required to have + valid syntax, since it will appear in a Received: + line; otherwise, a 501 error is to be sent. + + IMPLEMENTATION: + When HELO parameter validation fails, a suggested + procedure is to insert a note about the unknown + authenticity of the sender into the message header (e.g., + in the "Received:" line). + + 5.2.6 Mail Relay: RFC-821 Section 3.6 + + We distinguish three types of mail (store-and-) forwarding: + + (1) A simple forwarder or "mail exchanger" forwards a message + using private knowledge about the recipient; see section + 3.2 of RFC-821. + + (2) An SMTP mail "relay" forwards a message within an SMTP + mail environment as the result of an explicit source route + (as defined in section 3.6 of RFC-821). The SMTP relay + function uses the "@...:" form of source route from RFC- + 822 (see Section 5.2.19 below). + + (3) A mail "gateway" passes a message between different + environments. The rules for mail gateways are discussed + below in Section 5.3.7. + + An Internet host that is forwarding a message but is not a + gateway to a different mail environment (i.e., it falls under + (1) or (2)) SHOULD NOT alter any existing header fields, + although the host will add an appropriate Received: line as + required in Section 5.2.8. + + A Sender-SMTP SHOULD NOT send a RCPT TO: command containing an + explicit source route using the "@...:" address form. Thus, + the relay function defined in section 3.6 of RFC-821 should + not be used. + + + +Internet Engineering Task Force [Page 51] + + + + +RFC1123 MAIL -- SMTP & RFC-822 October 1989 + + + DISCUSSION: + The intent is to discourage all source routing and to + abolish explicit source routing for mail delivery within + the Internet environment. Source-routing is unnecessary; + the simple target address "user@domain" should always + suffice. This is the result of an explicit architectural + decision to use universal naming rather than source + routing for mail. Thus, SMTP provides end-to-end + connectivity, and the DNS provides globally-unique, + location-independent names. MX records handle the major + case where source routing might otherwise be needed. + + A receiver-SMTP MUST accept the explicit source route syntax in + the envelope, but it MAY implement the relay function as + defined in section 3.6 of RFC-821. If it does not implement + the relay function, it SHOULD attempt to deliver the message + directly to the host to the right of the right-most "@" sign. + + DISCUSSION: + For example, suppose a host that does not implement the + relay function receives a message with the SMTP command: + "RCPT TO:<@ALPHA,@BETA:joe@GAMMA>", where ALPHA, BETA, and + GAMMA represent domain names. Rather than immediately + refusing the message with a 550 error reply as suggested + on page 20 of RFC-821, the host should try to forward the + message to GAMMA directly, using: "RCPT TO:". + Since this host does not support relaying, it is not + required to update the reverse path. + + Some have suggested that source routing may be needed + occasionally for manually routing mail around failures; + however, the reality and importance of this need is + controversial. The use of explicit SMTP mail relaying for + this purpose is discouraged, and in fact it may not be + successful, as many host systems do not support it. Some + have used the "%-hack" (see Section 5.2.16) for this + purpose. + + 5.2.7 RCPT Command: RFC-821 Section 4.1.1 + + A host that supports a receiver-SMTP MUST support the reserved + mailbox "Postmaster". + + The receiver-SMTP MAY verify RCPT parameters as they arrive; + however, RCPT responses MUST NOT be delayed beyond a reasonable + time (see Section 5.3.2). + + Therefore, a "250 OK" response to a RCPT does not necessarily + + + +Internet Engineering Task Force [Page 52] + + + + +RFC1123 MAIL -- SMTP & RFC-822 October 1989 + + + imply that the delivery address(es) are valid. Errors found + after message acceptance will be reported by mailing a + notification message to an appropriate address (see Section + 5.3.3). + + DISCUSSION: + The set of conditions under which a RCPT parameter can be + validated immediately is an engineering design choice. + Reporting destination mailbox errors to the Sender-SMTP + before mail is transferred is generally desirable to save + time and network bandwidth, but this advantage is lost if + RCPT verification is lengthy. + + For example, the receiver can verify immediately any + simple local reference, such as a single locally- + registered mailbox. On the other hand, the "reasonable + time" limitation generally implies deferring verification + of a mailing list until after the message has been + transferred and accepted, since verifying a large mailing + list can take a very long time. An implementation might + or might not choose to defer validation of addresses that + are non-local and therefore require a DNS lookup. If a + DNS lookup is performed but a soft domain system error + (e.g., timeout) occurs, validity must be assumed. + + 5.2.8 DATA Command: RFC-821 Section 4.1.1 + + Every receiver-SMTP (not just one that "accepts a message for + relaying or for final delivery" [SMTP:1]) MUST insert a + "Received:" line at the beginning of a message. In this line, + called a "time stamp line" in RFC-821: + + * The FROM field SHOULD contain both (1) the name of the + source host as presented in the HELO command and (2) a + domain literal containing the IP address of the source, + determined from the TCP connection. + + * The ID field MAY contain an "@" as suggested in RFC-822, + but this is not required. + + * The FOR field MAY contain a list of entries when + multiple RCPT commands have been given. + + + An Internet mail program MUST NOT change a Received: line that + was previously added to the message header. + + + + + +Internet Engineering Task Force [Page 53] + + + + +RFC1123 MAIL -- SMTP & RFC-822 October 1989 + + + DISCUSSION: + Including both the source host and the IP source address + in the Received: line may provide enough information for + tracking illicit mail sources and eliminate a need to + explicitly verify the HELO parameter. + + Received: lines are primarily intended for humans tracing + mail routes, primarily of diagnosis of faults. See also + the discussion under 5.3.7. + + When the receiver-SMTP makes "final delivery" of a message, + then it MUST pass the MAIL FROM: address from the SMTP envelope + with the message, for use if an error notification message must + be sent later (see Section 5.3.3). There is an analogous + requirement when gatewaying from the Internet into a different + mail environment; see Section 5.3.7. + + DISCUSSION: + Note that the final reply to the DATA command depends only + upon the successful transfer and storage of the message. + Any problem with the destination address(es) must either + (1) have been reported in an SMTP error reply to the RCPT + command(s), or (2) be reported in a later error message + mailed to the originator. + + IMPLEMENTATION: + The MAIL FROM: information may be passed as a parameter or + in a Return-Path: line inserted at the beginning of the + message. + + 5.2.9 Command Syntax: RFC-821 Section 4.1.2 + + The syntax shown in RFC-821 for the MAIL FROM: command omits + the case of an empty path: "MAIL FROM: <>" (see RFC-821 Page + 15). An empty reverse path MUST be supported. + + 5.2.10 SMTP Replies: RFC-821 Section 4.2 + + A receiver-SMTP SHOULD send only the reply codes listed in + section 4.2.2 of RFC-821 or in this document. A receiver-SMTP + SHOULD use the text shown in examples in RFC-821 whenever + appropriate. + + A sender-SMTP MUST determine its actions only by the reply + code, not by the text (except for 251 and 551 replies); any + text, including no text at all, must be acceptable. The space + (blank) following the reply code is considered part of the + text. Whenever possible, a sender-SMTP SHOULD test only the + + + +Internet Engineering Task Force [Page 54] + + + + +RFC1123 MAIL -- SMTP & RFC-822 October 1989 + + + first digit of the reply code, as specified in Appendix E of + RFC-821. + + DISCUSSION: + Interoperability problems have arisen with SMTP systems + using reply codes that are not listed explicitly in RFC- + 821 Section 4.3 but are legal according to the theory of + reply codes explained in Appendix E. + + 5.2.11 Transparency: RFC-821 Section 4.5.2 + + Implementors MUST be sure that their mail systems always add + and delete periods to ensure message transparency. + + 5.2.12 WKS Use in MX Processing: RFC-974, p. 5 + + RFC-974 [SMTP:3] recommended that the domain system be queried + for WKS ("Well-Known Service") records, to verify that each + proposed mail target does support SMTP. Later experience has + shown that WKS is not widely supported, so the WKS step in MX + processing SHOULD NOT be used. + + The following are notes on RFC-822, organized by section of that + document. + + 5.2.13 RFC-822 Message Specification: RFC-822 Section 4 + + The syntax shown for the Return-path line omits the possibility + of a null return path, which is used to prevent looping of + error notifications (see Section 5.3.3). The complete syntax + is: + + return = "Return-path" ":" route-addr + / "Return-path" ":" "<" ">" + + The set of optional header fields is hereby expanded to include + the Content-Type field defined in RFC-1049 [SMTP:7]. This + field "allows mail reading systems to automatically identify + the type of a structured message body and to process it for + display accordingly". [SMTP:7] A User Agent MAY support this + field. + + 5.2.14 RFC-822 Date and Time Specification: RFC-822 Section 5 + + The syntax for the date is hereby changed to: + + date = 1*2DIGIT month 2*4DIGIT + + + + +Internet Engineering Task Force [Page 55] + + + + +RFC1123 MAIL -- SMTP & RFC-822 October 1989 + + + All mail software SHOULD use 4-digit years in dates, to ease + the transition to the next century. + + There is a strong trend towards the use of numeric timezone + indicators, and implementations SHOULD use numeric timezones + instead of timezone names. However, all implementations MUST + accept either notation. If timezone names are used, they MUST + be exactly as defined in RFC-822. + + The military time zones are specified incorrectly in RFC-822: + they count the wrong way from UT (the signs are reversed). As + a result, military time zones in RFC-822 headers carry no + information. + + Finally, note that there is a typo in the definition of "zone" + in the syntax summary of appendix D; the correct definition + occurs in Section 3 of RFC-822. + + 5.2.15 RFC-822 Syntax Change: RFC-822 Section 6.1 + + The syntactic definition of "mailbox" in RFC-822 is hereby + changed to: + + mailbox = addr-spec ; simple address + / [phrase] route-addr ; name & addr-spec + + That is, the phrase preceding a route address is now OPTIONAL. + This change makes the following header field legal, for + example: + + From: + + 5.2.16 RFC-822 Local-part: RFC-822 Section 6.2 + + The basic mailbox address specification has the form: "local- + part@domain". Here "local-part", sometimes called the "left- + hand side" of the address, is domain-dependent. + + A host that is forwarding the message but is not the + destination host implied by the right-hand side "domain" MUST + NOT interpret or modify the "local-part" of the address. + + When mail is to be gatewayed from the Internet mail environment + into a foreign mail environment (see Section 5.3.7), routing + information for that foreign environment MAY be embedded within + the "local-part" of the address. The gateway will then + interpret this local part appropriately for the foreign mail + environment. + + + +Internet Engineering Task Force [Page 56] + + + + +RFC1123 MAIL -- SMTP & RFC-822 October 1989 + + + DISCUSSION: + Although source routes are discouraged within the Internet + (see Section 5.2.6), there are non-Internet mail + environments whose delivery mechanisms do depend upon + source routes. Source routes for extra-Internet + environments can generally be buried in the "local-part" + of the address (see Section 5.2.16) while mail traverses + the Internet. When the mail reaches the appropriate + Internet mail gateway, the gateway will interpret the + local-part and build the necessary address or route for + the target mail environment. + + For example, an Internet host might send mail to: + "a!b!c!user@gateway-domain". The complex local part + "a!b!c!user" would be uninterpreted within the Internet + domain, but could be parsed and understood by the + specified mail gateway. + + An embedded source route is sometimes encoded in the + "local-part" using "%" as a right-binding routing + operator. For example, in: + + user%domain%relay3%relay2@relay1 + + the "%" convention implies that the mail is to be routed + from "relay1" through "relay2", "relay3", and finally to + "user" at "domain". This is commonly known as the "%- + hack". It is suggested that "%" have lower precedence + than any other routing operator (e.g., "!") hidden in the + local-part; for example, "a!b%c" would be interpreted as + "(a!b)%c". + + Only the target host (in this case, "relay1") is permitted + to analyze the local-part "user%domain%relay3%relay2". + + 5.2.17 Domain Literals: RFC-822 Section 6.2.3 + + A mailer MUST be able to accept and parse an Internet domain + literal whose content ("dtext"; see RFC-822) is a dotted- + decimal host address. This satisfies the requirement of + Section 2.1 for the case of mail. + + An SMTP MUST accept and recognize a domain literal for any of + its own IP addresses. + + + + + + + +Internet Engineering Task Force [Page 57] + + + + +RFC1123 MAIL -- SMTP & RFC-822 October 1989 + + + 5.2.18 Common Address Formatting Errors: RFC-822 Section 6.1 + + Errors in formatting or parsing 822 addresses are unfortunately + common. This section mentions only the most common errors. A + User Agent MUST accept all valid RFC-822 address formats, and + MUST NOT generate illegal address syntax. + + o A common error is to leave out the semicolon after a group + identifier. + + o Some systems fail to fully-qualify domain names in + messages they generate. The right-hand side of an "@" + sign in a header address field MUST be a fully-qualified + domain name. + + For example, some systems fail to fully-qualify the From: + address; this prevents a "reply" command in the user + interface from automatically constructing a return + address. + + DISCUSSION: + Although RFC-822 allows the local use of abbreviated + domain names within a domain, the application of + RFC-822 in Internet mail does not allow this. The + intent is that an Internet host must not send an SMTP + message header containing an abbreviated domain name + in an address field. This allows the address fields + of the header to be passed without alteration across + the Internet, as required in Section 5.2.6. + + o Some systems mis-parse multiple-hop explicit source routes + such as: + + @relay1,@relay2,@relay3:user@domain. + + + o Some systems over-qualify domain names by adding a + trailing dot to some or all domain names in addresses or + message-ids. This violates RFC-822 syntax. + + + 5.2.19 Explicit Source Routes: RFC-822 Section 6.2.7 + + Internet host software SHOULD NOT create an RFC-822 header + containing an address with an explicit source route, but MUST + accept such headers for compatibility with earlier systems. + + DISCUSSION: + + + +Internet Engineering Task Force [Page 58] + + + + +RFC1123 MAIL -- SMTP & RFC-822 October 1989 + + + In an understatement, RFC-822 says "The use of explicit + source routing is discouraged". Many hosts implemented + RFC-822 source routes incorrectly, so the syntax cannot be + used unambiguously in practice. Many users feel the + syntax is ugly. Explicit source routes are not needed in + the mail envelope for delivery; see Section 5.2.6. For + all these reasons, explicit source routes using the RFC- + 822 notations are not to be used in Internet mail headers. + + As stated in Section 5.2.16, it is necessary to allow an + explicit source route to be buried in the local-part of an + address, e.g., using the "%-hack", in order to allow mail + to be gatewayed into another environment in which explicit + source routing is necessary. The vigilant will observe + that there is no way for a User Agent to detect and + prevent the use of such implicit source routing when the + destination is within the Internet. We can only + discourage source routing of any kind within the Internet, + as unnecessary and undesirable. + + 5.3 SPECIFIC ISSUES + + 5.3.1 SMTP Queueing Strategies + + The common structure of a host SMTP implementation includes + user mailboxes, one or more areas for queueing messages in + transit, and one or more daemon processes for sending and + receiving mail. The exact structure will vary depending on the + needs of the users on the host and the number and size of + mailing lists supported by the host. We describe several + optimizations that have proved helpful, particularly for + mailers supporting high traffic levels. + + Any queueing strategy MUST include: + + o Timeouts on all activities. See Section 5.3.2. + + o Never sending error messages in response to error + messages. + + + 5.3.1.1 Sending Strategy + + The general model of a sender-SMTP is one or more processes + that periodically attempt to transmit outgoing mail. In a + typical system, the program that composes a message has some + method for requesting immediate attention for a new piece of + outgoing mail, while mail that cannot be transmitted + + + +Internet Engineering Task Force [Page 59] + + + + +RFC1123 MAIL -- SMTP & RFC-822 October 1989 + + + immediately MUST be queued and periodically retried by the + sender. A mail queue entry will include not only the + message itself but also the envelope information. + + The sender MUST delay retrying a particular destination + after one attempt has failed. In general, the retry + interval SHOULD be at least 30 minutes; however, more + sophisticated and variable strategies will be beneficial + when the sender-SMTP can determine the reason for non- + delivery. + + Retries continue until the message is transmitted or the + sender gives up; the give-up time generally needs to be at + least 4-5 days. The parameters to the retry algorithm MUST + be configurable. + + A sender SHOULD keep a list of hosts it cannot reach and + corresponding timeouts, rather than just retrying queued + mail items. + + DISCUSSION: + Experience suggests that failures are typically + transient (the target system has crashed), favoring a + policy of two connection attempts in the first hour the + message is in the queue, and then backing off to once + every two or three hours. + + The sender-SMTP can shorten the queueing delay by + cooperation with the receiver-SMTP. In particular, if + mail is received from a particular address, it is good + evidence that any mail queued for that host can now be + sent. + + The strategy may be further modified as a result of + multiple addresses per host (see Section 5.3.4), to + optimize delivery time vs. resource usage. + + A sender-SMTP may have a large queue of messages for + each unavailable destination host, and if it retried + all these messages in every retry cycle, there would be + excessive Internet overhead and the daemon would be + blocked for a long period. Note that an SMTP can + generally determine that a delivery attempt has failed + only after a timeout of a minute or more; a one minute + timeout per connection will result in a very large + delay if it is repeated for dozens or even hundreds of + queued messages. + + + + +Internet Engineering Task Force [Page 60] + + + + +RFC1123 MAIL -- SMTP & RFC-822 October 1989 + + + When the same message is to be delivered to several users on + the same host, only one copy of the message SHOULD be + transmitted. That is, the sender-SMTP should use the + command sequence: RCPT, RCPT,... RCPT, DATA instead of the + sequence: RCPT, DATA, RCPT, DATA,... RCPT, DATA. + Implementation of this efficiency feature is strongly urged. + + Similarly, the sender-SMTP MAY support multiple concurrent + outgoing mail transactions to achieve timely delivery. + However, some limit SHOULD be imposed to protect the host + from devoting all its resources to mail. + + The use of the different addresses of a multihomed host is + discussed below. + + 5.3.1.2 Receiving strategy + + The receiver-SMTP SHOULD attempt to keep a pending listen on + the SMTP port at all times. This will require the support + of multiple incoming TCP connections for SMTP. Some limit + MAY be imposed. + + IMPLEMENTATION: + When the receiver-SMTP receives mail from a particular + host address, it could notify the sender-SMTP to retry + any mail pending for that host address. + + 5.3.2 Timeouts in SMTP + + There are two approaches to timeouts in the sender-SMTP: (a) + limit the time for each SMTP command separately, or (b) limit + the time for the entire SMTP dialogue for a single mail + message. A sender-SMTP SHOULD use option (a), per-command + timeouts. Timeouts SHOULD be easily reconfigurable, preferably + without recompiling the SMTP code. + + DISCUSSION: + Timeouts are an essential feature of an SMTP + implementation. If the timeouts are too long (or worse, + there are no timeouts), Internet communication failures or + software bugs in receiver-SMTP programs can tie up SMTP + processes indefinitely. If the timeouts are too short, + resources will be wasted with attempts that time out part + way through message delivery. + + If option (b) is used, the timeout has to be very large, + e.g., an hour, to allow time to expand very large mailing + lists. The timeout may also need to increase linearly + + + +Internet Engineering Task Force [Page 61] + + + + +RFC1123 MAIL -- SMTP & RFC-822 October 1989 + + + with the size of the message, to account for the time to + transmit a very large message. A large fixed timeout + leads to two problems: a failure can still tie up the + sender for a very long time, and very large messages may + still spuriously time out (which is a wasteful failure!). + + Using the recommended option (a), a timer is set for each + SMTP command and for each buffer of the data transfer. + The latter means that the overall timeout is inherently + proportional to the size of the message. + + Based on extensive experience with busy mail-relay hosts, the + minimum per-command timeout values SHOULD be as follows: + + o Initial 220 Message: 5 minutes + + A Sender-SMTP process needs to distinguish between a + failed TCP connection and a delay in receiving the initial + 220 greeting message. Many receiver-SMTPs will accept a + TCP connection but delay delivery of the 220 message until + their system load will permit more mail to be processed. + + o MAIL Command: 5 minutes + + + o RCPT Command: 5 minutes + + A longer timeout would be required if processing of + mailing lists and aliases were not deferred until after + the message was accepted. + + o DATA Initiation: 2 minutes + + This is while awaiting the "354 Start Input" reply to a + DATA command. + + o Data Block: 3 minutes + + This is while awaiting the completion of each TCP SEND + call transmitting a chunk of data. + + o DATA Termination: 10 minutes. + + This is while awaiting the "250 OK" reply. When the + receiver gets the final period terminating the message + data, it typically performs processing to deliver the + message to a user mailbox. A spurious timeout at this + point would be very wasteful, since the message has been + + + +Internet Engineering Task Force [Page 62] + + + + +RFC1123 MAIL -- SMTP & RFC-822 October 1989 + + + successfully sent. + + A receiver-SMTP SHOULD have a timeout of at least 5 minutes + while it is awaiting the next command from the sender. + + 5.3.3 Reliable Mail Receipt + + When the receiver-SMTP accepts a piece of mail (by sending a + "250 OK" message in response to DATA), it is accepting + responsibility for delivering or relaying the message. It must + take this responsibility seriously, i.e., it MUST NOT lose the + message for frivolous reasons, e.g., because the host later + crashes or because of a predictable resource shortage. + + If there is a delivery failure after acceptance of a message, + the receiver-SMTP MUST formulate and mail a notification + message. This notification MUST be sent using a null ("<>") + reverse path in the envelope; see Section 3.6 of RFC-821. The + recipient of this notification SHOULD be the address from the + envelope return path (or the Return-Path: line). However, if + this address is null ("<>"), the receiver-SMTP MUST NOT send a + notification. If the address is an explicit source route, it + SHOULD be stripped down to its final hop. + + DISCUSSION: + For example, suppose that an error notification must be + sent for a message that arrived with: + "MAIL FROM:<@a,@b:user@d>". The notification message + should be sent to: "RCPT TO:". + + Some delivery failures after the message is accepted by + SMTP will be unavoidable. For example, it may be + impossible for the receiver-SMTP to validate all the + delivery addresses in RCPT command(s) due to a "soft" + domain system error or because the target is a mailing + list (see earlier discussion of RCPT). + + To avoid receiving duplicate messages as the result of + timeouts, a receiver-SMTP MUST seek to minimize the time + required to respond to the final "." that ends a message + transfer. See RFC-1047 [SMTP:4] for a discussion of this + problem. + + 5.3.4 Reliable Mail Transmission + + To transmit a message, a sender-SMTP determines the IP address + of the target host from the destination address in the + envelope. Specifically, it maps the string to the right of the + + + +Internet Engineering Task Force [Page 63] + + + + +RFC1123 MAIL -- SMTP & RFC-822 October 1989 + + + "@" sign into an IP address. This mapping or the transfer + itself may fail with a soft error, in which case the sender- + SMTP will requeue the outgoing mail for a later retry, as + required in Section 5.3.1.1. + + When it succeeds, the mapping can result in a list of + alternative delivery addresses rather than a single address, + because of (a) multiple MX records, (b) multihoming, or both. + To provide reliable mail transmission, the sender-SMTP MUST be + able to try (and retry) each of the addresses in this list in + order, until a delivery attempt succeeds. However, there MAY + also be a configurable limit on the number of alternate + addresses that can be tried. In any case, a host SHOULD try at + least two addresses. + + The following information is to be used to rank the host + addresses: + + (1) Multiple MX Records -- these contain a preference + indication that should be used in sorting. If there are + multiple destinations with the same preference and there + is no clear reason to favor one (e.g., by address + preference), then the sender-SMTP SHOULD pick one at + random to spread the load across multiple mail exchanges + for a specific organization; note that this is a + refinement of the procedure in [DNS:3]. + + (2) Multihomed host -- The destination host (perhaps taken + from the preferred MX record) may be multihomed, in which + case the domain name resolver will return a list of + alternative IP addresses. It is the responsibility of the + domain name resolver interface (see Section 6.1.3.4 below) + to have ordered this list by decreasing preference, and + SMTP MUST try them in the order presented. + + DISCUSSION: + Although the capability to try multiple alternative + addresses is required, there may be circumstances where + specific installations want to limit or disable the use of + alternative addresses. The question of whether a sender + should attempt retries using the different addresses of a + multihomed host has been controversial. The main argument + for using the multiple addresses is that it maximizes the + probability of timely delivery, and indeed sometimes the + probability of any delivery; the counter argument is that + it may result in unnecessary resource use. + + Note that resource use is also strongly determined by the + + + +Internet Engineering Task Force [Page 64] + + + + +RFC1123 MAIL -- SMTP & RFC-822 October 1989 + + + sending strategy discussed in Section 5.3.1. + + 5.3.5 Domain Name Support + + SMTP implementations MUST use the mechanism defined in Section + 6.1 for mapping between domain names and IP addresses. This + means that every Internet SMTP MUST include support for the + Internet DNS. + + In particular, a sender-SMTP MUST support the MX record scheme + [SMTP:3]. See also Section 7.4 of [DNS:2] for information on + domain name support for SMTP. + + 5.3.6 Mailing Lists and Aliases + + An SMTP-capable host SHOULD support both the alias and the list + form of address expansion for multiple delivery. When a + message is delivered or forwarded to each address of an + expanded list form, the return address in the envelope + ("MAIL FROM:") MUST be changed to be the address of a person + who administers the list, but the message header MUST be left + unchanged; in particular, the "From" field of the message is + unaffected. + + DISCUSSION: + An important mail facility is a mechanism for multi- + destination delivery of a single message, by transforming + or "expanding" a pseudo-mailbox address into a list of + destination mailbox addresses. When a message is sent to + such a pseudo-mailbox (sometimes called an "exploder"), + copies are forwarded or redistributed to each mailbox in + the expanded list. We classify such a pseudo-mailbox as + an "alias" or a "list", depending upon the expansion + rules: + + (a) Alias + + To expand an alias, the recipient mailer simply + replaces the pseudo-mailbox address in the envelope + with each of the expanded addresses in turn; the rest + of the envelope and the message body are left + unchanged. The message is then delivered or + forwarded to each expanded address. + + (b) List + + A mailing list may be said to operate by + "redistribution" rather than by "forwarding". To + + + +Internet Engineering Task Force [Page 65] + + + + +RFC1123 MAIL -- SMTP & RFC-822 October 1989 + + + expand a list, the recipient mailer replaces the + pseudo-mailbox address in the envelope with each of + the expanded addresses in turn. The return address in + the envelope is changed so that all error messages + generated by the final deliveries will be returned to + a list administrator, not to the message originator, + who generally has no control over the contents of the + list and will typically find error messages annoying. + + + 5.3.7 Mail Gatewaying + + Gatewaying mail between different mail environments, i.e., + different mail formats and protocols, is complex and does not + easily yield to standardization. See for example [SMTP:5a], + [SMTP:5b]. However, some general requirements may be given for + a gateway between the Internet and another mail environment. + + (A) Header fields MAY be rewritten when necessary as messages + are gatewayed across mail environment boundaries. + + DISCUSSION: + This may involve interpreting the local-part of the + destination address, as suggested in Section 5.2.16. + + The other mail systems gatewayed to the Internet + generally use a subset of RFC-822 headers, but some + of them do not have an equivalent to the SMTP + envelope. Therefore, when a message leaves the + Internet environment, it may be necessary to fold the + SMTP envelope information into the message header. A + possible solution would be to create new header + fields to carry the envelope information (e.g., "X- + SMTP-MAIL:" and "X-SMTP-RCPT:"); however, this would + require changes in mail programs in the foreign + environment. + + (B) When forwarding a message into or out of the Internet + environment, a gateway MUST prepend a Received: line, but + it MUST NOT alter in any way a Received: line that is + already in the header. + + DISCUSSION: + This requirement is a subset of the general + "Received:" line requirement of Section 5.2.8; it is + restated here for emphasis. + + Received: fields of messages originating from other + + + +Internet Engineering Task Force [Page 66] + + + + +RFC1123 MAIL -- SMTP & RFC-822 October 1989 + + + environments may not conform exactly to RFC822. + However, the most important use of Received: lines is + for debugging mail faults, and this debugging can be + severely hampered by well-meaning gateways that try + to "fix" a Received: line. + + The gateway is strongly encouraged to indicate the + environment and protocol in the "via" clauses of + Received field(s) that it supplies. + + (C) From the Internet side, the gateway SHOULD accept all + valid address formats in SMTP commands and in RFC-822 + headers, and all valid RFC-822 messages. Although a + gateway must accept an RFC-822 explicit source route + ("@...:" format) in either the RFC-822 header or in the + envelope, it MAY or may not act on the source route; see + Sections 5.2.6 and 5.2.19. + + DISCUSSION: + It is often tempting to restrict the range of + addresses accepted at the mail gateway to simplify + the translation into addresses for the remote + environment. This practice is based on the + assumption that mail users have control over the + addresses their mailers send to the mail gateway. In + practice, however, users have little control over the + addresses that are finally sent; their mailers are + free to change addresses into any legal RFC-822 + format. + + (D) The gateway MUST ensure that all header fields of a + message that it forwards into the Internet meet the + requirements for Internet mail. In particular, all + addresses in "From:", "To:", "Cc:", etc., fields must be + transformed (if necessary) to satisfy RFC-822 syntax, and + they must be effective and useful for sending replies. + + + (E) The translation algorithm used to convert mail from the + Internet protocols to another environment's protocol + SHOULD try to ensure that error messages from the foreign + mail environment are delivered to the return path from the + SMTP envelope, not to the sender listed in the "From:" + field of the RFC-822 message. + + DISCUSSION: + Internet mail lists usually place the address of the + mail list maintainer in the envelope but leave the + + + +Internet Engineering Task Force [Page 67] + + + + +RFC1123 MAIL -- SMTP & RFC-822 October 1989 + + + original message header intact (with the "From:" + field containing the original sender). This yields + the behavior the average recipient expects: a reply + to the header gets sent to the original sender, not + to a mail list maintainer; however, errors get sent + to the maintainer (who can fix the problem) and not + the sender (who probably cannot). + + (F) Similarly, when forwarding a message from another + environment into the Internet, the gateway SHOULD set the + envelope return path in accordance with an error message + return address, if any, supplied by the foreign + environment. + + + 5.3.8 Maximum Message Size + + Mailer software MUST be able to send and receive messages of at + least 64K bytes in length (including header), and a much larger + maximum size is highly desirable. + + DISCUSSION: + Although SMTP does not define the maximum size of a + message, many systems impose implementation limits. + + The current de facto minimum limit in the Internet is 64K + bytes. However, electronic mail is used for a variety of + purposes that create much larger messages. For example, + mail is often used instead of FTP for transmitting ASCII + files, and in particular to transmit entire documents. As + a result, messages can be 1 megabyte or even larger. We + note that the present document together with its lower- + layer companion contains 0.5 megabytes. + + + + + + + + + + + + + + + + + + +Internet Engineering Task Force [Page 68] + + + + +RFC1123 MAIL -- SMTP & RFC-822 October 1989 + + + 5.4 SMTP REQUIREMENTS SUMMARY + + | | | | |S| | + | | | | |H| |F + | | | | |O|M|o + | | |S| |U|U|o + | | |H| |L|S|t + | |M|O| |D|T|n + | |U|U|M| | |o + | |S|L|A|N|N|t + | |T|D|Y|O|O|t +FEATURE |SECTION | | | |T|T|e +-----------------------------------------------|----------|-|-|-|-|-|-- + | | | | | | | +RECEIVER-SMTP: | | | | | | | + Implement VRFY |5.2.3 |x| | | | | + Implement EXPN |5.2.3 | |x| | | | + EXPN, VRFY configurable |5.2.3 | | |x| | | + Implement SEND, SOML, SAML |5.2.4 | | |x| | | + Verify HELO parameter |5.2.5 | | |x| | | + Refuse message with bad HELO |5.2.5 | | | | |x| + Accept explicit src-route syntax in env. |5.2.6 |x| | | | | + Support "postmaster" |5.2.7 |x| | | | | + Process RCPT when received (except lists) |5.2.7 | | |x| | | + Long delay of RCPT responses |5.2.7 | | | | |x| + | | | | | | | + Add Received: line |5.2.8 |x| | | | | + Received: line include domain literal |5.2.8 | |x| | | | + Change previous Received: line |5.2.8 | | | | |x| + Pass Return-Path info (final deliv/gwy) |5.2.8 |x| | | | | + Support empty reverse path |5.2.9 |x| | | | | + Send only official reply codes |5.2.10 | |x| | | | + Send text from RFC-821 when appropriate |5.2.10 | |x| | | | + Delete "." for transparency |5.2.11 |x| | | | | + Accept and recognize self domain literal(s) |5.2.17 |x| | | | | + | | | | | | | + Error message about error message |5.3.1 | | | | |x| + Keep pending listen on SMTP port |5.3.1.2 | |x| | | | + Provide limit on recv concurrency |5.3.1.2 | | |x| | | + Wait at least 5 mins for next sender cmd |5.3.2 | |x| | | | + Avoidable delivery failure after "250 OK" |5.3.3 | | | | |x| + Send error notification msg after accept |5.3.3 |x| | | | | + Send using null return path |5.3.3 |x| | | | | + Send to envelope return path |5.3.3 | |x| | | | + Send to null address |5.3.3 | | | | |x| + Strip off explicit src route |5.3.3 | |x| | | | + Minimize acceptance delay (RFC-1047) |5.3.3 |x| | | | | +-----------------------------------------------|----------|-|-|-|-|-|-- + + + +Internet Engineering Task Force [Page 69] + + + + +RFC1123 MAIL -- SMTP & RFC-822 October 1989 + + + | | | | | | | +SENDER-SMTP: | | | | | | | + Canonicalized domain names in MAIL, RCPT |5.2.2 |x| | | | | + Implement SEND, SOML, SAML |5.2.4 | | |x| | | + Send valid principal host name in HELO |5.2.5 |x| | | | | + Send explicit source route in RCPT TO: |5.2.6 | | | |x| | + Use only reply code to determine action |5.2.10 |x| | | | | + Use only high digit of reply code when poss. |5.2.10 | |x| | | | + Add "." for transparency |5.2.11 |x| | | | | + | | | | | | | + Retry messages after soft failure |5.3.1.1 |x| | | | | + Delay before retry |5.3.1.1 |x| | | | | + Configurable retry parameters |5.3.1.1 |x| | | | | + Retry once per each queued dest host |5.3.1.1 | |x| | | | + Multiple RCPT's for same DATA |5.3.1.1 | |x| | | | + Support multiple concurrent transactions |5.3.1.1 | | |x| | | + Provide limit on concurrency |5.3.1.1 | |x| | | | + | | | | | | | + Timeouts on all activities |5.3.1 |x| | | | | + Per-command timeouts |5.3.2 | |x| | | | + Timeouts easily reconfigurable |5.3.2 | |x| | | | + Recommended times |5.3.2 | |x| | | | + Try alternate addr's in order |5.3.4 |x| | | | | + Configurable limit on alternate tries |5.3.4 | | |x| | | + Try at least two alternates |5.3.4 | |x| | | | + Load-split across equal MX alternates |5.3.4 | |x| | | | + Use the Domain Name System |5.3.5 |x| | | | | + Support MX records |5.3.5 |x| | | | | + Use WKS records in MX processing |5.2.12 | | | |x| | +-----------------------------------------------|----------|-|-|-|-|-|-- + | | | | | | | +MAIL FORWARDING: | | | | | | | + Alter existing header field(s) |5.2.6 | | | |x| | + Implement relay function: 821/section 3.6 |5.2.6 | | |x| | | + If not, deliver to RHS domain |5.2.6 | |x| | | | + Interpret 'local-part' of addr |5.2.16 | | | | |x| + | | | | | | | +MAILING LISTS AND ALIASES | | | | | | | + Support both |5.3.6 | |x| | | | + Report mail list error to local admin. |5.3.6 |x| | | | | + | | | | | | | +MAIL GATEWAYS: | | | | | | | + Embed foreign mail route in local-part |5.2.16 | | |x| | | + Rewrite header fields when necessary |5.3.7 | | |x| | | + Prepend Received: line |5.3.7 |x| | | | | + Change existing Received: line |5.3.7 | | | | |x| + Accept full RFC-822 on Internet side |5.3.7 | |x| | | | + Act on RFC-822 explicit source route |5.3.7 | | |x| | | + + + +Internet Engineering Task Force [Page 70] + + + + +RFC1123 MAIL -- SMTP & RFC-822 October 1989 + + + Send only valid RFC-822 on Internet side |5.3.7 |x| | | | | + Deliver error msgs to envelope addr |5.3.7 | |x| | | | + Set env return path from err return addr |5.3.7 | |x| | | | + | | | | | | | +USER AGENT -- RFC-822 | | | | | | | + Allow user to enter address |5.2.6 | | | |x| | + Support RFC-1049 Content Type field |5.2.13 | | |x| | | + Use 4-digit years |5.2.14 | |x| | | | + Generate numeric timezones |5.2.14 | |x| | | | + Accept all timezones |5.2.14 |x| | | | | + Use non-num timezones from RFC-822 |5.2.14 |x| | | | | + Omit phrase before route-addr |5.2.15 | | |x| | | + Accept and parse dot.dec. domain literals |5.2.17 |x| | | | | + Accept all RFC-822 address formats |5.2.18 |x| | | | | + Generate invalid RFC-822 address format |5.2.18 | | | | |x| + Fully-qualified domain names in header |5.2.18 |x| | | | | + Create explicit src route in header |5.2.19 | | | |x| | + Accept explicit src route in header |5.2.19 |x| | | | | + | | | | | | | +Send/recv at least 64KB messages |5.3.8 |x| | | | | + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +Internet Engineering Task Force [Page 71] + + + + +RFC1123 SUPPORT SERVICES -- DOMAINS October 1989 + + +6. SUPPORT SERVICES + + 6.1 DOMAIN NAME TRANSLATION + + 6.1.1 INTRODUCTION + + Every host MUST implement a resolver for the Domain Name System + (DNS), and it MUST implement a mechanism using this DNS + resolver to convert host names to IP addresses and vice-versa + [DNS:1, DNS:2]. + + In addition to the DNS, a host MAY also implement a host name + translation mechanism that searches a local Internet host + table. See Section 6.1.3.8 for more information on this + option. + + DISCUSSION: + Internet host name translation was originally performed by + searching local copies of a table of all hosts. This + table became too large to update and distribute in a + timely manner and too large to fit into many hosts, so the + DNS was invented. + + The DNS creates a distributed database used primarily for + the translation between host names and host addresses. + Implementation of DNS software is required. The DNS + consists of two logically distinct parts: name servers and + resolvers (although implementations often combine these + two logical parts in the interest of efficiency) [DNS:2]. + + Domain name servers store authoritative data about certain + sections of the database and answer queries about the + data. Domain resolvers query domain name servers for data + on behalf of user processes. Every host therefore needs a + DNS resolver; some host machines will also need to run + domain name servers. Since no name server has complete + information, in general it is necessary to obtain + information from more than one name server to resolve a + query. + + 6.1.2 PROTOCOL WALK-THROUGH + + An implementor must study references [DNS:1] and [DNS:2] + carefully. They provide a thorough description of the theory, + protocol, and implementation of the domain name system, and + reflect several years of experience. + + + + + +Internet Engineering Task Force [Page 72] + + + + +RFC1123 SUPPORT SERVICES -- DOMAINS October 1989 + + + 6.1.2.1 Resource Records with Zero TTL: RFC-1035 Section 3.2.1 + + All DNS name servers and resolvers MUST properly handle RRs + with a zero TTL: return the RR to the client but do not + cache it. + + DISCUSSION: + Zero TTL values are interpreted to mean that the RR can + only be used for the transaction in progress, and + should not be cached; they are useful for extremely + volatile data. + + 6.1.2.2 QCLASS Values: RFC-1035 Section 3.2.5 + + A query with "QCLASS=*" SHOULD NOT be used unless the + requestor is seeking data from more than one class. In + particular, if the requestor is only interested in Internet + data types, QCLASS=IN MUST be used. + + 6.1.2.3 Unused Fields: RFC-1035 Section 4.1.1 + + Unused fields in a query or response message MUST be zero. + + 6.1.2.4 Compression: RFC-1035 Section 4.1.4 + + Name servers MUST use compression in responses. + + DISCUSSION: + Compression is essential to avoid overflowing UDP + datagrams; see Section 6.1.3.2. + + 6.1.2.5 Misusing Configuration Info: RFC-1035 Section 6.1.2 + + Recursive name servers and full-service resolvers generally + have some configuration information containing hints about + the location of root or local name servers. An + implementation MUST NOT include any of these hints in a + response. + + DISCUSSION: + Many implementors have found it convenient to store + these hints as if they were cached data, but some + neglected to ensure that this "cached data" was not + included in responses. This has caused serious + problems in the Internet when the hints were obsolete + or incorrect. + + + + + +Internet Engineering Task Force [Page 73] + + + + +RFC1123 SUPPORT SERVICES -- DOMAINS October 1989 + + + 6.1.3 SPECIFIC ISSUES + + 6.1.3.1 Resolver Implementation + + A name resolver SHOULD be able to multiplex concurrent + requests if the host supports concurrent processes. + + In implementing a DNS resolver, one of two different models + MAY optionally be chosen: a full-service resolver, or a stub + resolver. + + + (A) Full-Service Resolver + + A full-service resolver is a complete implementation of + the resolver service, and is capable of dealing with + communication failures, failure of individual name + servers, location of the proper name server for a given + name, etc. It must satisfy the following requirements: + + o The resolver MUST implement a local caching + function to avoid repeated remote access for + identical requests, and MUST time out information + in the cache. + + o The resolver SHOULD be configurable with start-up + information pointing to multiple root name servers + and multiple name servers for the local domain. + This insures that the resolver will be able to + access the whole name space in normal cases, and + will be able to access local domain information + should the local network become disconnected from + the rest of the Internet. + + + (B) Stub Resolver + + A "stub resolver" relies on the services of a recursive + name server on the connected network or a "nearby" + network. This scheme allows the host to pass on the + burden of the resolver function to a name server on + another host. This model is often essential for less + capable hosts, such as PCs, and is also recommended + when the host is one of several workstations on a local + network, because it allows all of the workstations to + share the cache of the recursive name server and hence + reduce the number of domain requests exported by the + local network. + + + +Internet Engineering Task Force [Page 74] + + + + +RFC1123 SUPPORT SERVICES -- DOMAINS October 1989 + + + At a minimum, the stub resolver MUST be capable of + directing its requests to redundant recursive name + servers. Note that recursive name servers are allowed + to restrict the sources of requests that they will + honor, so the host administrator must verify that the + service will be provided. Stub resolvers MAY implement + caching if they choose, but if so, MUST timeout cached + information. + + + 6.1.3.2 Transport Protocols + + DNS resolvers and recursive servers MUST support UDP, and + SHOULD support TCP, for sending (non-zone-transfer) queries. + Specifically, a DNS resolver or server that is sending a + non-zone-transfer query MUST send a UDP query first. If the + Answer section of the response is truncated and if the + requester supports TCP, it SHOULD try the query again using + TCP. + + DNS servers MUST be able to service UDP queries and SHOULD + be able to service TCP queries. A name server MAY limit the + resources it devotes to TCP queries, but it SHOULD NOT + refuse to service a TCP query just because it would have + succeeded with UDP. + + Truncated responses MUST NOT be saved (cached) and later + used in such a way that the fact that they are truncated is + lost. + + DISCUSSION: + UDP is preferred over TCP for queries because UDP + queries have much lower overhead, both in packet count + and in connection state. The use of UDP is essential + for heavily-loaded servers, especially the root + servers. UDP also offers additional robustness, since + a resolver can attempt several UDP queries to different + servers for the cost of a single TCP query. + + It is possible for a DNS response to be truncated, + although this is a very rare occurrence in the present + Internet DNS. Practically speaking, truncation cannot + be predicted, since it is data-dependent. The + dependencies include the number of RRs in the answer, + the size of each RR, and the savings in space realized + by the name compression algorithm. As a rule of thumb, + truncation in NS and MX lists should not occur for + answers containing 15 or fewer RRs. + + + +Internet Engineering Task Force [Page 75] + + + + +RFC1123 SUPPORT SERVICES -- DOMAINS October 1989 + + + Whether it is possible to use a truncated answer + depends on the application. A mailer must not use a + truncated MX response, since this could lead to mail + loops. + + Responsible practices can make UDP suffice in the vast + majority of cases. Name servers must use compression + in responses. Resolvers must differentiate truncation + of the Additional section of a response (which only + loses extra information) from truncation of the Answer + section (which for MX records renders the response + unusable by mailers). Database administrators should + list only a reasonable number of primary names in lists + of name servers, MX alternatives, etc. + + However, it is also clear that some new DNS record + types defined in the future will contain information + exceeding the 512 byte limit that applies to UDP, and + hence will require TCP. Thus, resolvers and name + servers should implement TCP services as a backup to + UDP today, with the knowledge that they will require + the TCP service in the future. + + By private agreement, name servers and resolvers MAY arrange + to use TCP for all traffic between themselves. TCP MUST be + used for zone transfers. + + A DNS server MUST have sufficient internal concurrency that + it can continue to process UDP queries while awaiting a + response or performing a zone transfer on an open TCP + connection [DNS:2]. + + A server MAY support a UDP query that is delivered using an + IP broadcast or multicast address. However, the Recursion + Desired bit MUST NOT be set in a query that is multicast, + and MUST be ignored by name servers receiving queries via a + broadcast or multicast address. A host that sends broadcast + or multicast DNS queries SHOULD send them only as occasional + probes, caching the IP address(es) it obtains from the + response(s) so it can normally send unicast queries. + + DISCUSSION: + Broadcast or (especially) IP multicast can provide a + way to locate nearby name servers without knowing their + IP addresses in advance. However, general broadcasting + of recursive queries can result in excessive and + unnecessary load on both network and servers. + + + + +Internet Engineering Task Force [Page 76] + + + + +RFC1123 SUPPORT SERVICES -- DOMAINS October 1989 + + + 6.1.3.3 Efficient Resource Usage + + The following requirements on servers and resolvers are very + important to the health of the Internet as a whole, + particularly when DNS services are invoked repeatedly by + higher level automatic servers, such as mailers. + + (1) The resolver MUST implement retransmission controls to + insure that it does not waste communication bandwidth, + and MUST impose finite bounds on the resources consumed + to respond to a single request. See [DNS:2] pages 43- + 44 for specific recommendations. + + (2) After a query has been retransmitted several times + without a response, an implementation MUST give up and + return a soft error to the application. + + (3) All DNS name servers and resolvers SHOULD cache + temporary failures, with a timeout period of the order + of minutes. + + DISCUSSION: + This will prevent applications that immediately + retry soft failures (in violation of Section 2.2 + of this document) from generating excessive DNS + traffic. + + (4) All DNS name servers and resolvers SHOULD cache + negative responses that indicate the specified name, or + data of the specified type, does not exist, as + described in [DNS:2]. + + (5) When a DNS server or resolver retries a UDP query, the + retry interval SHOULD be constrained by an exponential + backoff algorithm, and SHOULD also have upper and lower + bounds. + + IMPLEMENTATION: + A measured RTT and variance (if available) should + be used to calculate an initial retransmission + interval. If this information is not available, a + default of no less than 5 seconds should be used. + Implementations may limit the retransmission + interval, but this limit must exceed twice the + Internet maximum segment lifetime plus service + delay at the name server. + + (6) When a resolver or server receives a Source Quench for + + + +Internet Engineering Task Force [Page 77] + + + + +RFC1123 SUPPORT SERVICES -- DOMAINS October 1989 + + + a query it has issued, it SHOULD take steps to reduce + the rate of querying that server in the near future. A + server MAY ignore a Source Quench that it receives as + the result of sending a response datagram. + + IMPLEMENTATION: + One recommended action to reduce the rate is to + send the next query attempt to an alternate + server, if there is one available. Another is to + backoff the retry interval for the same server. + + + 6.1.3.4 Multihomed Hosts + + When the host name-to-address function encounters a host + with multiple addresses, it SHOULD rank or sort the + addresses using knowledge of the immediately connected + network number(s) and any other applicable performance or + history information. + + DISCUSSION: + The different addresses of a multihomed host generally + imply different Internet paths, and some paths may be + preferable to others in performance, reliability, or + administrative restrictions. There is no general way + for the domain system to determine the best path. A + recommended approach is to base this decision on local + configuration information set by the system + administrator. + + IMPLEMENTATION: + The following scheme has been used successfully: + + (a) Incorporate into the host configuration data a + Network-Preference List, that is simply a list of + networks in preferred order. This list may be + empty if there is no preference. + + (b) When a host name is mapped into a list of IP + addresses, these addresses should be sorted by + network number, into the same order as the + corresponding networks in the Network-Preference + List. IP addresses whose networks do not appear + in the Network-Preference List should be placed at + the end of the list. + + + + + + +Internet Engineering Task Force [Page 78] + + + + +RFC1123 SUPPORT SERVICES -- DOMAINS October 1989 + + + 6.1.3.5 Extensibility + + DNS software MUST support all well-known, class-independent + formats [DNS:2], and SHOULD be written to minimize the + trauma associated with the introduction of new well-known + types and local experimentation with non-standard types. + + DISCUSSION: + The data types and classes used by the DNS are + extensible, and thus new types will be added and old + types deleted or redefined. Introduction of new data + types ought to be dependent only upon the rules for + compression of domain names inside DNS messages, and + the translation between printable (i.e., master file) + and internal formats for Resource Records (RRs). + + Compression relies on knowledge of the format of data + inside a particular RR. Hence compression must only be + used for the contents of well-known, class-independent + RRs, and must never be used for class-specific RRs or + RR types that are not well-known. The owner name of an + RR is always eligible for compression. + + A name server may acquire, via zone transfer, RRs that + the server doesn't know how to convert to printable + format. A resolver can receive similar information as + the result of queries. For proper operation, this data + must be preserved, and hence the implication is that + DNS software cannot use textual formats for internal + storage. + + The DNS defines domain name syntax very generally -- a + string of labels each containing up to 63 8-bit octets, + separated by dots, and with a maximum total of 255 + octets. Particular applications of the DNS are + permitted to further constrain the syntax of the domain + names they use, although the DNS deployment has led to + some applications allowing more general names. In + particular, Section 2.1 of this document liberalizes + slightly the syntax of a legal Internet host name that + was defined in RFC-952 [DNS:4]. + + 6.1.3.6 Status of RR Types + + Name servers MUST be able to load all RR types except MD and + MF from configuration files. The MD and MF types are + obsolete and MUST NOT be implemented; in particular, name + servers MUST NOT load these types from configuration files. + + + +Internet Engineering Task Force [Page 79] + + + + +RFC1123 SUPPORT SERVICES -- DOMAINS October 1989 + + + DISCUSSION: + The RR types MB, MG, MR, NULL, MINFO and RP are + considered experimental, and applications that use the + DNS cannot expect these RR types to be supported by + most domains. Furthermore these types are subject to + redefinition. + + The TXT and WKS RR types have not been widely used by + Internet sites; as a result, an application cannot rely + on the the existence of a TXT or WKS RR in most + domains. + + 6.1.3.7 Robustness + + DNS software may need to operate in environments where the + root servers or other servers are unavailable due to network + connectivity or other problems. In this situation, DNS name + servers and resolvers MUST continue to provide service for + the reachable part of the name space, while giving temporary + failures for the rest. + + DISCUSSION: + Although the DNS is meant to be used primarily in the + connected Internet, it should be possible to use the + system in networks which are unconnected to the + Internet. Hence implementations must not depend on + access to root servers before providing service for + local names. + + 6.1.3.8 Local Host Table + + DISCUSSION: + A host may use a local host table as a backup or + supplement to the DNS. This raises the question of + which takes precedence, the DNS or the host table; the + most flexible approach would make this a configuration + option. + + Typically, the contents of such a supplementary host + table will be determined locally by the site. However, + a publically-available table of Internet hosts is + maintained by the DDN Network Information Center (DDN + NIC), with a format documented in [DNS:4]. This table + can be retrieved from the DDN NIC using a protocol + described in [DNS:5]. It must be noted that this table + contains only a small fraction of all Internet hosts. + Hosts using this protocol to retrieve the DDN NIC host + table should use the VERSION command to check if the + + + +Internet Engineering Task Force [Page 80] + + + + +RFC1123 SUPPORT SERVICES -- DOMAINS October 1989 + + + table has changed before requesting the entire table + with the ALL command. The VERSION identifier should be + treated as an arbitrary string and tested only for + equality; no numerical sequence may be assumed. + + The DDN NIC host table includes administrative + information that is not needed for host operation and + is therefore not currently included in the DNS + database; examples include network and gateway entries. + However, much of this additional information will be + added to the DNS in the future. Conversely, the DNS + provides essential services (in particular, MX records) + that are not available from the DDN NIC host table. + + 6.1.4 DNS USER INTERFACE + + 6.1.4.1 DNS Administration + + This document is concerned with design and implementation + issues in host software, not with administrative or + operational issues. However, administrative issues are of + particular importance in the DNS, since errors in particular + segments of this large distributed database can cause poor + or erroneous performance for many sites. These issues are + discussed in [DNS:6] and [DNS:7]. + + 6.1.4.2 DNS User Interface + + Hosts MUST provide an interface to the DNS for all + application programs running on the host. This interface + will typically direct requests to a system process to + perform the resolver function [DNS:1, 6.1:2]. + + At a minimum, the basic interface MUST support a request for + all information of a specific type and class associated with + a specific name, and it MUST return either all of the + requested information, a hard error code, or a soft error + indication. When there is no error, the basic interface + returns the complete response information without + modification, deletion, or ordering, so that the basic + interface will not need to be changed to accommodate new + data types. + + DISCUSSION: + The soft error indication is an essential part of the + interface, since it may not always be possible to + access particular information from the DNS; see Section + 6.1.3.3. + + + +Internet Engineering Task Force [Page 81] + + + + +RFC1123 SUPPORT SERVICES -- DOMAINS October 1989 + + + A host MAY provide other DNS interfaces tailored to + particular functions, transforming the raw domain data into + formats more suited to these functions. In particular, a + host MUST provide a DNS interface to facilitate translation + between host addresses and host names. + + 6.1.4.3 Interface Abbreviation Facilities + + User interfaces MAY provide a method for users to enter + abbreviations for commonly-used names. Although the + definition of such methods is outside of the scope of the + DNS specification, certain rules are necessary to insure + that these methods allow access to the entire DNS name space + and to prevent excessive use of Internet resources. + + If an abbreviation method is provided, then: + + (a) There MUST be some convention for denoting that a name + is already complete, so that the abbreviation method(s) + are suppressed. A trailing dot is the usual method. + + (b) Abbreviation expansion MUST be done exactly once, and + MUST be done in the context in which the name was + entered. + + + DISCUSSION: + For example, if an abbreviation is used in a mail + program for a destination, the abbreviation should be + expanded into a full domain name and stored in the + queued message with an indication that it is already + complete. Otherwise, the abbreviation might be + expanded with a mail system search list, not the + user's, or a name could grow due to repeated + canonicalizations attempts interacting with wildcards. + + The two most common abbreviation methods are: + + (1) Interface-level aliases + + Interface-level aliases are conceptually implemented as + a list of alias/domain name pairs. The list can be + per-user or per-host, and separate lists can be + associated with different functions, e.g. one list for + host name-to-address translation, and a different list + for mail domains. When the user enters a name, the + interface attempts to match the name to the alias + component of a list entry, and if a matching entry can + + + +Internet Engineering Task Force [Page 82] + + + + +RFC1123 SUPPORT SERVICES -- DOMAINS October 1989 + + + be found, the name is replaced by the domain name found + in the pair. + + Note that interface-level aliases and CNAMEs are + completely separate mechanisms; interface-level aliases + are a local matter while CNAMEs are an Internet-wide + aliasing mechanism which is a required part of any DNS + implementation. + + (2) Search Lists + + A search list is conceptually implemented as an ordered + list of domain names. When the user enters a name, the + domain names in the search list are used as suffixes to + the user-supplied name, one by one, until a domain name + with the desired associated data is found, or the + search list is exhausted. Search lists often contain + the name of the local host's parent domain or other + ancestor domains. Search lists are often per-user or + per-process. + + It SHOULD be possible for an administrator to disable a + DNS search-list facility. Administrative denial may be + warranted in some cases, to prevent abuse of the DNS. + + There is danger that a search-list mechanism will + generate excessive queries to the root servers while + testing whether user input is a complete domain name, + lacking a final period to mark it as complete. A + search-list mechanism MUST have one of, and SHOULD have + both of, the following two provisions to prevent this: + + (a) The local resolver/name server can implement + caching of negative responses (see Section + 6.1.3.3). + + (b) The search list expander can require two or more + interior dots in a generated domain name before it + tries using the name in a query to non-local + domain servers, such as the root. + + DISCUSSION: + The intent of this requirement is to avoid + excessive delay for the user as the search list is + tested, and more importantly to prevent excessive + traffic to the root and other high-level servers. + For example, if the user supplied a name "X" and + the search list contained the root as a component, + + + +Internet Engineering Task Force [Page 83] + + + + +RFC1123 SUPPORT SERVICES -- DOMAINS October 1989 + + + a query would have to consult a root server before + the next search list alternative could be tried. + The resulting load seen by the root servers and + gateways near the root would be multiplied by the + number of hosts in the Internet. + + The negative caching alternative limits the effect + to the first time a name is used. The interior + dot rule is simpler to implement but can prevent + easy use of some top-level names. + + + 6.1.5 DOMAIN NAME SYSTEM REQUIREMENTS SUMMARY + + | | | | |S| | + | | | | |H| |F + | | | | |O|M|o + | | |S| |U|U|o + | | |H| |L|S|t + | |M|O| |D|T|n + | |U|U|M| | |o + | |S|L|A|N|N|t + | |T|D|Y|O|O|t +FEATURE |SECTION | | | |T|T|e +-----------------------------------------------|-----------|-|-|-|-|-|-- +GENERAL ISSUES | | | | | | | + | | | | | | | +Implement DNS name-to-address conversion |6.1.1 |x| | | | | +Implement DNS address-to-name conversion |6.1.1 |x| | | | | +Support conversions using host table |6.1.1 | | |x| | | +Properly handle RR with zero TTL |6.1.2.1 |x| | | | | +Use QCLASS=* unnecessarily |6.1.2.2 | |x| | | | + Use QCLASS=IN for Internet class |6.1.2.2 |x| | | | | +Unused fields zero |6.1.2.3 |x| | | | | +Use compression in responses |6.1.2.4 |x| | | | | + | | | | | | | +Include config info in responses |6.1.2.5 | | | | |x| +Support all well-known, class-indep. types |6.1.3.5 |x| | | | | +Easily expand type list |6.1.3.5 | |x| | | | +Load all RR types (except MD and MF) |6.1.3.6 |x| | | | | +Load MD or MF type |6.1.3.6 | | | | |x| +Operate when root servers, etc. unavailable |6.1.3.7 |x| | | | | +-----------------------------------------------|-----------|-|-|-|-|-|-- +RESOLVER ISSUES: | | | | | | | + | | | | | | | +Resolver support multiple concurrent requests |6.1.3.1 | |x| | | | +Full-service resolver: |6.1.3.1 | | |x| | | + Local caching |6.1.3.1 |x| | | | | + + + +Internet Engineering Task Force [Page 84] + + + + +RFC1123 SUPPORT SERVICES -- DOMAINS October 1989 + + + Information in local cache times out |6.1.3.1 |x| | | | | + Configurable with starting info |6.1.3.1 | |x| | | | +Stub resolver: |6.1.3.1 | | |x| | | + Use redundant recursive name servers |6.1.3.1 |x| | | | | + Local caching |6.1.3.1 | | |x| | | + Information in local cache times out |6.1.3.1 |x| | | | | +Support for remote multi-homed hosts: | | | | | | | + Sort multiple addresses by preference list |6.1.3.4 | |x| | | | + | | | | | | | +-----------------------------------------------|-----------|-|-|-|-|-|-- +TRANSPORT PROTOCOLS: | | | | | | | + | | | | | | | +Support UDP queries |6.1.3.2 |x| | | | | +Support TCP queries |6.1.3.2 | |x| | | | + Send query using UDP first |6.1.3.2 |x| | | | |1 + Try TCP if UDP answers are truncated |6.1.3.2 | |x| | | | +Name server limit TCP query resources |6.1.3.2 | | |x| | | + Punish unnecessary TCP query |6.1.3.2 | | | |x| | +Use truncated data as if it were not |6.1.3.2 | | | | |x| +Private agreement to use only TCP |6.1.3.2 | | |x| | | +Use TCP for zone transfers |6.1.3.2 |x| | | | | +TCP usage not block UDP queries |6.1.3.2 |x| | | | | +Support broadcast or multicast queries |6.1.3.2 | | |x| | | + RD bit set in query |6.1.3.2 | | | | |x| + RD bit ignored by server is b'cast/m'cast |6.1.3.2 |x| | | | | + Send only as occasional probe for addr's |6.1.3.2 | |x| | | | +-----------------------------------------------|-----------|-|-|-|-|-|-- +RESOURCE USAGE: | | | | | | | + | | | | | | | +Transmission controls, per [DNS:2] |6.1.3.3 |x| | | | | + Finite bounds per request |6.1.3.3 |x| | | | | +Failure after retries => soft error |6.1.3.3 |x| | | | | +Cache temporary failures |6.1.3.3 | |x| | | | +Cache negative responses |6.1.3.3 | |x| | | | +Retries use exponential backoff |6.1.3.3 | |x| | | | + Upper, lower bounds |6.1.3.3 | |x| | | | +Client handle Source Quench |6.1.3.3 | |x| | | | +Server ignore Source Quench |6.1.3.3 | | |x| | | +-----------------------------------------------|-----------|-|-|-|-|-|-- +USER INTERFACE: | | | | | | | + | | | | | | | +All programs have access to DNS interface |6.1.4.2 |x| | | | | +Able to request all info for given name |6.1.4.2 |x| | | | | +Returns complete info or error |6.1.4.2 |x| | | | | +Special interfaces |6.1.4.2 | | |x| | | + Name<->Address translation |6.1.4.2 |x| | | | | + | | | | | | | +Abbreviation Facilities: |6.1.4.3 | | |x| | | + + + +Internet Engineering Task Force [Page 85] + + + + +RFC1123 SUPPORT SERVICES -- DOMAINS October 1989 + + + Convention for complete names |6.1.4.3 |x| | | | | + Conversion exactly once |6.1.4.3 |x| | | | | + Conversion in proper context |6.1.4.3 |x| | | | | + Search list: |6.1.4.3 | | |x| | | + Administrator can disable |6.1.4.3 | |x| | | | + Prevention of excessive root queries |6.1.4.3 |x| | | | | + Both methods |6.1.4.3 | |x| | | | +-----------------------------------------------|-----------|-|-|-|-|-|-- +-----------------------------------------------|-----------|-|-|-|-|-|-- + +1. Unless there is private agreement between particular resolver and + particular server. + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +Internet Engineering Task Force [Page 86] + + + + +RFC1123 SUPPORT SERVICES -- INITIALIZATION October 1989 + + + 6.2 HOST INITIALIZATION + + 6.2.1 INTRODUCTION + + This section discusses the initialization of host software + across a connected network, or more generally across an + Internet path. This is necessary for a diskless host, and may + optionally be used for a host with disk drives. For a diskless + host, the initialization process is called "network booting" + and is controlled by a bootstrap program located in a boot ROM. + + To initialize a diskless host across the network, there are two + distinct phases: + + (1) Configure the IP layer. + + Diskless machines often have no permanent storage in which + to store network configuration information, so that + sufficient configuration information must be obtained + dynamically to support the loading phase that follows. + This information must include at least the IP addresses of + the host and of the boot server. To support booting + across a gateway, the address mask and a list of default + gateways are also required. + + (2) Load the host system code. + + During the loading phase, an appropriate file transfer + protocol is used to copy the system code across the + network from the boot server. + + A host with a disk may perform the first step, dynamic + configuration. This is important for microcomputers, whose + floppy disks allow network configuration information to be + mistakenly duplicated on more than one host. Also, + installation of new hosts is much simpler if they automatically + obtain their configuration information from a central server, + saving administrator time and decreasing the probability of + mistakes. + + 6.2.2 REQUIREMENTS + + 6.2.2.1 Dynamic Configuration + + A number of protocol provisions have been made for dynamic + configuration. + + o ICMP Information Request/Reply messages + + + +Internet Engineering Task Force [Page 87] + + + + +RFC1123 SUPPORT SERVICES -- INITIALIZATION October 1989 + + + This obsolete message pair was designed to allow a host + to find the number of the network it is on. + Unfortunately, it was useful only if the host already + knew the host number part of its IP address, + information that hosts requiring dynamic configuration + seldom had. + + o Reverse Address Resolution Protocol (RARP) [BOOT:4] + + RARP is a link-layer protocol for a broadcast medium + that allows a host to find its IP address given its + link layer address. Unfortunately, RARP does not work + across IP gateways and therefore requires a RARP server + on every network. In addition, RARP does not provide + any other configuration information. + + o ICMP Address Mask Request/Reply messages + + These ICMP messages allow a host to learn the address + mask for a particular network interface. + + o BOOTP Protocol [BOOT:2] + + This protocol allows a host to determine the IP + addresses of the local host and the boot server, the + name of an appropriate boot file, and optionally the + address mask and list of default gateways. To locate a + BOOTP server, the host broadcasts a BOOTP request using + UDP. Ad hoc gateway extensions have been used to + transmit the BOOTP broadcast through gateways, and in + the future the IP Multicasting facility will provide a + standard mechanism for this purpose. + + + The suggested approach to dynamic configuration is to use + the BOOTP protocol with the extensions defined in "BOOTP + Vendor Information Extensions" RFC-1084 [BOOT:3]. RFC-1084 + defines some important general (not vendor-specific) + extensions. In particular, these extensions allow the + address mask to be supplied in BOOTP; we RECOMMEND that the + address mask be supplied in this manner. + + DISCUSSION: + Historically, subnetting was defined long after IP, and + so a separate mechanism (ICMP Address Mask messages) + was designed to supply the address mask to a host. + However, the IP address mask and the corresponding IP + address conceptually form a pair, and for operational + + + +Internet Engineering Task Force [Page 88] + + + + +RFC1123 SUPPORT SERVICES -- INITIALIZATION October 1989 + + + simplicity they ought to be defined at the same time + and by the same mechanism, whether a configuration file + or a dynamic mechanism like BOOTP. + + Note that BOOTP is not sufficiently general to specify + the configurations of all interfaces of a multihomed + host. A multihomed host must either use BOOTP + separately for each interface, or configure one + interface using BOOTP to perform the loading, and + perform the complete initialization from a file later. + + Application layer configuration information is expected + to be obtained from files after loading of the system + code. + + 6.2.2.2 Loading Phase + + A suggested approach for the loading phase is to use TFTP + [BOOT:1] between the IP addresses established by BOOTP. + + TFTP to a broadcast address SHOULD NOT be used, for reasons + explained in Section 4.2.3.4. + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +Internet Engineering Task Force [Page 89] + + + + +RFC1123 SUPPORT SERVICES -- MANAGEMENT October 1989 + + + 6.3 REMOTE MANAGEMENT + + 6.3.1 INTRODUCTION + + The Internet community has recently put considerable effort + into the development of network management protocols. The + result has been a two-pronged approach [MGT:1, MGT:6]: the + Simple Network Management Protocol (SNMP) [MGT:4] and the + Common Management Information Protocol over TCP (CMOT) [MGT:5]. + + In order to be managed using SNMP or CMOT, a host will need to + implement an appropriate management agent. An Internet host + SHOULD include an agent for either SNMP or CMOT. + + Both SNMP and CMOT operate on a Management Information Base + (MIB) that defines a collection of management values. By + reading and setting these values, a remote application may + query and change the state of the managed system. + + A standard MIB [MGT:3] has been defined for use by both + management protocols, using data types defined by the Structure + of Management Information (SMI) defined in [MGT:2]. Additional + MIB variables can be introduced under the "enterprises" and + "experimental" subtrees of the MIB naming space [MGT:2]. + + Every protocol module in the host SHOULD implement the relevant + MIB variables. A host SHOULD implement the MIB variables as + defined in the most recent standard MIB, and MAY implement + other MIB variables when appropriate and useful. + + 6.3.2 PROTOCOL WALK-THROUGH + + The MIB is intended to cover both hosts and gateways, although + there may be detailed differences in MIB application to the two + cases. This section contains the appropriate interpretation of + the MIB for hosts. It is likely that later versions of the MIB + will include more entries for host management. + + A managed host must implement the following groups of MIB + object definitions: System, Interfaces, Address Translation, + IP, ICMP, TCP, and UDP. + + The following specific interpretations apply to hosts: + + o ipInHdrErrors + + Note that the error "time-to-live exceeded" can occur in a + host only when it is forwarding a source-routed datagram. + + + +Internet Engineering Task Force [Page 90] + + + + +RFC1123 SUPPORT SERVICES -- MANAGEMENT October 1989 + + + o ipOutNoRoutes + + This object counts datagrams discarded because no route + can be found. This may happen in a host if all the + default gateways in the host's configuration are down. + + o ipFragOKs, ipFragFails, ipFragCreates + + A host that does not implement intentional fragmentation + (see "Fragmentation" section of [INTRO:1]) MUST return the + value zero for these three objects. + + o icmpOutRedirects + + For a host, this object MUST always be zero, since hosts + do not send Redirects. + + o icmpOutAddrMaskReps + + For a host, this object MUST always be zero, unless the + host is an authoritative source of address mask + information. + + o ipAddrTable + + For a host, the "IP Address Table" object is effectively a + table of logical interfaces. + + o ipRoutingTable + + For a host, the "IP Routing Table" object is effectively a + combination of the host's Routing Cache and the static + route table described in "Routing Outbound Datagrams" + section of [INTRO:1]. + + Within each ipRouteEntry, ipRouteMetric1...4 normally will + have no meaning for a host and SHOULD always be -1, while + ipRouteType will normally have the value "remote". + + If destinations on the connected network do not appear in + the Route Cache (see "Routing Outbound Datagrams section + of [INTRO:1]), there will be no entries with ipRouteType + of "direct". + + + DISCUSSION: + The current MIB does not include Type-of-Service in an + ipRouteEntry, but a future revision is expected to make + + + +Internet Engineering Task Force [Page 91] + + + + +RFC1123 SUPPORT SERVICES -- MANAGEMENT October 1989 + + + this addition. + + We also expect the MIB to be expanded to allow the remote + management of applications (e.g., the ability to partially + reconfigure mail systems). Network service applications + such as mail systems should therefore be written with the + "hooks" for remote management. + + 6.3.3 MANAGEMENT REQUIREMENTS SUMMARY + + | | | | |S| | + | | | | |H| |F + | | | | |O|M|o + | | |S| |U|U|o + | | |H| |L|S|t + | |M|O| |D|T|n + | |U|U|M| | |o + | |S|L|A|N|N|t + | |T|D|Y|O|O|t +FEATURE |SECTION | | | |T|T|e +-----------------------------------------------|-----------|-|-|-|-|-|-- +Support SNMP or CMOT agent |6.3.1 | |x| | | | +Implement specified objects in standard MIB |6.3.1 | |x| | | | + + + + + + + + + + + + + + + + + + + + + + + + + + + + +Internet Engineering Task Force [Page 92] + + + + +RFC1123 SUPPORT SERVICES -- MANAGEMENT October 1989 + + +7. REFERENCES + + This section lists the primary references with which every + implementer must be thoroughly familiar. It also lists some + secondary references that are suggested additional reading. + + INTRODUCTORY REFERENCES: + + + [INTRO:1] "Requirements for Internet Hosts -- Communication Layers," + IETF Host Requirements Working Group, R. Braden, Ed., RFC-1122, + October 1989. + + [INTRO:2] "DDN Protocol Handbook," NIC-50004, NIC-50005, NIC-50006, + (three volumes), SRI International, December 1985. + + [INTRO:3] "Official Internet Protocols," J. Reynolds and J. Postel, + RFC-1011, May 1987. + + This document is republished periodically with new RFC numbers; + the latest version must be used. + + [INTRO:4] "Protocol Document Order Information," O. Jacobsen and J. + Postel, RFC-980, March 1986. + + [INTRO:5] "Assigned Numbers," J. Reynolds and J. Postel, RFC-1010, + May 1987. + + This document is republished periodically with new RFC numbers; + the latest version must be used. + + + TELNET REFERENCES: + + + [TELNET:1] "Telnet Protocol Specification," J. Postel and J. + Reynolds, RFC-854, May 1983. + + [TELNET:2] "Telnet Option Specification," J. Postel and J. Reynolds, + RFC-855, May 1983. + + [TELNET:3] "Telnet Binary Transmission," J. Postel and J. Reynolds, + RFC-856, May 1983. + + [TELNET:4] "Telnet Echo Option," J. Postel and J. Reynolds, RFC-857, + May 1983. + + [TELNET:5] "Telnet Suppress Go Ahead Option," J. Postel and J. + + + +Internet Engineering Task Force [Page 93] + + + + +RFC1123 SUPPORT SERVICES -- MANAGEMENT October 1989 + + + Reynolds, RFC-858, May 1983. + + [TELNET:6] "Telnet Status Option," J. Postel and J. Reynolds, RFC- + 859, May 1983. + + [TELNET:7] "Telnet Timing Mark Option," J. Postel and J. Reynolds, + RFC-860, May 1983. + + [TELNET:8] "Telnet Extended Options List," J. Postel and J. + Reynolds, RFC-861, May 1983. + + [TELNET:9] "Telnet End-Of-Record Option," J. Postel, RFC-855, + December 1983. + + [TELNET:10] "Telnet Terminal-Type Option," J. VanBokkelen, RFC-1091, + February 1989. + + This document supercedes RFC-930. + + [TELNET:11] "Telnet Window Size Option," D. Waitzman, RFC-1073, + October 1988. + + [TELNET:12] "Telnet Linemode Option," D. Borman, RFC-1116, August + 1989. + + [TELNET:13] "Telnet Terminal Speed Option," C. Hedrick, RFC-1079, + December 1988. + + [TELNET:14] "Telnet Remote Flow Control Option," C. Hedrick, RFC- + 1080, November 1988. + + + SECONDARY TELNET REFERENCES: + + + [TELNET:15] "Telnet Protocol," MIL-STD-1782, U.S. Department of + Defense, May 1984. + + This document is intended to describe the same protocol as RFC- + 854. In case of conflict, RFC-854 takes precedence, and the + present document takes precedence over both. + + [TELNET:16] "SUPDUP Protocol," M. Crispin, RFC-734, October 1977. + + [TELNET:17] "Telnet SUPDUP Option," M. Crispin, RFC-736, October + 1977. + + [TELNET:18] "Data Entry Terminal Option," J. Day, RFC-732, June 1977. + + + +Internet Engineering Task Force [Page 94] + + + + +RFC1123 SUPPORT SERVICES -- MANAGEMENT October 1989 + + + [TELNET:19] "TELNET Data Entry Terminal option -- DODIIS + Implementation," A. Yasuda and T. Thompson, RFC-1043, February + 1988. + + + FTP REFERENCES: + + + [FTP:1] "File Transfer Protocol," J. Postel and J. Reynolds, RFC- + 959, October 1985. + + [FTP:2] "Document File Format Standards," J. Postel, RFC-678, + December 1974. + + [FTP:3] "File Transfer Protocol," MIL-STD-1780, U.S. Department of + Defense, May 1984. + + This document is based on an earlier version of the FTP + specification (RFC-765) and is obsolete. + + + TFTP REFERENCES: + + + [TFTP:1] "The TFTP Protocol Revision 2," K. Sollins, RFC-783, June + 1981. + + + MAIL REFERENCES: + + + [SMTP:1] "Simple Mail Transfer Protocol," J. Postel, RFC-821, August + 1982. + + [SMTP:2] "Standard For The Format of ARPA Internet Text Messages," + D. Crocker, RFC-822, August 1982. + + This document obsoleted an earlier specification, RFC-733. + + [SMTP:3] "Mail Routing and the Domain System," C. Partridge, RFC- + 974, January 1986. + + This RFC describes the use of MX records, a mandatory extension + to the mail delivery process. + + [SMTP:4] "Duplicate Messages and SMTP," C. Partridge, RFC-1047, + February 1988. + + + + +Internet Engineering Task Force [Page 95] + + + + +RFC1123 SUPPORT SERVICES -- MANAGEMENT October 1989 + + + [SMTP:5a] "Mapping between X.400 and RFC 822," S. Kille, RFC-987, + June 1986. + + [SMTP:5b] "Addendum to RFC-987," S. Kille, RFC-???, September 1987. + + The two preceding RFC's define a proposed standard for + gatewaying mail between the Internet and the X.400 environments. + + [SMTP:6] "Simple Mail Transfer Protocol," MIL-STD-1781, U.S. + Department of Defense, May 1984. + + This specification is intended to describe the same protocol as + does RFC-821. However, MIL-STD-1781 is incomplete; in + particular, it does not include MX records [SMTP:3]. + + [SMTP:7] "A Content-Type Field for Internet Messages," M. Sirbu, + RFC-1049, March 1988. + + + DOMAIN NAME SYSTEM REFERENCES: + + + [DNS:1] "Domain Names - Concepts and Facilities," P. Mockapetris, + RFC-1034, November 1987. + + This document and the following one obsolete RFC-882, RFC-883, + and RFC-973. + + [DNS:2] "Domain Names - Implementation and Specification," RFC-1035, + P. Mockapetris, November 1987. + + + [DNS:3] "Mail Routing and the Domain System," C. Partridge, RFC-974, + January 1986. + + + [DNS:4] "DoD Internet Host Table Specification," K. Harrenstein, + RFC-952, M. Stahl, E. Feinler, October 1985. + + SECONDARY DNS REFERENCES: + + + [DNS:5] "Hostname Server," K. Harrenstein, M. Stahl, E. Feinler, + RFC-953, October 1985. + + [DNS:6] "Domain Administrators Guide," M. Stahl, RFC-1032, November + 1987. + + + + +Internet Engineering Task Force [Page 96] + + + + +RFC1123 SUPPORT SERVICES -- MANAGEMENT October 1989 + + + [DNS:7] "Domain Administrators Operations Guide," M. Lottor, RFC- + 1033, November 1987. + + [DNS:8] "The Domain Name System Handbook," Vol. 4 of Internet + Protocol Handbook, NIC 50007, SRI Network Information Center, + August 1989. + + + SYSTEM INITIALIZATION REFERENCES: + + + [BOOT:1] "Bootstrap Loading Using TFTP," R. Finlayson, RFC-906, June + 1984. + + [BOOT:2] "Bootstrap Protocol (BOOTP)," W. Croft and J. Gilmore, RFC- + 951, September 1985. + + [BOOT:3] "BOOTP Vendor Information Extensions," J. Reynolds, RFC- + 1084, December 1988. + + Note: this RFC revised and obsoleted RFC-1048. + + [BOOT:4] "A Reverse Address Resolution Protocol," R. Finlayson, T. + Mann, J. Mogul, and M. Theimer, RFC-903, June 1984. + + + MANAGEMENT REFERENCES: + + + [MGT:1] "IAB Recommendations for the Development of Internet Network + Management Standards," V. Cerf, RFC-1052, April 1988. + + [MGT:2] "Structure and Identification of Management Information for + TCP/IP-based internets," M. Rose and K. McCloghrie, RFC-1065, + August 1988. + + [MGT:3] "Management Information Base for Network Management of + TCP/IP-based internets," M. Rose and K. McCloghrie, RFC-1066, + August 1988. + + [MGT:4] "A Simple Network Management Protocol," J. Case, M. Fedor, + M. Schoffstall, and C. Davin, RFC-1098, April 1989. + + [MGT:5] "The Common Management Information Services and Protocol + over TCP/IP," U. Warrier and L. Besaw, RFC-1095, April 1989. + + [MGT:6] "Report of the Second Ad Hoc Network Management Review + Group," V. Cerf, RFC-1109, August 1989. + + + +Internet Engineering Task Force [Page 97] + + + + +RFC1123 SUPPORT SERVICES -- MANAGEMENT October 1989 + + +Security Considerations + + There are many security issues in the application and support + programs of host software, but a full discussion is beyond the scope + of this RFC. Security-related issues are mentioned in sections + concerning TFTP (Sections 4.2.1, 4.2.3.4, 4.2.3.5), the SMTP VRFY and + EXPN commands (Section 5.2.3), the SMTP HELO command (5.2.5), and the + SMTP DATA command (Section 5.2.8). + +Author's Address + + Robert Braden + USC/Information Sciences Institute + 4676 Admiralty Way + Marina del Rey, CA 90292-6695 + + Phone: (213) 822 1511 + + EMail: Braden@ISI.EDU + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +Internet Engineering Task Force [Page 98] + diff --git a/doc/rfc2228.txt b/doc/rfc2228.txt new file mode 100644 index 0000000..1fbfcbf --- /dev/null +++ b/doc/rfc2228.txt @@ -0,0 +1,1515 @@ + + + + + + +Network Working Group M. Horowitz +Request for Comments: 2228 Cygnus Solutions +Updates: 959 S. Lunt +Category: Standards Track Bellcore + October 1997 + + FTP Security Extensions + +Status of this Memo + + This document specifies an Internet standards track protocol for the + Internet community, and requests discussion and suggestions for + improvements. Please refer to the current edition of the "Internet + Official Protocol Standards" (STD 1) for the standardization state + and status of this protocol. Distribution of this memo is unlimited. + +Copyright Notice + + Copyright (C) The Internet Society (1997). All Rights Reserved. + +Abstract + + This document defines extensions to the FTP specification STD 9, RFC + 959, "FILE TRANSFER PROTOCOL (FTP)" (October 1985). These extensions + provide strong authentication, integrity, and confidentiality on both + the control and data channels with the introduction of new optional + commands, replies, and file transfer encodings. + + The following new optional commands are introduced in this + specification: + + AUTH (Authentication/Security Mechanism), + ADAT (Authentication/Security Data), + PROT (Data Channel Protection Level), + PBSZ (Protection Buffer Size), + CCC (Clear Command Channel), + MIC (Integrity Protected Command), + CONF (Confidentiality Protected Command), and + ENC (Privacy Protected Command). + + A new class of reply types (6yz) is also introduced for protected + replies. + + None of the above commands are required to be implemented, but + interdependencies exist. These dependencies are documented with the + commands. + + Note that this specification is compatible with STD 9, RFC 959. + + + +Horowitz & Lunt Standards Track [Page 1] + +RFC 2228 FTP Security Extensions October 1997 + + +1. Introduction + + The File Transfer Protocol (FTP) currently defined in STD 9, RFC 959 + and in place on the Internet uses usernames and passwords passed in + cleartext to authenticate clients to servers (via the USER and PASS + commands). Except for services such as "anonymous" FTP archives, + this represents a security risk whereby passwords can be stolen + through monitoring of local and wide-area networks. This either aids + potential attackers through password exposure and/or limits + accessibility of files by FTP servers who cannot or will not accept + the inherent security risks. + + Aside from the problem of authenticating users in a secure manner, + there is also the problem of authenticating servers, protecting + sensitive data and/or verifying its integrity. An attacker may be + able to access valuable or sensitive data merely by monitoring a + network, or through active means may be able to delete or modify the + data being transferred so as to corrupt its integrity. An active + attacker may also initiate spurious file transfers to and from a site + of the attacker's choice, and may invoke other commands on the + server. FTP does not currently have any provision for the encryption + or verification of the authenticity of commands, replies, or + transferred data. Note that these security services have value even + to anonymous file access. + + Current practice for sending files securely is generally either: + + 1. via FTP of files pre-encrypted under keys which are manually + distributed, + + 2. via electronic mail containing an encoding of a file encrypted + under keys which are manually distributed, + + 3. via a PEM message, or + + 4. via the rcp command enhanced to use Kerberos. + + None of these means could be considered even a de facto standard, and + none are truly interactive. A need exists to securely transfer files + using FTP in a secure manner which is supported within the FTP + protocol in a consistent manner and which takes advantage of existing + security infrastructure and technology. Extensions are necessary to + the FTP specification if these security services are to be introduced + into the protocol in an interoperable way. + + + + + + + +Horowitz & Lunt Standards Track [Page 2] + +RFC 2228 FTP Security Extensions October 1997 + + + Although the FTP control connection follows the Telnet protocol, and + Telnet has defined an authentication and encryption option [TELNET- + SEC], [RFC-1123] explicitly forbids the use of Telnet option + negotiation over the control connection (other than Synch and IP). + + Also, the Telnet authentication and encryption option does not + provide for integrity protection only (without confidentiality), and + does not address the protection of the data channel. + +2. FTP Security Overview + + At the highest level, the FTP security extensions seek to provide an + abstract mechanism for authenticating and/or authorizing connections, + and integrity and/or confidentiality protecting commands, replies, + and data transfers. + + In the context of FTP security, authentication is the establishment + of a client's identity and/or a server's identity in a secure way, + usually using cryptographic techniques. The basic FTP protocol does + not have a concept of authentication. + + Authorization is the process of validating a user for login. The + basic authorization process involves the USER, PASS, and ACCT + commands. With the FTP security extensions, authentication + established using a security mechanism may also be used to make the + authorization decision. + + Without the security extensions, authentication of the client, as + this term is usually understood, never happens. FTP authorization is + accomplished with a password, passed on the network in the clear as + the argument to the PASS command. The possessor of this password is + assumed to be authorized to transfer files as the user named in the + USER command, but the identity of the client is never securely + established. + + An FTP security interaction begins with a client telling the server + what security mechanism it wants to use with the AUTH command. The + server will either accept this mechanism, reject this mechanism, or, + in the case of a server which does not implement the security + extensions, reject the command completely. The client may try + multiple security mechanisms until it requests one which the server + accepts. This allows a rudimentary form of negotiation to take + place. (If more complex negotiation is desired, this may be + implemented as a security mechanism.) The server's reply will + indicate if the client must respond with additional data for the + + + + + + +Horowitz & Lunt Standards Track [Page 3] + +RFC 2228 FTP Security Extensions October 1997 + + + security mechanism to interpret. If none is needed, this will + usually mean that the mechanism is one where the password (specified + by the PASS command) is to be interpreted differently, such as with a + token or one-time password system. + + If the server requires additional security information, then the + client and server will enter into a security data exchange. The + client will send an ADAT command containing the first block of + security data. The server's reply will indicate if the data exchange + is complete, if there was an error, or if more data is needed. The + server's reply can optionally contain security data for the client to + interpret. If more data is needed, the client will send another ADAT + command containing the next block of data, and await the server's + reply. This exchange can continue as many times as necessary. Once + this exchange completes, the client and server have established a + security association. This security association may include + authentication (client, server, or mutual) and keying information for + integrity and/or confidentiality, depending on the mechanism in use. + + The term "security data" here is carefully chosen. The purpose of + the security data exchange is to establish a security association, + which might not actually include any authentication at all, between + the client and the server as described above. For instance, a + Diffie-Hellman exchange establishes a secret key, but no + authentication takes place. If an FTP server has an RSA key pair but + the client does not, then the client can authenticate the server, but + the server cannot authenticate the client. + + Once a security association is established, authentication which is a + part of this association may be used instead of or in addition to the + standard username/password exchange for authorizing a user to connect + to the server. A username specified by the USER command is always + required to specify the identity to be used on the server. + + In order to prevent an attacker from inserting or deleting commands + on the control stream, if the security association supports + integrity, then the server and client must use integrity protection + on the control stream, unless it first transmits a CCC command to + turn off this requirement. Integrity protection is performed with + the MIC and ENC commands, and the 63z reply codes. The CCC command + and its reply must be transmitted with integrity protection. + Commands and replies may be transmitted without integrity (that is, + in the clear or with confidentiality only) only if no security + association is established, the negotiated security association does + not support integrity, or the CCC command has succeeded. + + + + + + +Horowitz & Lunt Standards Track [Page 4] + +RFC 2228 FTP Security Extensions October 1997 + + + Once the client and server have negotiated with the PBSZ command an + acceptable buffer size for encapsulating protected data over the data + channel, the security mechanism may also be used to protect data + channel transfers. + + Policy is not specified by this document. In particular, client and + server implementations may choose to implement restrictions on what + operations can be performed depending on the security association + which exists. For example, a server may require that a client + authorize via a security mechanism rather than using a password, + require that the client provide a one-time password from a token, + require at least integrity protection on the command channel, or + require that certain files only be transmitted encrypted. An + anonymous ftp client might refuse to do file transfers without + integrity protection in order to insure the validity of files + downloaded. + + No particular set of functionality is required, except as + dependencies described in the next section. This means that none of + authentication, integrity, or confidentiality are required of an + implementation, although a mechanism which does none of these is not + of much use. For example, it is acceptable for a mechanism to + implement only integrity protection, one-way authentication and/or + encryption, encryption without any authentication or integrity + protection, or any other subset of functionality if policy or + technical considerations make this desirable. Of course, one peer + might require as a matter of policy stronger protection than the + other is able to provide, preventing perfect interoperability. + +3. New FTP Commands + + The following commands are optional, but dependent on each other. + They are extensions to the FTP Access Control Commands. + + The reply codes documented here are generally described as + recommended, rather than required. The intent is that reply codes + describing the full range of success and failure modes exist, but + that servers be allowed to limit information presented to the client. + For example, a server might implement a particular security + mechanism, but have a policy restriction against using it. The + server should respond with a 534 reply code in this case, but may + respond with a 504 reply code if it does not wish to divulge that the + disallowed mechanism is supported. If the server does choose to use + a different reply code than the recommended one, it should try to use + a reply code which only differs in the last digit. In all cases, the + server must use a reply code which is documented as returnable from + the command received, and this reply code must begin with the same + digit as the recommended reply code for the situation. + + + +Horowitz & Lunt Standards Track [Page 5] + +RFC 2228 FTP Security Extensions October 1997 + + + AUTHENTICATION/SECURITY MECHANISM (AUTH) + + The argument field is a Telnet string identifying a supported + mechanism. This string is case-insensitive. Values must be + registered with the IANA, except that values beginning with "X-" + are reserved for local use. + + If the server does not recognize the AUTH command, it must respond + with reply code 500. This is intended to encompass the large + deployed base of non-security-aware ftp servers, which will + respond with reply code 500 to any unrecognized command. If the + server does recognize the AUTH command but does not implement the + security extensions, it should respond with reply code 502. + + If the server does not understand the named security mechanism, it + should respond with reply code 504. + + If the server is not willing to accept the named security + mechanism, it should respond with reply code 534. + + If the server is not able to accept the named security mechanism, + such as if a required resource is unavailable, it should respond + with reply code 431. + + If the server is willing to accept the named security mechanism, + but requires security data, it must respond with reply code 334. + + If the server is willing to accept the named security mechanism, + and does not require any security data, it must respond with reply + code 234. + + If the server is responding with a 334 reply code, it may include + security data as described in the next section. + + Some servers will allow the AUTH command to be reissued in order + to establish new authentication. The AUTH command, if accepted, + removes any state associated with prior FTP Security commands. + The server must also require that the user reauthorize (that is, + reissue some or all of the USER, PASS, and ACCT commands) in this + case (see section 4 for an explanation of "authorize" in this + context). + + + + + + + + + + +Horowitz & Lunt Standards Track [Page 6] + +RFC 2228 FTP Security Extensions October 1997 + + + AUTHENTICATION/SECURITY DATA (ADAT) + + The argument field is a Telnet string representing base 64 encoded + security data (see Section 9, "Base 64 Encoding"). If a reply + code indicating success is returned, the server may also use a + string of the form "ADAT=base64data" as the text part of the reply + if it wishes to convey security data back to the client. + + The data in both cases is specific to the security mechanism + specified by the previous AUTH command. The ADAT command, and the + associated replies, allow the client and server to conduct an + arbitrary security protocol. The security data exchange must + include enough information for both peers to be aware of which + optional features are available. For example, if the client does + not support data encryption, the server must be made aware of + this, so it will know not to send encrypted command channel + replies. It is strongly recommended that the security mechanism + provide sequencing on the command channel, to insure that commands + are not deleted, reordered, or replayed. + + The ADAT command must be preceded by a successful AUTH command, + and cannot be issued once a security data exchange completes + (successfully or unsuccessfully), unless it is preceded by an AUTH + command to reset the security state. + + If the server has not yet received an AUTH command, or if a prior + security data exchange completed, but the security state has not + been reset with an AUTH command, it should respond with reply code + 503. + + If the server cannot base 64 decode the argument, it should + respond with reply code 501. + + If the server rejects the security data (if a checksum fails, for + instance), it should respond with reply code 535. + + If the server accepts the security data, and requires additional + data, it should respond with reply code 335. + + If the server accepts the security data, but does not require any + additional data (i.e., the security data exchange has completed + successfully), it must respond with reply code 235. + + If the server is responding with a 235 or 335 reply code, then it + may include security data in the text part of the reply as + specified above. + + + + + +Horowitz & Lunt Standards Track [Page 7] + +RFC 2228 FTP Security Extensions October 1997 + + + If the ADAT command returns an error, the security data exchange + will fail, and the client must reset its internal security state. + If the client becomes unsynchronized with the server (for example, + the server sends a 234 reply code to an AUTH command, but the + client has more data to transmit), then the client must reset the + server's security state. + + PROTECTION BUFFER SIZE (PBSZ) + + The argument is a decimal integer representing the maximum size, + in bytes, of the encoded data blocks to be sent or received during + file transfer. This number shall be no greater than can be + represented in a 32-bit unsigned integer. + + This command allows the FTP client and server to negotiate a + maximum protected buffer size for the connection. There is no + default size; the client must issue a PBSZ command before it can + issue the first PROT command. + + The PBSZ command must be preceded by a successful security data + exchange. + + If the server cannot parse the argument, or if it will not fit in + 32 bits, it should respond with a 501 reply code. + + If the server has not completed a security data exchange with the + client, it should respond with a 503 reply code. + + Otherwise, the server must reply with a 200 reply code. If the + size provided by the client is too large for the server, it must + use a string of the form "PBSZ=number" in the text part of the + reply to indicate a smaller buffer size. The client and the + server must use the smaller of the two buffer sizes if both buffer + sizes are specified. + + DATA CHANNEL PROTECTION LEVEL (PROT) + + The argument is a single Telnet character code specifying the data + channel protection level. + + This command indicates to the server what type of data channel + protection the client and server will be using. The following + codes are assigned: + + C - Clear + S - Safe + E - Confidential + P - Private + + + +Horowitz & Lunt Standards Track [Page 8] + +RFC 2228 FTP Security Extensions October 1997 + + + The default protection level if no other level is specified is + Clear. The Clear protection level indicates that the data channel + will carry the raw data of the file transfer, with no security + applied. The Safe protection level indicates that the data will + be integrity protected. The Confidential protection level + indicates that the data will be confidentiality protected. The + Private protection level indicates that the data will be integrity + and confidentiality protected. + + It is reasonable for a security mechanism not to provide all data + channel protection levels. It is also reasonable for a mechanism + to provide more protection at a level than is required (for + instance, a mechanism might provide Confidential protection, but + include integrity-protection in that encoding, due to API or other + considerations). + + The PROT command must be preceded by a successful protection + buffer size negotiation. + + If the server does not understand the specified protection level, + it should respond with reply code 504. + + If the current security mechanism does not support the specified + protection level, the server should respond with reply code 536. + + If the server has not completed a protection buffer size + negotiation with the client, it should respond with a 503 reply + code. + + The PROT command will be rejected and the server should reply 503 + if no previous PBSZ command was issued. + + If the server is not willing to accept the specified protection + level, it should respond with reply code 534. + + If the server is not able to accept the specified protection + level, such as if a required resource is unavailable, it should + respond with reply code 431. + + Otherwise, the server must reply with a 200 reply code to indicate + that the specified protection level is accepted. + + CLEAR COMMAND CHANNEL (CCC) + + This command does not take an argument. + + + + + + +Horowitz & Lunt Standards Track [Page 9] + +RFC 2228 FTP Security Extensions October 1997 + + + It is desirable in some environments to use a security mechanism + to authenticate and/or authorize the client and server, but not to + perform any integrity checking on the subsequent commands. This + might be used in an environment where IP security is in place, + insuring that the hosts are authenticated and that TCP streams + cannot be tampered, but where user authentication is desired. + + If unprotected commands are allowed on any connection, then an + attacker could insert a command on the control stream, and the + server would have no way to know that it was invalid. In order to + prevent such attacks, once a security data exchange completes + successfully, if the security mechanism supports integrity, then + integrity (via the MIC or ENC command, and 631 or 632 reply) must + be used, until the CCC command is issued to enable non-integrity + protected control channel messages. The CCC command itself must + be integrity protected. + + Once the CCC command completes successfully, if a command is not + protected, then the reply to that command must also not be + protected. This is to support interoperability with clients which + do not support protection once the CCC command has been issued. + + This command must be preceded by a successful security data + exchange. + + If the command is not integrity-protected, the server must respond + with a 533 reply code. + + If the server is not willing to turn off the integrity + requirement, it should respond with a 534 reply code. + + Otherwise, the server must reply with a 200 reply code to indicate + that unprotected commands and replies may now be used on the + command channel. + + INTEGRITY PROTECTED COMMAND (MIC) and + CONFIDENTIALITY PROTECTED COMMAND (CONF) and + PRIVACY PROTECTED COMMAND (ENC) + + The argument field of MIC is a Telnet string consisting of a base + 64 encoded "safe" message produced by a security mechanism + specific message integrity procedure. The argument field of CONF + is a Telnet string consisting of a base 64 encoded "confidential" + message produced by a security mechanism specific confidentiality + procedure. The argument field of ENC is a Telnet string + consisting of a base 64 encoded "private" message produced by a + security mechanism specific message integrity and confidentiality + procedure. + + + +Horowitz & Lunt Standards Track [Page 10] + +RFC 2228 FTP Security Extensions October 1997 + + + The server will decode and/or verify the encoded message. + + This command must be preceded by a successful security data + exchange. + + A server may require that the first command after a successful + security data exchange be CCC, and not implement the protection + commands at all. In this case, the server should respond with a + 502 reply code. + + If the server cannot base 64 decode the argument, it should + respond with a 501 reply code. + + If the server has not completed a security data exchange with the + client, it should respond with a 503 reply code. + + If the server has completed a security data exchange with the + client using a mechanism which supports integrity, and requires a + CCC command due to policy or implementation limitations, it should + respond with a 503 reply code. + + If the server rejects the command because it is not supported by + the current security mechanism, the server should respond with + reply code 537. + + If the server rejects the command (if a checksum fails, for + instance), it should respond with reply code 535. + + If the server is not willing to accept the command (if privacy is + required by policy, for instance, or if a CONF command is received + before a CCC command), it should respond with reply code 533. + + Otherwise, the command will be interpreted as an FTP command. An + end-of-line code need not be included, but if one is included, it + must be a Telnet end-of-line code, not a local end-of-line code. + + The server may require that, under some or all circumstances, all + commands be protected. In this case, it should make a 533 reply + to commands other than MIC, CONF, and ENC. + +4. Login Authorization + + The security data exchange may, among other things, establish the + identity of the client in a secure way to the server. This identity + may be used as one input to the login authorization process. + + + + + + +Horowitz & Lunt Standards Track [Page 11] + +RFC 2228 FTP Security Extensions October 1997 + + + In response to the FTP login commands (AUTH, PASS, ACCT), the server + may choose to change the sequence of commands and replies specified + by RFC 959 as follows. There are also some new replies available. + + If the server is willing to allow the user named by the USER command + to log in based on the identity established by the security data + exchange, it should respond with reply code 232. + + If the security mechanism requires a challenge/response password, it + should respond to the USER command with reply code 336. The text + part of the reply should contain the challenge. The client must + display the challenge to the user before prompting for the password + in this case. This is particularly relevant to more sophisticated + clients or graphical user interfaces which provide dialog boxes or + other modal input. These clients should be careful not to prompt for + the password before the username has been sent to the server, in case + the user needs the challenge in the 336 reply to construct a valid + password. + +5. New FTP Replies + + The new reply codes are divided into two classes. The first class is + new replies made necessary by the new FTP Security commands. The + second class is a new reply type to indicate protected replies. + + 5.1. New individual reply codes + + 232 User logged in, authorized by security data exchange. + 234 Security data exchange complete. + 235 [ADAT=base64data] + ; This reply indicates that the security data exchange + ; completed successfully. The square brackets are not + ; to be included in the reply, but indicate that + ; security data in the reply is optional. + + 334 [ADAT=base64data] + ; This reply indicates that the requested security mechanism + ; is ok, and includes security data to be used by the client + ; to construct the next command. The square brackets are not + ; to be included in the reply, but indicate that + ; security data in the reply is optional. + 335 [ADAT=base64data] + ; This reply indicates that the security data is + ; acceptable, and more is required to complete the + ; security data exchange. The square brackets + ; are not to be included in the reply, but indicate + ; that security data in the reply is optional. + + + + +Horowitz & Lunt Standards Track [Page 12] + +RFC 2228 FTP Security Extensions October 1997 + + + 336 Username okay, need password. Challenge is "...." + ; The exact representation of the challenge should be chosen + ; by the mechanism to be sensible to the human user of the + ; system. + + 431 Need some unavailable resource to process security. + + 533 Command protection level denied for policy reasons. + 534 Request denied for policy reasons. + 535 Failed security check (hash, sequence, etc). + 536 Requested PROT level not supported by mechanism. + 537 Command protection level not supported by security mechanism. + + 5.2. Protected replies. + + One new reply type is introduced: + + 6yz Protected reply + + There are three reply codes of this type. The first, reply + code 631 indicates an integrity protected reply. The + second, reply code 632, indicates a confidentiality and + integrity protected reply. the third, reply code 633, + indicates a confidentiality protected reply. + + The text part of a 631 reply is a Telnet string consisting + of a base 64 encoded "safe" message produced by a security + mechanism specific message integrity procedure. The text + part of a 632 reply is a Telnet string consisting of a base + 64 encoded "private" message produced by a security + mechanism specific message confidentiality and integrity + procedure. The text part of a 633 reply is a Telnet string + consisting of a base 64 encoded "confidential" message + produced by a security mechanism specific message + confidentiality procedure. + + The client will decode and verify the encoded reply. How + failures decoding or verifying replies are handled is + implementation-specific. An end-of-line code need not be + included, but if one is included, it must be a Telnet end- + of-line code, not a local end-of-line code. + + A protected reply may only be sent if a security data + exchange has succeeded. + + The 63z reply may be a multiline reply. In this case, the + plaintext reply must be broken up into a number of + fragments. Each fragment must be protected, then base 64 + + + +Horowitz & Lunt Standards Track [Page 13] + +RFC 2228 FTP Security Extensions October 1997 + + + encoded in order into a separate line of the multiline + reply. There need not be any correspondence between the + line breaks in the plaintext reply and the encoded reply. + Telnet end-of-line codes must appear in the plaintext of the + encoded reply, except for the final end-of-line code, which + is optional. + + The multiline reply must be formatted more strictly than the + continuation specification in RFC 959. In particular, each + line before the last must be formed by the reply code, + followed immediately by a hyphen, followed by a base 64 + encoded fragment of the reply. + + For example, if the plaintext reply is + + 123-First line + Second line + 234 A line beginning with numbers + 123 The last line + + then the resulting protected reply could be any of the + following (the first example has a line break only to fit + within the margins): + + 631 base64(protect("123-First line\r\nSecond line\r\n 234 A line + 631-base64(protect("123-First line\r\n")) + 631-base64(protect("Second line\r\n")) + 631-base64(protect(" 234 A line beginning with numbers\r\n")) + 631 base64(protect("123 The last line")) + + 631-base64(protect("123-First line\r\nSecond line\r\n 234 A line b")) + 631 base64(protect("eginning with numbers\r\n123 The last line\r\n")) + +6. Data Channel Encapsulation + + When data transfers are protected between the client and server (in + either direction), certain transformations and encapsulations must be + performed so that the recipient can properly decode the transmitted + file. + + The sender must apply all protection services after transformations + associated with the representation type, file structure, and transfer + mode have been performed. The data sent over the data channel is, + for the purposes of protection, to be treated as a byte stream. + + When performing a data transfer in an authenticated manner, the + authentication checks are performed on individual blocks of the file, + rather than on the file as a whole. Consequently, it is possible for + + + +Horowitz & Lunt Standards Track [Page 14] + +RFC 2228 FTP Security Extensions October 1997 + + + insertion attacks to insert blocks into the data stream (i.e., + replays) that authenticate correctly, but result in a corrupted file + being undetected by the receiver. To guard against such attacks, the + specific security mechanism employed should include mechanisms to + protect against such attacks. Many GSS-API mechanisms usable with + the specification in Appendix I, and the Kerberos mechanism in + Appendix II do so. + + The sender must take the input byte stream, and break it up into + blocks such that each block, when encoded using a security mechanism + specific procedure, will be no larger than the buffer size negotiated + by the client with the PBSZ command. Each block must be encoded, + then transmitted with the length of the encoded block prepended as a + four byte unsigned integer, most significant byte first. + + When the end of the file is reached, the sender must encode a block + of zero bytes, and send this final block to the recipient before + closing the data connection. + + The recipient will read the four byte length, read a block of data + that many bytes long, then decode and verify this block with a + security mechanism specific procedure. This must be repeated until a + block encoding a buffer of zero bytes is received. This indicates + the end of the encoded byte stream. + + Any transformations associated with the representation type, file + structure, and transfer mode are to be performed by the recipient on + the byte stream resulting from the above process. + + When using block transfer mode, the sender's (cleartext) buffer size + is independent of the block size. + + The server will reply 534 to a STOR, STOU, RETR, LIST, NLST, or APPE + command if the current protection level is not at the level dictated + by the server's security requirements for the particular file + transfer. + + If any data protection services fail at any time during data transfer + at the server end (including an attempt to send a buffer size greater + than the negotiated maximum), the server will send a 535 reply to the + data transfer command (either STOR, STOU, RETR, LIST, NLST, or APPE). + + + + + + + + + + +Horowitz & Lunt Standards Track [Page 15] + +RFC 2228 FTP Security Extensions October 1997 + + +7. Potential policy considerations + + While there are no restrictions on client and server policy, there + are a few recommendations which an implementation should implement. + + - Once a security data exchange takes place, a server should require + all commands be protected (with integrity and/or confidentiality), + and it should protect all replies. Replies should use the same + level of protection as the command which produced them. This + includes replies which indicate failure of the MIC, CONF, and ENC + commands. In particular, it is not meaningful to require that + AUTH and ADAT be protected; it is meaningful and useful to require + that PROT and PBSZ be protected. In particular, the use of CCC is + not recommended, but is defined in the interest of + interoperability between implementations which might desire such + functionality. + + - A client should encrypt the PASS command whenever possible. It is + reasonable for the server to refuse to accept a non-encrypted PASS + command if the server knows encryption is available. + + - Although no security commands are required to be implemented, it + is recommended that an implementation provide all commands which + can be implemented, given the mechanisms supported and the policy + considerations of the site (export controls, for instance). + +8. Declarative specifications + + These sections are modelled after sections 5.3 and 5.4 of RFC 959, + which describe the same information, except for the standard FTP + commands and replies. + + 8.1. FTP Security commands and arguments + + AUTH + ADAT + PROT + PBSZ + MIC + CONF + ENC + + ::= + ::= + ; must be formatted as described in section 9 + ::= C | S | E | P + ::= any decimal integer from 1 to (2^32)-1 + + + + +Horowitz & Lunt Standards Track [Page 16] + +RFC 2228 FTP Security Extensions October 1997 + + + 8.2. Command-Reply sequences + + Security Association Setup + AUTH + 234 + 334 + 502, 504, 534, 431 + 500, 501, 421 + ADAT + 235 + 335 + 503, 501, 535 + 500, 501, 421 + Data protection negotiation commands + PBSZ + 200 + 503 + 500, 501, 421, 530 + PROT + 200 + 504, 536, 503, 534, 431 + 500, 501, 421, 530 + Command channel protection commands + MIC + 535, 533 + 500, 501, 421 + CONF + 535, 533 + 500, 501, 421 + ENC + 535, 533 + 500, 501, 421 + Security-Enhanced login commands (only new replies listed) + USER + 232 + 336 + Data channel commands (only new replies listed) + STOR + 534, 535 + STOU + 534, 535 + RETR + 534, 535 + + + + + + + + +Horowitz & Lunt Standards Track [Page 17] + +RFC 2228 FTP Security Extensions October 1997 + + + LIST + 534, 535 + NLST + 534, 535 + APPE + 534, 535 + + In addition to these reply codes, any security command can return + 500, 501, 502, 533, or 421. Any ftp command can return a reply + code encapsulated in a 631, 632, or 633 reply once a security data + exchange has completed successfully. + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +Horowitz & Lunt Standards Track [Page 18] + +RFC 2228 FTP Security Extensions October 1997 + + +9. State Diagrams + + This section includes a state diagram which demonstrates the flow of + authentication and authorization in a security enhanced FTP + implementation. The rectangular blocks show states where the client + must issue a command, and the diamond blocks show states where the + server must issue a response. + + + ,------------------, USER + __\| Unauthenticated |_________\ + | /| (new connection) | /| + | `------------------' | + | | | + | | AUTH | + | V | + | / \ | + | 4yz,5yz / \ 234 | + |<--------< >------------->. | + | \ / | | + | \_/ | | + | | | | + | | 334 | | + | V | | + | ,--------------------, | | + | | Need Security Data |<--. | | + | `--------------------' | | | + | | | | | + | | ADAT | | | + | V | | | + | / \ | | | + | 4yz,5yz / \ 335 | | | + `<--------< >-----------' | | + \ / | | + \_/ | | + | | | + | 235 | | + V | | + ,---------------. | | + ,--->| Authenticated |<--------' | After the client and server + | `---------------' | have completed authenti- + | | | cation, command must be + | | USER | integrity-protected if + | | | integrity is available. The + | |<-------------------' CCC command may be issued to + | V relax this restriction. + + + + + +Horowitz & Lunt Standards Track [Page 19] + +RFC 2228 FTP Security Extensions October 1997 + + + | / \ + | 4yz,5yz / \ 2yz + |<--------< >------------->. + | \ / | + | \_/ | + | | | + | | 3yz | + | V | + | ,---------------. | + | | Need Password | | + | `---------------' | + | | | + | | PASS | + | V | + | / \ | + | 4yz,5yz / \ 2yz | + |<--------< >------------->| + | \ / | + | \_/ | + | | | + | | 3yz | + | V | + | ,--------------. | + | | Need Account | | + | `--------------' | + | | | + | | ACCT | + | V | + | / \ | + | 4yz,5yz / \ 2yz | + `<--------< >------------->| + \ / | + \_/ | + | | + | 3yz | + V | + ,-------------. | + | Authorized |/________| + | (Logged in) |\ + `-------------' + + + + + + + + + + + +Horowitz & Lunt Standards Track [Page 20] + +RFC 2228 FTP Security Extensions October 1997 + + +10. Base 64 Encoding + + Base 64 encoding is the same as the Printable Encoding described in + Section 4.3.2.4 of [RFC-1421], except that line breaks must not be + included. This encoding is defined as follows. + + Proceeding from left to right, the bit string resulting from the + mechanism specific protection routine is encoded into characters + which are universally representable at all sites, though not + necessarily with the same bit patterns (e.g., although the character + "E" is represented in an ASCII-based system as hexadecimal 45 and as + hexadecimal C5 in an EBCDIC-based system, the local significance of + the two representations is equivalent). + + A 64-character subset of International Alphabet IA5 is used, enabling + 6 bits to be represented per printable character. (The proposed + subset of characters is represented identically in IA5 and ASCII.) + The character "=" signifies a special processing function used for + padding within the printable encoding procedure. + + The encoding process represents 24-bit groups of input bits as output + strings of 4 encoded characters. Proceeding from left to right + across a 24-bit input group output from the security mechanism + specific message protection procedure, each 6-bit group is used as an + index into an array of 64 printable characters, namely "[A-Z][a- + z][0-9]+/". The character referenced by the index is placed in the + output string. These characters are selected so as to be universally + representable, and the set excludes characters with particular + significance to Telnet (e.g., "", "", IAC). + + Special processing is performed if fewer than 24 bits are available + in an input group at the end of a message. A full encoding quantum + is always completed at the end of a message. When fewer than 24 + input bits are available in an input group, zero bits are added (on + the right) to form an integral number of 6-bit groups. Output + character positions which are not required to represent actual input + data are set to the character "=". Since all canonically encoded + output is an integral number of octets, only the following cases can + arise: (1) the final quantum of encoding input is an integral + multiple of 24 bits; here, the final unit of encoded output will be + an integral multiple of 4 characters with no "=" padding, (2) the + final quantum of encoding input is exactly 8 bits; here, the final + unit of encoded output will be two characters followed by two "=" + padding characters, or (3) the final quantum of encoding input is + exactly 16 bits; here, the final unit of encoded output will be three + characters followed by one "=" padding character. + + + + + +Horowitz & Lunt Standards Track [Page 21] + +RFC 2228 FTP Security Extensions October 1997 + + + Implementors must keep in mind that the base 64 encodings in ADAT, + MIC, CONF, and ENC commands, and in 63z replies may be arbitrarily + long. Thus, the entire line must be read before it can be processed. + Several successive reads on the control channel may be necessary. It + is not appropriate to for a server to reject a command containing a + base 64 encoding simply because it is too long (assuming that the + decoding is otherwise well formed in the context in which it was + sent). + + Case must not be ignored when reading commands and replies containing + base 64 encodings. + +11. Security Considerations + + This entire document deals with security considerations related to + the File Transfer Protocol. + + Third party file transfers cannot be secured using these extensions, + since a security context cannot be established between two servers + using these facilities (no control connection exists between servers + over which to pass ADAT tokens). Further work in this area is + deferred. + +12. Acknowledgements + + I would like to thank the members of the CAT WG, as well as all + participants in discussions on the "cat-ietf@mit.edu" mailing list, + for their contributions to this document. I would especially like to + thank Sam Sjogren, John Linn, Ted Ts'o, Jordan Brown, Michael Kogut, + Derrick Brashear, John Gardiner Myers, Denis Pinkas, and Karri Balk + for their contributions to this work. Of course, without Steve Lunt, + the author of the first six revisions of this document, it would not + exist at all. + +13. References + + [TELNET-SEC] Borman, D., "Telnet Authentication and Encryption + Option", Work in Progress. + + [RFC-1123] Braden, R., "Requirements for Internet Hosts -- + Application and Support", STD 3, RFC 1123, October 1989. + + [RFC-1421] Linn, J., "Privacy Enhancement for Internet Electronic + Mail: Part I: Message Encryption and Authentication Procedures", + RFC 1421, February 1993. + + + + + + +Horowitz & Lunt Standards Track [Page 22] + +RFC 2228 FTP Security Extensions October 1997 + + +14. Author's Address + + Marc Horowitz + Cygnus Solutions + 955 Massachusetts Avenue + Cambridge, MA 02139 + + Phone: +1 617 354 7688 + EMail: marc@cygnus.com + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +Horowitz & Lunt Standards Track [Page 23] + +RFC 2228 FTP Security Extensions October 1997 + + +Appendix I: Specification under the GSSAPI + + In order to maximise the utility of new security mechanisms, it is + desirable that new mechanisms be implemented as GSSAPI mechanisms + rather than as FTP security mechanisms. This will enable existing + ftp implementations to support the new mechanisms more easily, since + little or no code will need to be changed. In addition, the + mechanism will be usable by other protocols, such as IMAP, which are + built on top of the GSSAPI, with no additional specification or + implementation work needed by the mechanism designers. + + The security mechanism name (for the AUTH command) associated with + all mechanisms employing the GSSAPI is GSSAPI. If the server + supports a security mechanism employing the GSSAPI, it must respond + with a 334 reply code indicating that an ADAT command is expected + next. + + The client must begin the authentication exchange by calling + GSS_Init_Sec_Context, passing in 0 for input_context_handle + (initially), and a targ_name equal to output_name from + GSS_Import_Name called with input_name_type of Host-Based Service and + input_name_string of "ftp@hostname" where "hostname" is the fully + qualified host name of the server with all letters in lower case. + (Failing this, the client may try again using input_name_string of + "host@hostname".) The output_token must then be base 64 encoded and + sent to the server as the argument to an ADAT command. If + GSS_Init_Sec_Context returns GSS_S_CONTINUE_NEEDED, then the client + must expect a token to be returned in the reply to the ADAT command. + This token must subsequently be passed to another call to + GSS_Init_Sec_Context. In this case, if GSS_Init_Sec_Context returns + no output_token, then the reply code from the server for the previous + ADAT command must have been 235. If GSS_Init_Sec_Context returns + GSS_S_COMPLETE, then no further tokens are expected from the server, + and the client must consider the server authenticated. + + The server must base 64 decode the argument to the ADAT command and + pass the resultant token to GSS_Accept_Sec_Context as input_token, + setting acceptor_cred_handle to NULL (for "use default credentials"), + and 0 for input_context_handle (initially). If an output_token is + returned, it must be base 64 encoded and returned to the client by + including "ADAT=base64string" in the text of the reply. If + GSS_Accept_Sec_Context returns GSS_S_COMPLETE, the reply code must be + 235, and the server must consider the client authenticated. If + GSS_Accept_Sec_Context returns GSS_S_CONTINUE_NEEDED, the reply code + must be 335. Otherwise, the reply code should be 535, and the text + of the reply should contain a descriptive error message. + + + + + +Horowitz & Lunt Standards Track [Page 24] + +RFC 2228 FTP Security Extensions October 1997 + + + The chan_bindings input to GSS_Init_Sec_Context and + GSS_Accept_Sec_Context should use the client internet address and + server internet address as the initiator and acceptor addresses, + respectively. The address type for both should be GSS_C_AF_INET. No + application data should be specified. + + Since GSSAPI supports anonymous peers to security contexts, it is + possible that the client's authentication of the server does not + actually establish an identity. + + The procedure associated with MIC commands, 631 replies, and Safe + file transfers is: + + GSS_Wrap for the sender, with conf_flag == FALSE + + GSS_Unwrap for the receiver + + The procedure associated with ENC commands, 632 replies, and Private + file transfers is: + + GSS_Wrap for the sender, with conf_flag == TRUE + GSS_Unwrap for the receiver + + CONF commands and 633 replies are not supported. + + Both the client and server should inspect the value of conf_avail to + determine whether the peer supports confidentiality services. + + When the security state is reset (when AUTH is received a second + time, or when REIN is received), this should be done by calling the + GSS_Delete_sec_context function. + +Appendix II: Specification under Kerberos version 4 + + The security mechanism name (for the AUTH command) associated with + Kerberos Version 4 is KERBEROS_V4. If the server supports + KERBEROS_V4, it must respond with a 334 reply code indicating that an + ADAT command is expected next. + + The client must retrieve a ticket for the Kerberos principal + "ftp.hostname@realm" by calling krb_mk_req(3) with a principal name + of "ftp", an instance equal to the first part of the canonical host + name of the server with all letters in lower case (as returned by + krb_get_phost(3)), the server's realm name (as returned by + krb_realmofhost(3)), and an arbitrary checksum. The ticket must then + be base 64 encoded and sent as the argument to an ADAT command. + + + + + +Horowitz & Lunt Standards Track [Page 25] + +RFC 2228 FTP Security Extensions October 1997 + + + If the "ftp" principal name is not a registered principal in the + Kerberos database, then the client may fall back on the "rcmd" + principal name (same instance and realm). However, servers must + accept only one or the other of these principal names, and must not + be willing to accept either. Generally, if the server has a key for + the "ftp" principal in its srvtab, then that principal only must be + used, otherwise the "rcmd" principal only must be used. + + The server must base 64 decode the argument to the ADAT command and + pass the result to krb_rd_req(3). The server must add one to the + checksum from the authenticator, convert the result to network byte + order (most significant byte first), and sign it using + krb_mk_safe(3), and base 64 encode the result. Upon success, the + server must reply to the client with a 235 code and include + "ADAT=base64string" in the text of the reply. Upon failure, the + server should reply 535. + + Upon receipt of the 235 reply from the server, the client must parse + the text of the reply for the base 64 encoded data, decode it, + convert it from network byte order, and pass the result to + krb_rd_safe(3). The client must consider the server authenticated if + the resultant checksum is equal to one plus the value previously + sent. + + The procedure associated with MIC commands, 631 replies, and Safe + file transfers is: + + krb_mk_safe(3) for the sender + krb_rd_safe(3) for the receiver + + The procedure associated with ENC commands, 632 replies, and Private + file transfers is: + + krb_mk_priv(3) for the sender + krb_rd_priv(3) for the receiver + + CONF commands and 633 replies are not supported. + + Note that this specification for KERBEROS_V4 contains no provision + for negotiating alternate means for integrity and confidentiality + routines. Note also that the ADAT exchange does not convey whether + the peer supports confidentiality services. + + In order to stay within the allowed PBSZ, implementors must take note + that a cleartext buffer will grow by 31 bytes when processed by + krb_mk_safe(3) and will grow by 26 bytes when processed by + krb_mk_priv(3). + + + + +Horowitz & Lunt Standards Track [Page 26] + +RFC 2228 FTP Security Extensions October 1997 + + +Full Copyright Statement + + Copyright (C) The Internet Society (1997). All Rights Reserved. + + This document and translations of it may be copied and furnished to + others, and derivative works that comment on or otherwise explain it + or assist in its implmentation may be prepared, copied, published + andand distributed, in whole or in part, without restriction of any + kind, provided that the above copyright notice and this paragraph are + included on all such copies and derivative works. However, this + document itself may not be modified in any way, such as by removing + the copyright notice or references to the Internet Society or other + Internet organizations, except as needed for the purpose of + developing Internet standards in which case the procedures for + copyrights defined in the Internet Standards process must be + followed, or as required to translate it into languages other than + English. + + The limited permissions granted above are perpetual and will not be + revoked by the Internet Society or its successors or assigns. + + This document and the information contained herein is provided on an + "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING + TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING + BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION + HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF + MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. + + + + + + + + + + + + + + + + + + + + + + + + +Horowitz & Lunt Standards Track [Page 27] + diff --git a/doc/rfc2640.txt b/doc/rfc2640.txt new file mode 100644 index 0000000..73ff879 --- /dev/null +++ b/doc/rfc2640.txt @@ -0,0 +1,1515 @@ + + + + + + +Network Working Group B. Curtin +Request for Comments: 2640 Defense Information Systems Agency +Updates: 959 July 1999 +Category: Proposed Standard + + + Internationalization of the File Transfer Protocol + +Status of this Memo + + This document specifies an Internet standards track protocol for the + Internet community, and requests discussion and suggestions for + improvements. Please refer to the current edition of the "Internet + Official Protocol Standards" (STD 1) for the standardization state + and status of this protocol. Distribution of this memo is unlimited. + +Copyright Notice + + Copyright (C) The Internet Society (1999). All Rights Reserved. + +Abstract + + The File Transfer Protocol, as defined in RFC 959 [RFC959] and RFC + 1123 Section 4 [RFC1123], is one of the oldest and widely used + protocols on the Internet. The protocol's primary character set, 7 + bit ASCII, has served the protocol well through the early growth + years of the Internet. However, as the Internet becomes more global, + there is a need to support character sets beyond 7 bit ASCII. + + This document addresses the internationalization (I18n) of FTP, which + includes supporting the multiple character sets and languages found + throughout the Internet community. This is achieved by extending the + FTP specification and giving recommendations for proper + internationalization support. + +Table of Contents + + ABSTRACT.......................................................1 + 1 INTRODUCTION.................................................2 + 1.1 Requirements Terminology..................................2 + 2 INTERNATIONALIZATION.........................................3 + 2.1 International Character Set...............................3 + 2.2 Transfer Encoding Set.....................................4 + 3 PATHNAMES....................................................5 + 3.1 General compliance........................................5 + 3.2 Servers compliance........................................6 + 3.3 Clients compliance........................................7 + 4 LANGUAGE SUPPORT.............................................7 + + + +Curtin Proposed Standard [Page 1] + +RFC 2640 FTP Internalization July 1999 + + + 4.1 The LANG command..........................................8 + 4.2 Syntax of the LANG command................................9 + 4.3 Feat response for LANG command...........................11 + 4.3.1 Feat examples.........................................11 + 5 SECURITY CONSIDERATIONS.....................................12 + 6 ACKNOWLEDGMENTS.............................................12 + 7 GLOSSARY....................................................13 + 8 BIBLIOGRAPHY................................................13 + 9 AUTHOR'S ADDRESS............................................15 + ANNEX A - IMPLEMENTATION CONSIDERATIONS.......................16 + A.1 General Considerations...................................16 + A.2 Transition Considerations................................18 + ANNEX B - SAMPLE CODE AND EXAMPLES............................19 + B.1 Valid UTF-8 check........................................19 + B.2 Conversions..............................................20 + B.2.1 Conversion from Local Character Set to UTF-8..........20 + B.2.2 Conversion from UTF-8 to Local Character Set..........23 + B.2.3 ISO/IEC 8859-8 Example................................25 + B.2.4 Vendor Codepage Example...............................25 + B.3 Pseudo Code for Translating Servers......................26 + Full Copyright Statement......................................27 + +1 Introduction + + As the Internet grows throughout the world the requirement to support + character sets outside of the ASCII [ASCII] / Latin-1 [ISO-8859] + character set becomes ever more urgent. For FTP, because of the + large installed base, it is paramount that this is done without + breaking existing clients and servers. This document addresses this + need. In doing so it defines a solution which will still allow the + installed base to interoperate with new clients and servers. + + This document enhances the capabilities of the File Transfer Protocol + by removing the 7-bit restrictions on pathnames used in client + commands and server responses, RECOMMENDs the use of a Universal + Character Set (UCS) ISO/IEC 10646 [ISO-10646], RECOMMENDs a UCS + transformation format (UTF) UTF-8 [UTF-8], and defines a new command + for language negotiation. + + The recommendations made in this document are consistent with the + recommendations expressed by the IETF policy related to character + sets and languages as defined in RFC 2277 [RFC2277]. + +1.1. Requirements Terminology + + The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", + "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this + document are to be interpreted as described in BCP 14 [BCP14]. + + + +Curtin Proposed Standard [Page 2] + +RFC 2640 FTP Internalization July 1999 + + +2 Internationalization + + The File Transfer Protocol was developed when the predominate + character sets were 7 bit ASCII and 8 bit EBCDIC. Today these + character sets cannot support the wide range of characters needed by + multinational systems. Given that there are a number of character + sets in current use that provide more characters than 7-bit ASCII, it + makes sense to decide on a convenient way to represent the union of + those possibilities. To work globally either requires support of a + number of character sets and to be able to convert between them, or + the use of a single preferred character set. To assure global + interoperability this document RECOMMENDS the latter approach and + defines a single character set, in addition to NVT ASCII and EBCDIC, + which is understandable by all systems. For FTP this character set + SHALL be ISO/IEC 10646:1993. For support of global compatibility it + is STRONGLY RECOMMENDED that clients and servers use UTF-8 encoding + when exchanging pathnames. Clients and servers are, however, under + no obligation to perform any conversion on the contents of a file for + operations such as STOR or RETR. + + The character set used to store files SHALL remain a local decision + and MAY depend on the capability of local operating systems. Prior to + the exchange of pathnames they SHOULD be converted into a ISO/IEC + 10646 format and UTF-8 encoded. This approach, while allowing + international exchange of pathnames, will still allow backward + compatibility with older systems because the code set positions for + ASCII characters are identical to the one byte sequence in UTF-8. + + Sections 2.1 and 2.2 give a brief description of the international + character set and transfer encoding RECOMMENDED by this document. A + more thorough description of UTF-8, ISO/IEC 10646, and UNICODE + [UNICODE], beyond that given in this document, can be found in RFC + 2279 [RFC2279]. + +2.1 International Character Set + + The character set defined for international support of FTP SHALL be + the Universal Character Set as defined in ISO 10646:1993 as amended. + This standard incorporates the character sets of many existing + international, national, and corporate standards. ISO/IEC 10646 + defines two alternate forms of encoding, UCS-4 and UCS-2. UCS-4 is a + four byte (31 bit) encoding containing 2**31 code positions divided + into 128 groups of 256 planes. Each plane consists of 256 rows of 256 + cells. UCS-2 is a 2 byte (16 bit) character set consisting of plane + zero or the Basic Multilingual Plane (BMP). Currently, no codesets + have been defined outside of the 2 byte BMP. + + + + + +Curtin Proposed Standard [Page 3] + +RFC 2640 FTP Internalization July 1999 + + + The Unicode standard version 2.0 [UNICODE] is consistent with the + UCS-2 subset of ISO/IEC 10646. The Unicode standard version 2.0 + includes the repertoire of IS 10646 characters, amendments 1-7 of IS + 10646, and editorial and technical corrigenda. + +2.2 Transfer Encoding + + UCS Transformation Format 8 (UTF-8), in the past referred to as UTF-2 + or UTF-FSS, SHALL be used as a transfer encoding to transmit the + international character set. UTF-8 is a file safe encoding which + avoids the use of byte values that have special significance during + the parsing of pathname character strings. UTF-8 is an 8 bit encoding + of the characters in the UCS. Some of UTF-8's benefits are that it is + compatible with 7 bit ASCII, so it doesn't affect programs that give + special meanings to various ASCII characters; it is immune to + synchronization errors; its encoding rules allow for easy + identification; and it has enough space to support a large number of + character sets. + + UTF-8 encoding represents each UCS character as a sequence of 1 to 6 + bytes in length. For all sequences of one byte the most significant + bit is ZERO. For all sequences of more than one byte the number of + ONE bits in the first byte, starting from the most significant bit + position, indicates the number of bytes in the UTF-8 sequence + followed by a ZERO bit. For example, the first byte of a 3 byte UTF-8 + sequence would have 1110 as its most significant bits. Each + additional bytes (continuing bytes) in the UTF-8 sequence, contain a + ONE bit followed by a ZERO bit as their most significant bits. The + remaining free bit positions in the continuing bytes are used to + identify characters in the UCS. The relationship between UCS and + UTF-8 is demonstrated in the following table: + + UCS-4 range(hex) UTF-8 byte sequence(binary) + 00000000 - 0000007F 0xxxxxxx + 00000080 - 000007FF 110xxxxx 10xxxxxx + 00000800 - 0000FFFF 1110xxxx 10xxxxxx 10xxxxxx + 00010000 - 001FFFFF 11110xxx 10xxxxxx 10xxxxxx 10xxxxxx + 00200000 - 03FFFFFF 111110xx 10xxxxxx 10xxxxxx 10xxxxxx + 10xxxxxx + 04000000 - 7FFFFFFF 1111110x 10xxxxxx 10xxxxxx 10xxxxxx + 10xxxxxx 10xxxxxx + + A beneficial property of UTF-8 is that its single byte sequence is + consistent with the ASCII character set. This feature will allow a + transition where old ASCII-only clients can still interoperate with + new servers that support the UTF-8 encoding. + + + + + +Curtin Proposed Standard [Page 4] + +RFC 2640 FTP Internalization July 1999 + + + Another feature is that the encoding rules make it very unlikely that + a character sequence from a different character set will be mistaken + for a UTF-8 encoded character sequence. Clients and servers can use a + simple routine to determine if the character set being exchanged is + valid UTF-8. Section B.1 shows a code example of this check. + +3 Pathnames + +3.1 General compliance + + - The 7-bit restriction for pathnames exchanged is dropped. + + - Many operating system allow the use of spaces , carriage return + , and line feed characters as part of the pathname. The + exchange of pathnames with these special command characters will + cause the pathnames to be parsed improperly. This is because ftp + commands associated with pathnames have the form: + + COMMAND . + + To allow the exchange of pathnames containing these characters, the + definition of pathname is changed from + + ::= ; in BNF format + to + pathname = 1*(%x01..%xFF) ; in ABNF format [ABNF]. + + To avoid mistaking these characters within pathnames as special + command characters the following rules will apply: + + There MUST be only one between a ftp command and the pathname. + Implementations MUST assume characters following the initial + as part of the pathname. For example the pathname in STOR + foo.bar is foo.bar. + + Current implementations, which may allow multiple characters as + separators between the command and pathname, MUST assure that they + comply with this single convention. Note: Implementations which + treat 3 character commands (e.g. CWD, MKD, etc.) as a fixed 4 + character command by padding the command with a trailing are in + non-compliance to this specification. + + When a character is encountered as part of a pathname it MUST be + padded with a character prior to sending the command. On + receipt of a pathname containing a sequence the + character MUST be stripped away. This approach is described in the + Telnet protocol [RFC854] on pages 11 and 12. For example, to store a + pathname fooboo.bar the pathname would become + + + +Curtin Proposed Standard [Page 5] + +RFC 2640 FTP Internalization July 1999 + + + fooboo.bar prior to sending the command STOR + fooboo.bar. Upon receipt of the altered + pathname the character following the would be stripped + away to form the original pathname. + + - Conforming clients and servers MUST support UTF-8 for the transfer + and receipt of pathnames. Clients and servers MAY in addition give + users a choice of specifying interpretation of pathnames in another + encoding. Note that configuring clients and servers to use + character sets / encoding other than UTF-8 is outside of the scope + of this document. While it is recognized that in certain + operational scenarios this may be desirable, this is left as a + quality of implementation and operational issue. + + - Pathnames are sequences of bytes. The encoding of names that are + valid UTF-8 sequences is assumed to be UTF-8. The character set of + other names is undefined. Clients and servers, unless otherwise + configured to support a specific native character set, MUST check + for a valid UTF-8 byte sequence to determine if the pathname being + presented is UTF-8. + + - To avoid data loss, clients and servers SHOULD use the UTF-8 + encoded pathnames when unable to convert them to a usable code set. + + - There may be cases when the code set / encoding presented to the + server or client cannot be determined. In such cases the raw bytes + SHOULD be used. + +3.2 Servers compliance + + - Servers MUST support the UTF-8 feature in response to the FEAT + command [RFC2389]. The UTF-8 feature is a line containing the exact + string "UTF8". This string is not case sensitive, but SHOULD be + transmitted in upper case. The response to a FEAT command SHOULD + be: + + C> feat + S> 211- + S> ... + S> UTF8 + S> ... + S> 211 end + + The ellipses indicate placeholders where other features may be + included, but are NOT REQUIRED. The one space indentation of the + feature lines is mandatory [RFC2389]. + + + + + +Curtin Proposed Standard [Page 6] + +RFC 2640 FTP Internalization July 1999 + + + - Mirror servers may want to exactly reflect the site that they are + mirroring. In such cases servers MAY store and present the exact + pathname bytes that it received from the main server. + +3.3 Clients compliance + + - Clients which do not require display of pathnames are under no + obligation to do so. Non-display clients do not need to conform to + requirements associated with display. + + - Clients, which are presented UTF-8 pathnames by the server, SHOULD + parse UTF-8 correctly and attempt to display the pathname within + the limitation of the resources available. + + - Clients MUST support the FEAT command and recognize the "UTF8" + feature (defined in 3.2 above) to determine if a server supports + UTF-8 encoding. + + - Character semantics of other names shall remain undefined. If a + client detects that a server is non UTF-8, it SHOULD change its + display appropriately. How a client implementation handles non + UTF-8 is a quality of implementation issue. It MAY try to assume + some other encoding, give the user a chance to try to assume + something, or save encoding assumptions for a server from one FTP + session to another. + + - Glyph rendering is outside the scope of this document. How a client + presents characters it cannot display is a quality of + implementation issue. This document RECOMMENDS that octets + corresponding to non-displayable characters SHOULD be presented in + URL %HH format defined in RFC 1738 [RFC1738]. They MAY, however, + display them as question marks, with their UCS hexadecimal value, + or in any other suitable fashion. + + - Many existing clients interpret 8-bit pathnames as being in the + local character set. They MAY continue to do so for pathnames that + are not valid UTF-8. + +4. Language Support + + The Character Set Workshop Report [RFC2130] suggests that clients and + servers SHOULD negotiate a language for "greetings" and "error + messages". This specification interprets the use of the term "error + message", by RFC 2130, to mean any explanatory text string returned + by server-PI in response to a user-PI command. + + + + + + +Curtin Proposed Standard [Page 7] + +RFC 2640 FTP Internalization July 1999 + + + Implementers SHOULD note that FTP commands and numeric responses are + protocol elements. As such, their use is not affected by any guidance + expressed by this specification. + + Language support of greetings and command responses shall be the + default language supported by the server or the language supported by + the server and selected by the client. + + It may be possible to achieve language support through a virtual host + as described in [MLST]. However, an FTP server might not support + virtual servers, or virtual servers might be configured to support an + environment without regard for language. To allow language + negotiation this specification defines a new LANG command. Clients + and servers that comply with this specification MUST support the LANG + command. + +4.1 The LANG command + + A new command "LANG" is added to the FTP command set to allow + server-FTP process to determine in which language to present server + greetings and the textual part of command responses. The parameter + associated with the LANG command SHALL be one of the language tags + defined in RFC 1766 [RFC1766]. If a LANG command without a parameter + is issued the server's default language will be used. + + Greetings and responses issued prior to language negotiation SHALL be + in the server's default language. Paragraph 4.5 of [RFC2277] state + that this "default language MUST be understandable by an English- + speaking person". This specification RECOMMENDS that the server + default language be English encoded using ASCII. This text may be + augmented by text from other languages. Once negotiated, server-PI + MUST return server messages and textual part of command responses in + the negotiated language and encoded in UTF-8. Server-PI MAY wish to + re-send previously issued server messages in the newly negotiated + language. + + The LANG command only affects presentation of greeting messages and + explanatory text associated with command responses. No attempt should + be made by the server to translate protocol elements (FTP commands + and numeric responses) or data transmitted over the data connection. + + User-PI MAY issue the LANG command at any time during an FTP session. + In order to gain the full benefit of this command, it SHOULD be + presented prior to authentication. In general, it will be issued + after the HOST command [MLST]. Note that the issuance of a HOST or + + + + + + +Curtin Proposed Standard [Page 8] + +RFC 2640 FTP Internalization July 1999 + + + REIN command [RFC959] will negate the affect of the LANG command. + User-PI SHOULD be capable of supporting UTF-8 encoding for the + language negotiated. Guidance on interpretation and rendering of + UTF-8, defined in section 3, SHALL apply. + + Although NOT REQUIRED by this specification, a user-PI SHOULD issue a + FEAT command [RFC2389] prior to a LANG command. This will allow the + user-PI to determine if the server supports the LANG command and + which language options. + + In order to aid the server in identifying whether a connection has + been established with a client which conforms to this specification + or an older client, user-PI MUST send a HOST [MLST] and/or LANG + command prior to issuing any other command (other than FEAT + [RFC2389]). If user-PI issues a HOST command, and the server's + default language is acceptable, it need not issue a LANG command. + However, if the implementation does not support the HOST command, a + LANG command MUST be issued. Until server-PI is presented with either + a HOST or LANG command it SHOULD assume that the user-PI does not + comply with this specification. + +4.2 Syntax of the LANG command + + The LANG command is defined as follows: + + lang-command = "Lang" [(SP lang-tag)] CRLF + lang-tag = Primary-tag *( "-" Sub-tag) + Primary-tag = 1*8ALPHA + Sub-tag = 1*8ALPHA + + lang-response = lang-ok / error-response + lang-ok = "200" [SP *(%x00..%xFF) ] CRLF + error-response = command-unrecognized / bad-argument / + not-implemented / unsupported-parameter + command-unrecognized = "500" [SP *(%x01..%xFF) ] CRLF + bad-argument = "501" [SP *(%x01..%xFF) ] CRLF + not-implemented = "502" [SP *(%x01..%xFF) ] CRLF + unsupported-parameter = "504" [SP *(%x01..%xFF) ] CRLF + + The "lang" command word is case independent and may be specified in + any character case desired. Therefore "LANG", "lang", "Lang", and + "lAnG" are equivalent commands. + + The OPTIONAL "Lang-tag" given as a parameter specifies the primary + language tags and zero or more sub-tags as defined in [RFC1766]. As + described in [RFC1766] language tags are treated as case insensitive. + If omitted server-PI MUST use the server's default language. + + + + +Curtin Proposed Standard [Page 9] + +RFC 2640 FTP Internalization July 1999 + + + Server-FTP responds to the "Lang" command with either "lang-ok" or + "error-response". "lang-ok" MUST be sent if Server-FTP supports the + "Lang" command and can support some form of the "lang-tag". Support + SHOULD be as follows: + + - If server-FTP receives "Lang" with no parameters it SHOULD return + messages and command responses in the server default language. + + - If server-FTP receives "Lang" with only a primary tag argument + (e.g. en, fr, de, ja, zh, etc.), which it can support, it SHOULD + return messages and command responses in the language associated + with that primary tag. It is possible that server-FTP will only + support the primary tag when combined with a sub-tag (e.g. en-US, + en-UK, etc.). In such cases, server-FTP MAY determine the + appropriate variant to use during the session. How server-FTP makes + that determination is outside the scope of this specification. If + server-FTP cannot determine if a sub-tag variant is appropriate it + SHOULD return an "unsupported-parameter" (504) response. + + - If server-FTP receives "Lang" with a primary tag and sub-tag(s) + argument, which is implemented, it SHOULD return messages and + command responses in support of the language argument. It is + possible that server-FTP can support the primary tag of the "Lang" + argument but not the sub-tag(s). In such cases server-FTP MAY + return messages and command responses in the most appropriate + variant of the primary tag that has been implemented. How server- + FTP makes that determination is outside the scope of this + specification. If server-FTP cannot determine if a sub-tag variant + is appropriate it SHOULD return an "unsupported-parameter" (504) + response. + + For example if client-FTP sends a "LANG en-AU" command and server-FTP + has implemented language tags en-US and en-UK it may decide that the + most appropriate language tag is en-UK and return "200 en-AU not + supported. Language set to en-UK". The numeric response is a protocol + element and can not be changed. The associated string is for + illustrative purposes only. + + Clients and servers that conform to this specification MUST support + the LANG command. Clients SHOULD, however, anticipate receiving a 500 + or 502 command response, in cases where older or non-compliant + servers do not recognize or have not implemented the "Lang". A 501 + response SHOULD be sent if the argument to the "Lang" command is not + syntactically correct. A 504 response SHOULD be sent if the "Lang" + argument, while syntactically correct, is not implemented. As noted + above, an argument may be considered a lexicon match even though it + is not an exact syntax match. + + + + +Curtin Proposed Standard [Page 10] + +RFC 2640 FTP Internalization July 1999 + + +4.3 Feat response for LANG command + + A server-FTP process that supports the LANG command, and language + support for messages and command responses, MUST include in the + response to the FEAT command [RFC2389], a feature line indicating + that the LANG command is supported and a fact list of the supported + language tags. A response to a FEAT command SHALL be in the following + format: + + Lang-feat = SP "LANG" SP lang-fact CRLF + lang-fact = lang-tag ["*"] *(";" lang-tag ["*"]) + + lang-tag = Primary-tag *( "-" Sub-tag) + Primary-tag= 1*8ALPHA + Sub-tag = 1*8ALPHA + + The lang-feat response contains the string "LANG" followed by a + language fact. This string is not case sensitive, but SHOULD be + transmitted in upper case, as recommended in [RFC2389]. The initial + space shown in the Lang-feat response is REQUIRED by the FEAT + command. It MUST be a single space character. More or less space + characters are not permitted. The lang-fact SHALL include the lang- + tags which server-FTP can support. At least one lang-tag MUST be + included with the FEAT response. The lang-tag SHALL be in the form + described earlier in this document. The OPTIONAL asterisk, when + present, SHALL indicate the current lang-tag being used by server-FTP + for messages and responses. + +4.3.1 Feat examples + + C> feat + S> 211- + S> ... + S> LANG EN* + S> ... + S> 211 end + + In this example server-FTP can only support English, which is the + current language (as shown by the asterisk) being used by the server + for messages and command responses. + + C> feat + S> 211- + S> ... + S> LANG EN*;FR + S> ... + S> 211 end + + + + +Curtin Proposed Standard [Page 11] + +RFC 2640 FTP Internalization July 1999 + + + C> LANG fr + S> 200 Le response sera changez au francais + + C> feat + S> 211- + S> ... + S> LANG EN;FR* + S> ... + S> 211 end + + In this example server-FTP supports both English and French as shown + by the initial response to the FEAT command. The asterisk indicates + that English is the current language in use by server-FTP. After a + LANG command is issued to change the language to French, the FEAT + response shows French as the current language in use. + + In the above examples ellipses indicate placeholders where other + features may be included, but are NOT REQUIRED. + +5 Security Considerations + + This document addresses the support of character sets beyond 1 byte + and a new language negotiation command. Conformance to this document + should not induce a security risk. + +6 Acknowledgments + + The following people have contributed to this document: + + D. J. Bernstein + Martin J. Duerst + Mark Harris + Paul Hethmon + Alun Jones + Gregory Lundberg + James Matthews + Keith Moore + Sandra O'Donnell + Benjamin Riefenstahl + Stephen Tihor + + (and others from the FTPEXT working group) + + + + + + + + + +Curtin Proposed Standard [Page 12] + +RFC 2640 FTP Internalization July 1999 + + +7 Glossary + + BIDI - abbreviation for Bi-directional, a reference to mixed right- + to-left and left-to-right text. + + Character Set - a collection of characters used to represent textual + information in which each character has a numeric value + + Code Set - (see character set). + + Glyph - a character image represented on a display device. + + I18N - "I eighteen N", the first and last letters of the word + "internationalization" and the eighteen letters in between. + + UCS-2 - the ISO/IEC 10646 two octet Universal Character Set form. + + UCS-4 - the ISO/IEC 10646 four octet Universal Character Set form. + + UTF-8 - the UCS Transformation Format represented in 8 bits. + + TF-16 - A 16-bit format including the BMP (directly encoded) and + surrogate pairs to represent characters in planes 01-16; equivalent + to Unicode. + +8 Bibliography + + [ABNF] Crocker, D. and P. Overell, "Augmented BNF for Syntax + Specifications: ABNF", RFC 2234, November 1997. + + [ASCII] ANSI X3.4:1986 Coded Character Sets - 7 Bit American + National Standard Code for Information Interchange (7- + bit ASCII) + + [ISO-8859] ISO 8859. International standard -- Information + processing -- 8-bit single-byte coded graphic character + sets -- Part 1:Latin alphabet No. 1 (1987) -- Part 2: + Latin alphabet No. 2 (1987) -- Part 3: Latin alphabet + No. 3 (1988) -- Part 4: Latin alphabet No. 4 (1988) -- + Part 5: Latin/Cyrillic alphabet (1988) -- Part 6: + Latin/Arabic alphabet (1987) -- Part : Latin/Greek + alphabet (1987) -- Part 8: Latin/Hebrew alphabet (1988) + -- Part 9: Latin alphabet No. 5 (1989) -- Part10: Latin + alphabet No. 6 (1992) + + [BCP14] Bradner, S., "Key words for use in RFCs to Indicate + Requirement Levels", BCP 14, RFC 2119, March 1997. + + + + +Curtin Proposed Standard [Page 13] + +RFC 2640 FTP Internalization July 1999 + + + [ISO-10646] ISO/IEC 10646-1:1993. International standard -- + Information technology -- Universal multiple-octet coded + character set (UCS) -- Part 1: Architecture and basic + multilingual plane. + + [MLST] Elz, R. and P. Hethmon, "Extensions to FTP", Work in + Progress. + + [RFC854] Postel, J. and J. Reynolds, "Telnet Protocol + Specification", STD 8, RFC 854, May 1983. + + [RFC959] Postel, J. and J. Reynolds, "File Transfer Protocol + (FTP)", STD 9, RFC 959, October 1985. + + [RFC1123] Braden, R., "Requirements for Internet Hosts -- + Application and Support", STD 3, RFC 1123, October 1989. + + [RFC1738] Berners-Lee, T., Masinter, L. and M. McCahill, "Uniform + Resource Locators (URL)", RFC 1738, December 1994. + + [RFC1766] Alvestrand, H., "Tags for the Identification of + Languages", RFC 1766, March 1995. + + [RFC2130] Weider, C., Preston, C., Simonsen, K., Alvestrand, H., + Atkinson, R., Crispin, M. and P. Svanberg, "Character + Set Workshop Report", RFC 2130, April 1997. + + [RFC2277] Alvestrand, H., " IETF Policy on Character Sets and + Languages", RFC 2277, January 1998. + + [RFC2279] Yergeau, F., "UTF-8, a transformation format of ISO + 10646", RFC 2279, January 1998. + + [RFC2389] Elz, R. and P. Hethmon, "Feature Negotiation Mechanism + for the File Transfer Protocol", RFC 2389, August 1998. + + [UNICODE] The Unicode Consortium, "The Unicode Standard - Version + 2.0", Addison Westley Developers Press, July 1996. + + [UTF-8] ISO/IEC 10646-1:1993 AMENDMENT 2 (1996). UCS + Transformation Format 8 (UTF-8). + + + + + + + + + + +Curtin Proposed Standard [Page 14] + +RFC 2640 FTP Internalization July 1999 + + +9 Author's Address + + Bill Curtin + JIEO + Attn: JEBBD + Ft. Monmouth, N.J. 07703-5613 + + EMail: curtinw@ftm.disa.mil + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +Curtin Proposed Standard [Page 15] + +RFC 2640 FTP Internalization July 1999 + + +Annex A - Implementation Considerations + +A.1 General Considerations + + - Implementers should ensure that their code accounts for potential + problems, such as using a NULL character to terminate a string or + no longer being able to steal the high order bit for internal use, + when supporting the extended character set. + + - Implementers should be aware that there is a chance that pathnames + that are non UTF-8 may be parsed as valid UTF-8. The probabilities + are low for some encoding or statistically zero to zero for others. + A recent non-scientific analysis found that EUC encoded Japanese + words had a 2.7% false reading; SJIS had a 0.0005% false reading; + other encoding such as ASCII or KOI-8 have a 0% false reading. This + probability is highest for short pathnames and decreases as + pathname size increases. Implementers may want to look for signs + that pathnames which parse as UTF-8 are not valid UTF-8, such as + the existence of multiple local character sets in short pathnames. + Hopefully, as more implementations conform to UTF-8 transfer + encoding there will be a smaller need to guess at the encoding. + + - Client developers should be aware that it will be possible for + pathnames to contain mixed characters (e.g. + //Latin1DirectoryName/HebrewFileName). They should be prepared to + handle the Bi-directional (BIDI) display of these character sets + (i.e. right to left display for the directory and left to right + display for the filename). While bi-directional display is outside + the scope of this document and more complicated than the above + example, an algorithm for bi-directional display can be found in + the UNICODE 2.0 [UNICODE] standard. Also note that pathnames can + have different byte ordering yet be logically and display-wise + equivalent due to the insertion of BIDI control characters at + different points during composition. Also note that mixed character + sets may also present problems with font swapping. + + - A server that copies pathnames transparently from a local + filesystem may continue to do so. It is then up to the local file + creators to use UTF-8 pathnames. + + - Servers can supports charset labeling of files and/or directories, + such that different pathnames may have different charsets. The + server should attempt to convert all pathnames to UTF-8, but if it + can't then it should leave that name in its raw form. + + - Some server's OS do not mandate character sets, but allow + administrators to configure it in the FTP server. These servers + should be configured to use a particular mapping table (either + + + +Curtin Proposed Standard [Page 16] + +RFC 2640 FTP Internalization July 1999 + + + external or built-in). This will allow the flexibility of defining + different charsets for different directories. + + - If the server's OS does not mandate the character set and the FTP + server cannot be configured, the server should simply use the raw + bytes in the file name. They might be ASCII or UTF-8. + + - If the server is a mirror, and wants to look just like the site it + is mirroring, it should store the exact file name bytes that it + received from the main server. + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +Curtin Proposed Standard [Page 17] + +RFC 2640 FTP Internalization July 1999 + + +A.2 Transition Considerations + + - Servers which support this specification, when presented a pathname + from an old client (one which does not support this specification), + can nearly always tell whether the pathname is in UTF-8 (see B.1) + or in some other code set. In order to support these older clients, + servers may wish to default to a non UTF-8 code set. However, how a + server supports non UTF-8 is outside the scope of this + specification. + + - Clients which support this specification will be able to determine + if the server can support UTF-8 (i.e. supports this specification) + by the ability of the server to support the FEAT command and the + UTF8 feature (defined in 3.2). If the newer clients determine that + the server does not support UTF-8 it may wish to default to a + different code set. Client developers should take into + consideration that pathnames, associated with older servers, might + be stored in UTF-8. However, how a client supports non UTF-8 is + outside the scope of this specification. + + - Clients and servers can transition to UTF-8 by either converting + to/from the local encoding, or the users can store UTF-8 filenames. + The former approach is easier on tightly controlled file systems + (e.g. PCs and MACs). The latter approach is easier on more free + form file systems (e.g. Unix). + + - For interactive use attention should be focused on user interface + and ease of use. Non-interactive use requires a consistent and + controlled behavior. + + - There may be many applications which reference files under their + old raw pathname (e.g. linked URLs). Changing the pathname to UTF-8 + will cause access to the old URL to fail. A solution may be for the + server to act as if there was 2 different pathnames associated with + the file. This might be done internal to the server on controlled + file systems or by using symbolic links on free form systems. While + this approach may work for single file transfer non-interactive + use, a non-interactive transfer of all of the files in a directory + will produce duplicates. Interactive users may be presented with + lists of files which are double the actual number files. + + + + + + + + + + + +Curtin Proposed Standard [Page 18] + +RFC 2640 FTP Internalization July 1999 + + +Annex B - Sample Code and Examples + +B.1 Valid UTF-8 check + + The following routine checks if a byte sequence is valid UTF-8. This + is done by checking for the proper tagging of the first and following + bytes to make sure they conform to the UTF-8 format. It then checks + to assure that the data part of the UTF-8 sequence conforms to the + proper range allowed by the encoding. Note: This routine will not + detect characters that have not been assigned and therefore do not + exist. + +int utf8_valid(const unsigned char *buf, unsigned int len) +{ + const unsigned char *endbuf = buf + len; + unsigned char byte2mask=0x00, c; + int trailing = 0; // trailing (continuation) bytes to follow + + while (buf != endbuf) + { + c = *buf++; + if (trailing) + if ((c&0xC0) == 0x80) // Does trailing byte follow UTF-8 format? + {if (byte2mask) // Need to check 2nd byte for proper range? + if (c&byte2mask) // Are appropriate bits set? + byte2mask=0x00; + else + return 0; + trailing--; } + else + return 0; + else + if ((c&0x80) == 0x00) continue; // valid 1 byte UTF-8 + else if ((c&0xE0) == 0xC0) // valid 2 byte UTF-8 + if (c&0x1E) // Is UTF-8 byte in + // proper range? + trailing =1; + else + return 0; + else if ((c&0xF0) == 0xE0) // valid 3 byte UTF-8 + {if (!(c&0x0F)) // Is UTF-8 byte in + // proper range? + byte2mask=0x20; // If not set mask + // to check next byte + trailing = 2;} + else if ((c&0xF8) == 0xF0) // valid 4 byte UTF-8 + {if (!(c&0x07)) // Is UTF-8 byte in + // proper range? + + + +Curtin Proposed Standard [Page 19] + +RFC 2640 FTP Internalization July 1999 + + + byte2mask=0x30; // If not set mask + // to check next byte + trailing = 3;} + else if ((c&0xFC) == 0xF8) // valid 5 byte UTF-8 + {if (!(c&0x03)) // Is UTF-8 byte in + // proper range? + byte2mask=0x38; // If not set mask + // to check next byte + trailing = 4;} + else if ((c&0xFE) == 0xFC) // valid 6 byte UTF-8 + {if (!(c&0x01)) // Is UTF-8 byte in + // proper range? + byte2mask=0x3C; // If not set mask + // to check next byte + trailing = 5;} + else return 0; + } + return trailing == 0; +} + +B.2 Conversions + + The code examples in this section closely reflect the algorithm in + ISO 10646 and may not present the most efficient solution for + converting to / from UTF-8 encoding. If efficiency is an issue, + implementers should use the appropriate bitwise operators. + + Additional code examples and numerous mapping tables can be found at + the Unicode site, HTTP://www.unicode.org or FTP://unicode.org. + + Note that the conversion examples below assume that the local + character set supported in the operating system is something other + than UCS2/UTF-16. There are some operating systems that already + support UCS2/UTF-16 (notably Plan 9 and Windows NT). In this case no + conversion will be necessary from the local character set to the UCS. + +B.2.1 Conversion from Local Character Set to UTF-8 + + Conversion from the local filesystem character set to UTF-8 will + normally involve a two step process. First convert the local + character set to the UCS; then convert the UCS to UTF-8. + + The first step in the process can be performed by maintaining a + mapping table that includes the local character set code and the + corresponding UCS code. For instance the ISO/IEC 8859-8 [ISO-8859] + code for the Hebrew letter "VAV" is 0xE4. The corresponding 4 byte + ISO/IEC 10646 code is 0x000005D5. + + + + +Curtin Proposed Standard [Page 20] + +RFC 2640 FTP Internalization July 1999 + + + The next step is to convert the UCS character code to the UTF-8 + encoding. The following routine can be used to determine and encode + the correct number of bytes based on the UCS-4 character code: + + unsigned int ucs4_to_utf8 (unsigned long *ucs4_buf, unsigned int + ucs4_len, unsigned char *utf8_buf) + + { + const unsigned long *ucs4_endbuf = ucs4_buf + ucs4_len; + unsigned int utf8_len = 0; // return value for UTF8 size + unsigned char *t_utf8_buf = utf8_buf; // Temporary pointer + // to load UTF8 values + + while (ucs4_buf != ucs4_endbuf) + { + if ( *ucs4_buf <= 0x7F) // ASCII chars no conversion needed + { + *t_utf8_buf++ = (unsigned char) *ucs4_buf; + utf8_len++; + ucs4_buf++; + } + else + if ( *ucs4_buf <= 0x07FF ) // In the 2 byte utf-8 range + { + *t_utf8_buf++= (unsigned char) (0xC0 + (*ucs4_buf/0x40)); + *t_utf8_buf++= (unsigned char) (0x80 + (*ucs4_buf%0x40)); + utf8_len+=2; + ucs4_buf++; + } + else + if ( *ucs4_buf <= 0xFFFF ) /* In the 3 byte utf-8 range. The + values 0x0000FFFE, 0x0000FFFF + and 0x0000D800 - 0x0000DFFF do + not occur in UCS-4 */ + { + *t_utf8_buf++= (unsigned char) (0xE0 + + (*ucs4_buf/0x1000)); + *t_utf8_buf++= (unsigned char) (0x80 + + ((*ucs4_buf/0x40)%0x40)); + *t_utf8_buf++= (unsigned char) (0x80 + (*ucs4_buf%0x40)); + utf8_len+=3; + ucs4_buf++; + } + else + if ( *ucs4_buf <= 0x1FFFFF ) //In the 4 byte utf-8 range + { + *t_utf8_buf++= (unsigned char) (0xF0 + + (*ucs4_buf/0x040000)); + + + +Curtin Proposed Standard [Page 21] + +RFC 2640 FTP Internalization July 1999 + + + *t_utf8_buf++= (unsigned char) (0x80 + + ((*ucs4_buf/0x10000)%0x40)); + *t_utf8_buf++= (unsigned char) (0x80 + + ((*ucs4_buf/0x40)%0x40)); + *t_utf8_buf++= (unsigned char) (0x80 + (*ucs4_buf%0x40)); + utf8_len+=4; + ucs4_buf++; + + } + else + if ( *ucs4_buf <= 0x03FFFFFF )//In the 5 byte utf-8 range + { + *t_utf8_buf++= (unsigned char) (0xF8 + + (*ucs4_buf/0x01000000)); + *t_utf8_buf++= (unsigned char) (0x80 + + ((*ucs4_buf/0x040000)%0x40)); + *t_utf8_buf++= (unsigned char) (0x80 + + ((*ucs4_buf/0x1000)%0x40)); + *t_utf8_buf++= (unsigned char) (0x80 + + ((*ucs4_buf/0x40)%0x40)); + *t_utf8_buf++= (unsigned char) (0x80 + + (*ucs4_buf%0x40)); + utf8_len+=5; + ucs4_buf++; + } + else + if ( *ucs4_buf <= 0x7FFFFFFF )//In the 6 byte utf-8 range + { + *t_utf8_buf++= (unsigned char) + (0xF8 +(*ucs4_buf/0x40000000)); + *t_utf8_buf++= (unsigned char) (0x80 + + ((*ucs4_buf/0x01000000)%0x40)); + *t_utf8_buf++= (unsigned char) (0x80 + + ((*ucs4_buf/0x040000)%0x40)); + *t_utf8_buf++= (unsigned char) (0x80 + + ((*ucs4_buf/0x1000)%0x40)); + *t_utf8_buf++= (unsigned char) (0x80 + + ((*ucs4_buf/0x40)%0x40)); + *t_utf8_buf++= (unsigned char) (0x80 + + (*ucs4_buf%0x40)); + utf8_len+=6; + ucs4_buf++; + + } + } + return (utf8_len); + } + + + + +Curtin Proposed Standard [Page 22] + +RFC 2640 FTP Internalization July 1999 + + +B.2.2 Conversion from UTF-8 to Local Character Set + + When moving from UTF-8 encoding to the local character set the + reverse procedure is used. First the UTF-8 encoding is transformed + into the UCS-4 character set. The UCS-4 is then converted to the + local character set from a mapping table (i.e. the opposite of the + table used to form the UCS-4 character code). + + To convert from UTF-8 to UCS-4 the free bits (those that do not + define UTF-8 sequence size or signify continuation bytes) in a UTF-8 + sequence are concatenated as a bit string. The bits are then + distributed into a four-byte sequence starting from the least + significant bits. Those bits not assigned a bit in the four-byte + sequence are padded with ZERO bits. The following routine converts + the UTF-8 encoding to UCS-4 character codes: + + int utf8_to_ucs4 (unsigned long *ucs4_buf, unsigned int utf8_len, + unsigned char *utf8_buf) + { + + const unsigned char *utf8_endbuf = utf8_buf + utf8_len; + unsigned int ucs_len=0; + + while (utf8_buf != utf8_endbuf) + { + + if ((*utf8_buf & 0x80) == 0x00) /*ASCII chars no conversion + needed */ + { + *ucs4_buf++ = (unsigned long) *utf8_buf; + utf8_buf++; + ucs_len++; + } + else + if ((*utf8_buf & 0xE0)== 0xC0) //In the 2 byte utf-8 range + { + *ucs4_buf++ = (unsigned long) (((*utf8_buf - 0xC0) * 0x40) + + ( *(utf8_buf+1) - 0x80)); + utf8_buf += 2; + ucs_len++; + } + else + if ( (*utf8_buf & 0xF0) == 0xE0 ) /*In the 3 byte utf-8 + range */ + { + *ucs4_buf++ = (unsigned long) (((*utf8_buf - 0xE0) * 0x1000) + + (( *(utf8_buf+1) - 0x80) * 0x40) + + ( *(utf8_buf+2) - 0x80)); + + + +Curtin Proposed Standard [Page 23] + +RFC 2640 FTP Internalization July 1999 + + + utf8_buf+=3; + ucs_len++; + } + else + if ((*utf8_buf & 0xF8) == 0xF0) /* In the 4 byte utf-8 + range */ + { + *ucs4_buf++ = (unsigned long) + (((*utf8_buf - 0xF0) * 0x040000) + + (( *(utf8_buf+1) - 0x80) * 0x1000) + + (( *(utf8_buf+2) - 0x80) * 0x40) + + ( *(utf8_buf+3) - 0x80)); + utf8_buf+=4; + ucs_len++; + } + else + if ((*utf8_buf & 0xFC) == 0xF8) /* In the 5 byte utf-8 + range */ + { + *ucs4_buf++ = (unsigned long) + (((*utf8_buf - 0xF8) * 0x01000000) + + ((*(utf8_buf+1) - 0x80) * 0x040000) + + (( *(utf8_buf+2) - 0x80) * 0x1000) + + (( *(utf8_buf+3) - 0x80) * 0x40) + + ( *(utf8_buf+4) - 0x80)); + utf8_buf+=5; + ucs_len++; + } + else + if ((*utf8_buf & 0xFE) == 0xFC) /* In the 6 byte utf-8 + range */ + { + *ucs4_buf++ = (unsigned long) + (((*utf8_buf - 0xFC) * 0x40000000) + + ((*(utf8_buf+1) - 0x80) * 0x010000000) + + ((*(utf8_buf+2) - 0x80) * 0x040000) + + (( *(utf8_buf+3) - 0x80) * 0x1000) + + (( *(utf8_buf+4) - 0x80) * 0x40) + + ( *(utf8_buf+5) - 0x80)); + utf8_buf+=6; + ucs_len++; + } + + } + return (ucs_len); + } + + + + + +Curtin Proposed Standard [Page 24] + +RFC 2640 FTP Internalization July 1999 + + +B.2.3 ISO/IEC 8859-8 Example + + This example demonstrates mapping ISO/IEC 8859-8 character set to + UTF-8 and back to ISO/IEC 8859-8. As noted earlier, the Hebrew letter + "VAV" is convertd from the ISO/IEC 8859-8 character code 0xE4 to the + corresponding 4 byte ISO/IEC 10646 code of 0x000005D5 by a simple + lookup of a conversion/mapping file. + + The UCS-4 character code is transformed into UTF-8 using the + ucs4_to_utf8 routine described earlier by: + + 1. Because the UCS-4 character is between 0x80 and 0x07FF it will map + to a 2 byte UTF-8 sequence. + 2. The first byte is defined by (0xC0 + (0x000005D5 / 0x40)) = 0xD7. + + 3. The second byte is defined by (0x80 + (0x000005D5 % 0x40)) = 0x95. + + The UTF-8 encoding is transferred back to UCS-4 by using the + utf8_to_ucs4 routine described earlier by: + + 1. Because the first byte of the sequence, when the '&' operator with + a value of 0xE0 is applied, will produce 0xC0 (0xD7 & 0xE0 = 0xC0) + the UTF-8 is a 2 byte sequence. + 2. The four byte UCS-4 character code is produced by (((0xD7 - 0xC0) + * 0x40) + (0x95 -0x80)) = 0x000005D5. + + Finally, the UCS-4 character code is converted to ISO/IEC 8859-8 + character code (using the mapping table which matches ISO/IEC 8859-8 + to UCS-4 ) to produce the original 0xE4 code for the Hebrew letter + "VAV". + +B.2.4 Vendor Codepage Example + + This example demonstrates the mapping of a codepage to UTF-8 and back + to a vendor codepage. Mapping between vendor codepages can be done in + a very similar manner as described above. For instance both the PC + and Mac codepages reflect the character set from the Thai standard + TIS 620-2533. The character code on both platforms for the Thai + letter "SO SO" is 0xAB. This character can then be mapped into the + UCS-4 by way of a conversion/mapping file to produce the UCS-4 code + of 0x0E0B. + + The UCS-4 character code is transformed into UTF-8 using the + ucs4_to_utf8 routine described earlier by: + + 1. Because the UCS-4 character is between 0x0800 and 0xFFFF it will + map to a 3 byte UTF-8 sequence. + 2. The first byte is defined by (0xE0 + (0x00000E0B / 0x1000) = 0xE0. + + + +Curtin Proposed Standard [Page 25] + +RFC 2640 FTP Internalization July 1999 + + + 3. The second byte is defined by (0x80 + ((0x00000E0B / 0x40) % + 0x40))) = 0xB8. + 4. The third byte is defined by (0x80 + (0x00000E0B % 0x40)) = 0x8B. + + The UTF-8 encoding is transferred back to UCS-4 by using the + utf8_to_ucs4 routine described earlier by: + + 1. Because the first byte of the sequence, when the '&' operator with + a value of 0xF0 is applied, will produce 0xE0 (0xE0 & 0xF0 = 0xE0) + the UTF-8 is a 3 byte sequence. + 2. The four byte UCS-4 character code is produced by (((0xE0 - 0xE0) + * 0x1000) + ((0xB8 - 0x80) * 0x40) + (0x8B -0x80) = 0x0000E0B. + + Finally, the UCS-4 character code is converted to either the PC or + MAC codepage character code (using the mapping table which matches + codepage to UCS-4 ) to produce the original 0xAB code for the Thai + letter "SO SO". + +B.3 Pseudo Code for a High-Quality Translating Server + + if utf8_valid(fn) + { + attempt to convert fn to the local charset, producing localfn + if (conversion fails temporarily) return error + if (conversion succeeds) + { + attempt to open localfn + if (open fails temporarily) return error + if (open succeeds) return success + } + } + attempt to open fn + if (open fails temporarily) return error + if (open succeeds) return success + return permanent error + + + + + + + + + + + + + + + + +Curtin Proposed Standard [Page 26] + +RFC 2640 FTP Internalization July 1999 + + +Full Copyright Statement + + Copyright (C) The Internet Society (1999). All Rights Reserved. + + This document and translations of it may be copied and furnished to + others, and derivative works that comment on or otherwise explain it + or assist in its implementation may be prepared, copied, published + and distributed, in whole or in part, without restriction of any + kind, provided that the above copyright notice and this paragraph are + included on all such copies and derivative works. However, this + document itself may not be modified in any way, such as by removing + the copyright notice or references to the Internet Society or other + Internet organizations, except as needed for the purpose of + developing Internet standards in which case the procedures for + copyrights defined in the Internet Standards process must be + followed, or as required to translate it into languages other than + English. + + The limited permissions granted above are perpetual and will not be + revoked by the Internet Society or its successors or assigns. + + This document and the information contained herein is provided on an + "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING + TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING + BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION + HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF + MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. + +Acknowledgement + + Funding for the RFC Editor function is currently provided by the + Internet Society. + + + + + + + + + + + + + + + + + + + +Curtin Proposed Standard [Page 27] + diff --git a/doc/rfc2773.txt b/doc/rfc2773.txt new file mode 100644 index 0000000..4f93d6b --- /dev/null +++ b/doc/rfc2773.txt @@ -0,0 +1,507 @@ + + + + + + +Network Working Group R. Housley +Request for Comments: 2773 P. Yee +Updates: 959 SPYRUS +Category: Experimental W. Nace + NSA + February 2000 + + + Encryption using KEA and SKIPJACK + +Status of this Memo + + This memo defines an Experimental Protocol for the Internet + community. It does not specify an Internet standard of any kind. + Discussion and suggestions for improvement are requested. + Distribution of this memo is unlimited. + +Copyright Notice + + Copyright (C) The Internet Society (2000). All Rights Reserved. + +Abstract + + This document defines a method to encrypt a file transfer using the + FTP specification STD 9, RFC 959, "File Transfer Protocol (FTP)", + (October 1985) [3] and RFC 2228, "FTP Security Extensions" (October + 1997) [1]. This method will use the Key Exchange Algorithm (KEA) to + give mutual authentication and establish the data encryption keys. + SKIPJACK is used to encrypt file data and the FTP command channel. + +1.0 Introduction + + The File Transfer Protocol (FTP) provides no protocol security except + for a user authentication password which is transmitted in the clear. + In addition, the protocol does not protect the file transfer session + beyond the original authentication phase. + + The Internet Engineering Task Force (IETF) Common Authentication + Technology (CAT) Working Group has proposed security extensions to + FTP. These extensions allow the protocol to use more flexible + security schemes, and in particular allows for various levels of + protection for the FTP command and data connections. This document + describes a profile for the FTP Security Extensions by which these + mechanisms may be provisioned using the Key Exchange Algorithm (KEA) + in conjunction with the SKIPJACK symmetric encryption algorithm. + + + + + + +Housley, et al. Experimental [Page 1] + +RFC 2773 Encryption using KEA and SKIPJACK February 2000 + + + FTP Security Extensions [1] provides: + + * user authentication -- augmenting the normal password + mechanism; + + * server authentication -- normally done in conjunction with user + authentication; + + * session parameter negotiation -- in particular, encryption keys + and attributes; + + * command connection protection -- integrity, confidentiality, or + both; + + * data transfer protection -- same as for command connection + protection. + + In order to support the above security services, the two FTP entities + negotiate a mechanism. This process is open-ended and completes when + both entities agree on an acceptable mechanism or when the initiating + party (always the client) is unable to suggest an agreeable + mechanism. Once the entities agree upon a mechanism, they may + commence authentication and/or parameter negotiation. + + Authentication and parameter negotiation occur within an unbounded + series of exchanges. At the completion of the exchanges, the + entities will either be authenticated (unilateral or mutually), and + may, additionally, be ready to protect FTP commands and data. + + Following the exchanges, the entities negotiate the size of the + buffers they will use in protecting the commands and data that + follow. This process is accomplished in two steps: the client offers + a suggested buffer size and the server may either refuse it, counter + it, or accept it. + + At this point, the entities may issue protected commands within the + bounds of the parameters negotiated through the security exchanges. + Protected commands are issued by applying the protection services + required to the normal commands and Base64 encoding the results. The + encoded results are sent as the data field within a ENC (integrity + and confidentiality) command. Base64 is an encoding for mapping + binary data onto a textual character set that is able to pass through + most 7-bit systems without loss. The server sends back responses in + new result codes which allow the identical protections and Base64 + encoding to be applied to the results. Protection of the data + transfers can be specified via the PROT command which supports the + + + + + +Housley, et al. Experimental [Page 2] + +RFC 2773 Encryption using KEA and SKIPJACK February 2000 + + + same protections as those afforded the other FTP commands. PROT + commands may be sent on a transfer-by-transfer basis, however, the + session parameters may not be changed within a session. + +2.0 Key Exchange Algorithm (KEA) Profile + + This paper profiles KEA with SKIPJACK to achieve certain security + services when used in conjunction with the FTP Security Extensions + framework. FTP entities may use KEA to give mutual authentication + and establish data encryption keys. We specify a simple token format + and set of exchanges to deliver these services. Functions that may + be performed by the Fortezza Crypto Card. + + The reader should be familiar with the extensions in order to + understand the protocol steps that follow. In the context of the FTP + Security Extensions, we suggest the usage of KEA with SKIPJACK for + authentication, integrity, and confidentiality. + + A client may mutually authenticate with a server. What follows are + the protocol steps necessary to perform KEA authentication under the + FTP Security Extensions framework. Where failure modes are + encountered, the return codes follow those specified in the + Extensions. They are not enumerated in this document as they are + invariant among the mechanisms used. The certificates are ASN.1 + encoded. + + The exchanges detailed below presume a working knowledge of the FTP + Security Extensions. The notation for concatenation is " || ". + Decryption of encrypted data and certification path validation is + implicitly assumed, but is not shown. + +--------------------------------------------------------------------- + Client Server + + AUTH KEA-SKIPJACK --> + <-- 334 ADAT=Base64( Certb || Rb ) + ADAT Base64( Certa || Ra || + WMEK || IV || Encrypt( + Label-Type || Label-Length || + Label-List || pad || ICV ) ) --> + <-- 235 ADAT=Base64( IV ) +--------------------------------------------------------------------- + Figure 1 + + The server and client certificates contain KEA public keys. The + client and server use KEA to generate a shared SKIPJACK symmetric + key, called the TEK. The client uses the random number generator to + create a second SKIPJACK key, called the MEK. The MEK is wrapped in + + + +Housley, et al. Experimental [Page 3] + +RFC 2773 Encryption using KEA and SKIPJACK February 2000 + + + the TEK for transfer to the server. An initialization vector (IV) + associated with the MEK is generated by the client and transferred to + the server. A list of security labels that the client wants to use + for this FTP session may be transferred to the server encrypted in + the MEK. As shown in Figure 2, the security label data is formatted + as a one octet type value, a four octet label length, the security + label list, padding, followed by an eight octet integrity check value + (ICV). Figure 3 lists the label types. If the label type is absent + (value of zero length), then the label size must be zero. + + In order to ensure that the length of the plain text is a multiple of + the cryptographic block size, padding shall be performed as follows. + The input to the SKIPJACK CBC encryption process shall be padded to a + multiple of 8 octets. Let n be the length in octets of the input. + Pad the input by appending 8 - (n mod 8) octets to the end of the + message, each having the value 8 - (n mod 8), the number of octets + being added. In hexadecimal, he possible pad strings are: 01, 0202, + 030303, 04040404, 0505050505, 060606060606, 07070707070707, and + 0808080808080808. All input is padded with 1 to 8 octets to produce + a multiple of 8 octets in length. This pad technique is used + whenever SKIPJACK CBC encryption is performed. + + An ICV is calculated over the plaintext security label and padding. + The ICV algorithm used is the 32-bit one's complement addition of + each 32-bit block followed by 32 zero bits. This ICV technique is + used in conjunction with SKIPJACK CBC encryption to provide data + integrity. + + --------------------------------------------------------------------- + Label Type 1 Octet + Label Length 4 octets + Label List variable length + Pad 1 to 8 octets + ICV 8 octets + --------------------------------------------------------------------- + Figure 2 + + --------------------------------------------------------------------- + Label Type Label Syntax Reference + 0 Absent Not applicable + 1 MSP SDN.701[2] + 2-255 Reserved for Future Use To Be Determined + + --------------------------------------------------------------------- + Figure 3 + + + + + + +Housley, et al. Experimental [Page 4] + +RFC 2773 Encryption using KEA and SKIPJACK February 2000 + + + FTP command channel operations are now confidentiality protected. To + provide integrity, the command sequence number, padding, and ICV are + appended to each command prior to encryption. + + Sequence integrity is provided by using a 16-bit sequence number + which is incremented for each command. The sequence number is + initialized with the least significant 16-bits of Ra. The server + response will include the same sequence number as the client command. + + An ICV is calculated over the individual commands (including the + carriage return and line feed required to terminate commands), the + sequence number, and pad. + + --------------------------------------------------------------------- + Client Server + + ENC Base64(Encrypt("PBSZ 65535" + || SEQ || pad || ICV )) --> + <-- 632 Base64(Encrypt("200" || + SEQ || pad || ICV)) + ENC Base64(Encrypt("USER yee" + || SEQ || pad || ICV)) --> + <-- 632 Base64(Encrypt("331" || + SEQ || pad || ICV)) + ENC Base64(Encrypt("PASS + fortezza" || SEQ || + pad || ICV)) --> + <-- 631 Base64(Sign("230")) + --------------------------------------------------------------------- + Figure 4 + + After decryption, both parties verifying the integrity of the PBSZ + commands by checking for the expected sequence number and correct + ICV. The correct SKIPJACK key calculation, ICV checking, and the + validation of the certificates containing the KEA public keys + provides mutual identification and authentication. + + --------------------------------------------------------------------- + Client Server + + ENC Base64(Encrypt("PROT P" || + SEQ || pad || ICV)) --> + <-- 632 Base64(Encrypt("200" || SEQ + || pad || ICV)) + --------------------------------------------------------------------- + Figure 5 + + + + + +Housley, et al. Experimental [Page 5] + +RFC 2773 Encryption using KEA and SKIPJACK February 2000 + + + At this point, files may be sent or received with encryption and + integrity services in use. If encryption is used, then the first + buffer will contain the token followed by enough encrypted file + octets to completely fill the buffer (unless the file is too short to + fill the buffer). Subsequent buffers contain only encrypted file + octets. All buffers are completely full except the final buffer. + + --------------------------------------------------------------------- + Client Server + + ENC Base64(Encrypt( + ("RETR foo.bar") || + SEQ || pad || ICV)) --> + <-- 632 Base64(Encrypt("150" || + SEQ || pad || ICV)) + --------------------------------------------------------------------- + Figure 6 + + The next figure shows the header information and the file data. + + --------------------------------------------------------------------- + Plaintext Token IV 24 octets + WMEK 12 octets + Hashvalue 20 octets + IV 24 octets + Label Type 1 octets + Label Length 4 octets + Label Label Length octets + Pad 1 to 8 octets + ICV 8 octets + --------------------------------------------------------------------- + Figure 7 + +2.1 Pre-encrypted File Support + + In order to support both on-the-fly encryption and pre-encrypted + files, a token is defined for carrying a file encryption key (FEK). + To prevent truncation and ensure file integrity, the token also + includes a hash computed on the complete file. The token also + contains the security label associate with the file. This FEK is + wrapped in the session TEK. The token is encrypted in the session + TEK using SKIPJACK CBC mode. The token contains a 12 octet wrapped + FEK, a 20 octet file hash, a 24 octet file IV, a 1 octet label type, + a 4 octet label length, a variable length label value, a one to 8 + octet pad, and an 8 octet ICV. The first 24 octets of the token are + the plaintext IV used to encrypt the remainder of the token. The + token requires its own encryption IV because it is transmitted across + the data channel, not the command channel, and ordering between the + + + +Housley, et al. Experimental [Page 6] + +RFC 2773 Encryption using KEA and SKIPJACK February 2000 + + + channels cannot be guaranteed. Storage of precomputed keys and + hashes for files in the file system is a local implementation matter; + however, it is suggested that if a file is pre-encrypted, then the + FEK be wrapped in a local storage key. When the file is needed, the + FEK is unwrapped using the local storage key, and then rewrapped in + the session TEK. Figure 8 shows the assembled token. + + --------------------------------------------------------------------- + Plaintext Token IV 24 octets + Wrapped FEK 12 octets + Hashvalue 20 octets + IV 24 octets + Label Type 1 octet + Label Length 4 octets + Label Label Length octets + Pad 1 to 8 octets + ICV 8 octets + --------------------------------------------------------------------- + Figure 8 + +3.0 Table of Key Terminology + + In order to clarify the usage of various keys in this protocol, + Figure 9 summarizes key types and their usage: + + --------------------------------------------------------------------- + Key Type Usage + TEK Encryption of token at the beginning of + each file, also wraps the MEK and the FEK + MEK Encryption of command channel + FEK Encryption of the file itself (may be + done out of scope of FTP) + + --------------------------------------------------------------------- + Figure 9 + +4.0 Security Considerations + + This entire memo is about security mechanisms. For KEA to provide + the authentication and key management discussed, the implementation + must protect the private key from disclosure. For SKIPJACK to + provide the confidentiality discussed, the implementation must + protect the shared symmetric keys from disclosure. + +5.0 Acknowledgements + + We would like to thank Todd Horting for insights gained during + implementation of this specification. + + + +Housley, et al. Experimental [Page 7] + +RFC 2773 Encryption using KEA and SKIPJACK February 2000 + + +6.0 References + + [1] Horowitz, M. and S. Lunt, "FTP Security Extensions", RFC 2228, + October 1997. + + [2] Message Security Protocol 4.0 (MSP), Revision A. Secure Data + Network System (SDNS) Specification, SDN.701, February 6, 1997. + + [3] Postel, J. and J. Reynolds, "File Transfer Protocol", STD 9, RFC + 959, October 1985. + +7.0 Authors' Addresses + + Russell Housley + SPYRUS + 381 Elden Street + Suite 1120 + Herndon, VA 20170 + USA + + Phone: +1 703 707-0696 + EMail: housley@spyrus.com + + + Peter Yee + SPYRUS + 5303 Betsy Ross Drive + Santa Clara, CA 95054 + USA + + Phone: +1 408 327-1901 + EMail: yee@spyrus.com + + + + + + + + + + + + + + + + + + + +Housley, et al. Experimental [Page 8] + +RFC 2773 Encryption using KEA and SKIPJACK February 2000 + + +8.0 Full Copyright Statement + + Copyright (C) The Internet Society (2000). All Rights Reserved. + + This document and translations of it may be copied and furnished to + others, and derivative works that comment on or otherwise explain it + or assist in its implementation may be prepared, copied, published + and distributed, in whole or in part, without restriction of any + kind, provided that the above copyright notice and this paragraph are + included on all such copies and derivative works. However, this + document itself may not be modified in any way, such as by removing + the copyright notice or references to the Internet Society or other + Internet organizations, except as needed for the purpose of + developing Internet standards in which case the procedures for + copyrights defined in the Internet Standards process must be + followed, or as required to translate it into languages other than + English. + + The limited permissions granted above are perpetual and will not be + revoked by the Internet Society or its successors or assigns. + + This document and the information contained herein is provided on an + "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING + TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING + BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION + HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF + MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. + +Acknowledgement + + Funding for the RFC Editor function is currently provided by the + Internet Society. + + + + + + + + + + + + + + + + + + + +Housley, et al. Experimental [Page 9] + diff --git a/doc/rfc3659.txt b/doc/rfc3659.txt new file mode 100644 index 0000000..a13c89b --- /dev/null +++ b/doc/rfc3659.txt @@ -0,0 +1,3419 @@ + + + + + + +Network Working Group P. Hethmon +Request for Comments: 3659 Hethmon Software +Updates: 959 March 2007 +Category: Standards Track + + + Extensions to FTP + +Status of This Memo + + This document specifies an Internet standards track protocol for the + Internet community, and requests discussion and suggestions for + improvements. Please refer to the current edition of the "Internet + Official Protocol Standards" (STD 1) for the standardization state + and status of this protocol. Distribution of this memo is unlimited. + +Copyright Notice + + Copyright (C) The IETF Trust (2007). + +Abstract + + This document specifies new FTP commands to obtain listings of remote + directories in a defined format, and to permit restarts of + interrupted data transfers in STREAM mode. It allows character sets + other than US-ASCII, and also defines an optional virtual file + storage structure. + + + + + + + + + + + + + + + + + + + + + + + + +Hethmon Standards Track [Page 1] + +RFC 3659 Extensions to FTP March 2007 + + +Table of Contents + + 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3 + 2. Document Conventions . . . . . . . . . . . . . . . . . . . . . 3 + 2.1. Basic Tokens . . . . . . . . . . . . . . . . . . . . . . 4 + 2.2. Pathnames. . . . . . . . . . . . . . . . . . . . . . . . 4 + 2.3. Times. . . . . . . . . . . . . . . . . . . . . . . . . . 6 + 2.4. Server Replies . . . . . . . . . . . . . . . . . . . . . 7 + 2.5. Interpreting Examples. . . . . . . . . . . . . . . . . . 8 + 3. File Modification Time (MDTM). . . . . . . . . . . . . . . . . 8 + 3.1. Syntax . . . . . . . . . . . . . . . . . . . . . . . . . 9 + 3.2. Error Responses. . . . . . . . . . . . . . . . . . . . . 9 + 3.3. FEAT Response for MDTM . . . . . . . . . . . . . . . . . 10 + 3.4. MDTM Examples. . . . . . . . . . . . . . . . . . . . . . 10 + 4. File SIZE. . . . . . . . . . . . . . . . . . . . . . . . . . . 11 + 4.1. Syntax . . . . . . . . . . . . . . . . . . . . . . . . . 11 + 4.2. Error Responses. . . . . . . . . . . . . . . . . . . . . 12 + 4.3. FEAT Response for SIZE . . . . . . . . . . . . . . . . . 12 + 4.4. Size Examples. . . . . . . . . . . . . . . . . . . . . . 12 + 5. Restart of Interrupted Transfer (REST) . . . . . . . . . . . . 13 + 5.1. Restarting in STREAM Mode. . . . . . . . . . . . . . . . 14 + 5.2. Error Recovery and Restart . . . . . . . . . . . . . . . 14 + 5.3. Syntax . . . . . . . . . . . . . . . . . . . . . . . . . 15 + 5.4. FEAT Response for REST . . . . . . . . . . . . . . . . . 16 + 5.5. REST Example . . . . . . . . . . . . . . . . . . . . . . 17 + 6. A Trivial Virtual File Store (TVFS). . . . . . . . . . . . . . 17 + 6.1. TVFS File Names. . . . . . . . . . . . . . . . . . . . . 18 + 6.2. TVFS Pathnames . . . . . . . . . . . . . . . . . . . . . 18 + 6.3. FEAT Response for TVFS . . . . . . . . . . . . . . . . . 20 + 6.4. OPTS for TVFS. . . . . . . . . . . . . . . . . . . . . . 21 + 6.5. TVFS Examples. . . . . . . . . . . . . . . . . . . . . . 21 + 7. Listings for Machine Processing (MLST and MLSD). . . . . . . . 23 + 7.1. Format of MLSx Requests. . . . . . . . . . . . . . . . . 23 + 7.2. Format of MLSx Response. . . . . . . . . . . . . . . . . 24 + 7.3. File Name Encoding . . . . . . . . . . . . . . . . . . . 26 + 7.4. Format of Facts. . . . . . . . . . . . . . . . . . . . . 28 + 7.5. Standard Facts . . . . . . . . . . . . . . . . . . . . . 28 + 7.6. System Dependent and Local Facts . . . . . . . . . . . . 36 + 7.7. MLSx Examples. . . . . . . . . . . . . . . . . . . . . . 37 + 7.8. FEAT Response for MLSx . . . . . . . . . . . . . . . . . 49 + 7.9. OPTS Parameters for MLST . . . . . . . . . . . . . . . . 51 + 8. Impact on Other FTP Commands . . . . . . . . . . . . . . . . . 54 + 9. Character Sets and Internationalization. . . . . . . . . . . . 55 + 10. IANA Considerations. . . . . . . . . . . . . . . . . . . . . . 55 + 10.1. The OS Specific Fact Registry. . . . . . . . . . . . . . 56 + 10.2. The OS Specific Filetype Registry. . . . . . . . . . . . 56 + + + + + +Hethmon Standards Track [Page 2] + +RFC 3659 Extensions to FTP March 2007 + + + 11. Security Considerations. . . . . . . . . . . . . . . . . . . . 57 + 12. Normative References . . . . . . . . . . . . . . . . . . . . . 58 + Acknowledgments. . . . . . . . . . . . . . . . . . . . . . . . . . 59 + +1. Introduction + + This document updates the File Transfer Protocol (FTP) [3]. Four new + commands are added: "SIZE", "MDTM", "MLST", and "MLSD". The existing + command "REST" is modified. Of those, the "SIZE" and "MDTM" + commands, and the modifications to "REST" have been in wide use for + many years. The others are new. + + These commands allow a client to restart an interrupted transfer in + transfer modes not previously supported in any documented way, and to + obtain a directory listing in a machine friendly, predictable, + format. + + An optional structure for the server's file store (NVFS) is also + defined, allowing servers that support such a structure to convey + that information to clients in a standard way, thus allowing clients + more certainty in constructing and interpreting pathnames. + +2. Document Conventions + + This document makes use of the document conventions defined in BCP + 14, RFC 2119 [4]. That provides the interpretation of capitalized + imperative words like MUST, SHOULD, etc. + + This document also uses notation defined in STD 9, RFC 959 [3]. In + particular, the terms "reply", "user", "NVFS" (Network Virtual File + System), "file", "pathname", "FTP commands", "DTP" (data transfer + process), "user-FTP process", "user-PI" (user protocol interpreter), + "user-DTP", "server-FTP process", "server-PI", "server-DTP", "mode", + "type", "NVT" (Network Virtual Terminal), "control connection", "data + connection", and "ASCII", are all used here as defined there. + + Syntax required is defined using the Augmented BNF defined in [5]. + Some general ABNF definitions that are required throughout the + document will be defined later in this section. At first reading, it + may be wise to simply recall that these definitions exist here, and + skip to the next section. + + + + + + + + + + +Hethmon Standards Track [Page 3] + +RFC 3659 Extensions to FTP March 2007 + + +2.1. Basic Tokens + + This document imports the core ABNF definitions given in Appendix A + of [5]. There definitions will be found for basic ABNF elements like + ALPHA, DIGIT, SP, etc. The following terms are added for use in this + document. + + TCHAR = VCHAR / SP / HTAB ; visible plus white space + RCHAR = ALPHA / DIGIT / "," / "." / ":" / "!" / + "@" / "#" / "$" / "%" / "^" / + "&" / "(" / ")" / "-" / "_" / + "+" / "?" / "/" / "\" / "'" / + DQUOTE ; <"> -- double quote character (%x22) + SCHAR = RCHAR / "=" ; + + The VCHAR (from [5]), RCHAR, SCHAR, and TCHAR types give basic + character types from varying sub-sets of the ASCII character set for + use in various commands and responses. + + token = 1*RCHAR + + A "token" is a string whose precise meaning depends upon the context + in which it is used. In some cases it will be a value from a set of + possible values maintained elsewhere. In others it might be a string + invented by one party to an FTP conversation from whatever sources it + finds relevant. + + Note that in ABNF, string literals are case insensitive. That + convention is preserved in this document, and implies that FTP + commands added by this specification have names that can be + represented in any case. That is, "MDTM" is the same as "mdtm", + "Mdtm" and "MdTm" etc. However note that ALPHA, in particular, is + case sensitive. That implies that a "token" is a case sensitive + value. That implication is correct, except where explicitly stated + to the contrary in this document, or in some other specification that + defines the values this document specifies be used in a particular + context. + +2.2. Pathnames + + Various FTP commands take pathnames as arguments, or return pathnames + in responses. When the MLST command is supported, as indicated in + the response to the FEAT command [6], pathnames are to be transferred + in one of the following two formats. + + + + + + + +Hethmon Standards Track [Page 4] + +RFC 3659 Extensions to FTP March 2007 + + + pathname = utf-8-name / raw + utf-8-name = + raw = + + Which format is used is at the option of the user-PI or server-PI + sending the pathname. UTF-8 encodings [2] contain enough internal + structure that it is always, in practice, possible to determine + whether a UTF-8 or raw encoding has been used, in those cases where + it matters. While it is useful for the user-PI to be able to + correctly display a pathname received from the server-PI to the user, + it is far more important for the user-PI to be able to retain and + retransmit the identical pathname when required. Implementations are + advised against converting a UTF-8 pathname to a local charset that + isn't capable of representing the full Unicode character repertoire, + and then attempting to invert the charset translation later. Note + that ASCII is a subset of UTF-8. See also [1]. + + Unless otherwise specified, the pathname is terminated by the CRLF + that terminates the FTP command, or by the CRLF that ends a reply. + Any trailing spaces preceding that CRLF form part of the name. + Exactly one space will precede the pathname and serve as a separator + from the preceding syntax element. Any additional spaces form part + of the pathname. See [7] for a fuller explanation of the character + encoding issues. All implementations supporting MLST MUST support + [7]. + + Note: for pathnames transferred over a data connection, there is no + way to represent a pathname containing the characters CR and LF in + sequence, and distinguish that from the end of line indication. + Hence, pathnames containing the CRLF pair of characters cannot be + transmitted over a data connection. Data connections only contain + file names transmitted from server-FTP to user-FTP as the result of + one of the directory listing commands. Files with names containing + the CRLF sequence must either have that sequence converted to some + other form, such that the other form can be recognised and be + correctly converted back to CRLF, or be omitted from the listing. + + Implementations should also beware that the FTP control connection + uses Telnet NVT conventions [8], and that the Telnet IAC character, + if part of a pathname sent over the control connection, MUST be + correctly escaped as defined by the Telnet protocol. + + NVT also distinguishes between CR, LF, and the end of line CRLF, and + so would permit pathnames containing the pair of characters CR and LF + to be correctly transmitted. However, because such a sequence cannot + be transmitted over a data connection (as part of the result of a + LIST, NLST, or MLSD command), such pathnames are best avoided. + + + + +Hethmon Standards Track [Page 5] + +RFC 3659 Extensions to FTP March 2007 + + + Implementors should also be aware that, although Telnet NVT + conventions are used over the control connections, Telnet option + negotiation MUST NOT be attempted. See section 4.1.2.12 of [9]. + +2.2.1. Pathname Syntax + + Except where TVFS is supported (see section 6), this specification + imposes no syntax upon pathnames. Nor does it restrict the character + set from which pathnames are created. This does not imply that the + NVFS is required to make sense of all possible pathnames. Server-PIs + may restrict the syntax of valid pathnames in their NVFS in any + manner appropriate to their implementation or underlying file system. + Similarly, a server-PI may parse the pathname and assign meaning to + the components detected. + +2.2.2. Wildcarding + + For the commands defined in this specification, all pathnames are to + be treated literally. That is, for a pathname given as a parameter + to a command, the file whose name is identical to the pathname given + is implied. No characters from the pathname may be treated as + special or "magic", thus no pattern matching (other than for exact + equality) between the pathname given and the files present in the + NVFS of the server-FTP is permitted. + + Clients that desire some form of pattern matching functionality must + obtain a listing of the relevant directory, or directories, and + implement their own file name selection procedures. + +2.3. Times + + The syntax of a time value is: + + time-val = 14DIGIT [ "." 1*DIGIT ] + + The leading, mandatory, fourteen digits are to be interpreted as, in + order from the leftmost, four digits giving the year, with a range of + 1000--9999, two digits giving the month of the year, with a range of + 01--12, two digits giving the day of the month, with a range of + 01--31, two digits giving the hour of the day, with a range of + 00--23, two digits giving minutes past the hour, with a range of + 00--59, and finally, two digits giving seconds past the minute, with + a range of 00--60 (with 60 being used only at a leap second). Years + in the tenth century, and earlier, cannot be expressed. This is not + considered a serious defect of the protocol. + + + + + + +Hethmon Standards Track [Page 6] + +RFC 3659 Extensions to FTP March 2007 + + + The optional digits, which are preceded by a period, give decimal + fractions of a second. These may be given to whatever precision is + appropriate to the circumstance, however implementations MUST NOT add + precision to time-vals where that precision does not exist in the + underlying value being transmitted. + + Symbolically, a time-val may be viewed as + + YYYYMMDDHHMMSS.sss + + The "." and subsequent digits ("sss") are optional. However the "." + MUST NOT appear unless at least one following digit also appears. + + Time values are always represented in UTC (GMT), and in the Gregorian + calendar regardless of what calendar may have been in use at the date + and time indicated at the location of the server-PI. + + The technical differences among GMT, TAI, UTC, UT1, UT2, etc., are + not considered here. A server-FTP process should always use the same + time reference, so the times it returns will be consistent. Clients + are not expected to be time synchronized with the server, so the + possible difference in times that might be reported by the different + time standards is not considered important. + +2.4. Server Replies + + Section 4.2 of [3] defines the format and meaning of replies by the + server-PI to FTP commands from the user-PI. Those reply conventions + are used here without change. + + error-response = error-code SP *TCHAR CRLF + error-code = ("4" / "5") 2DIGIT + + Implementors should note that the ABNF syntax used in this document + and in other FTP related documents (but not used in [3]), sometimes + shows replies using the one-line format. Unless otherwise explicitly + stated, that is not intended to imply that multi-line responses are + not permitted. Implementors should assume that, unless stated to the + contrary, any reply to any FTP command (including QUIT) may use the + multi-line format described in [3]. + + Throughout this document, replies will be identified by the three + digit code that is their first element. Thus the term "500 reply" + means a reply from the server-PI using the three digit code "500". + + + + + + + +Hethmon Standards Track [Page 7] + +RFC 3659 Extensions to FTP March 2007 + + +2.5. Interpreting Examples + + In the examples of FTP dialogs presented in this document, lines that + begin "C> " were sent over the control connection from the user-PI to + the server-PI, lines that begin "S> " were sent over the control + connection from the server-PI to the user-PI, and each sequence of + lines that begin "D> " was sent from the server-PI to the user-PI + over a data connection created just to send those lines and closed + immediately after. No examples here show data transferred over a + data connection from the client to the server. In all cases, the + prefixes shown above, including the one space, have been added for + the purposes of this document, and are not a part of the data + exchanged between client and server. + +3. File Modification Time (MDTM) + + The FTP command, MODIFICATION TIME (MDTM), can be used to determine + when a file in the server NVFS was last modified. This command has + existed in many FTP servers for many years, as an adjunct to the REST + command for STREAM mode, thus is widely available. However, where + supported, the "modify" fact that can be provided in the result from + the new MLST command is recommended as a superior alternative. + + When attempting to restart a RETRieve, the user-FTP can use the MDTM + command or the "modify" fact to check if the modification time of the + source file is more recent than the modification time of the + partially transferred file. If it is, then most likely the source + file has changed, and it would be unsafe to restart the previously + incomplete file transfer. + + Because the user- and server-FTPs' clocks are not necessarily + synchronised, user-FTPs intending to use this method should usually + obtain the modification time of the file from the server before the + initial RETRieval, and compare that with the modification time before + a RESTart. If they differ, the files may have changed, and RESTart + would be inadvisable. Where this is not possible, the user-FTP + should make sure to allow for possible clock skew when comparing + times. + + When attempting to restart a STORe, the User FTP can use the MDTM + command to discover the modification time of the partially + transferred file. If it is older than the modification time of the + file that is about to be STORed, then most likely the source file has + changed, and it would be unsafe to restart the file transfer. + + + + + + + +Hethmon Standards Track [Page 8] + +RFC 3659 Extensions to FTP March 2007 + + + Note that using MLST (described below), where available, can provide + this information and much more, thus giving an even better indication + that a file has changed and that restarting a transfer would not give + valid results. + + Note that this is applicable to any RESTart attempt, regardless of + the mode of the file transfer. + +3.1. Syntax + + The syntax for the MDTM command is: + + mdtm = "MdTm" SP pathname CRLF + + As with all FTP commands, the "MDTM" command label is interpreted in + a case-insensitive manner. + + The "pathname" specifies an object in the NVFS that may be the object + of a RETR command. Attempts to query the modification time of files + that exist but are unable to be retrieved may generate an error- + response, or can result in a positive response carrying a time-val + with an unspecified value, the choice being made by the server-PI. + + The server-PI will respond to the MDTM command with a 213 reply + giving the last modification time of the file whose pathname was + supplied, or a 550 reply if the file does not exist, the modification + time is unavailable, or some other error has occurred. + + mdtm-response = "213" SP time-val CRLF / + error-response + + Note that when the 213 response is issued, that is, when there is no + error, the format MUST be exactly as specified. Multi-line responses + are not permitted. + +3.2. Error Responses + + Where the command is correctly parsed but the modification time is + not available, either because the pathname identifies no existing + entity or because the information is not available for the entity + named, then a 550 reply should be sent. Where the command cannot be + correctly parsed, a 500 or 501 reply should be sent, as specified in + [3]. Various 4xy replies are also possible in appropriate + circumstances. + + + + + + + +Hethmon Standards Track [Page 9] + +RFC 3659 Extensions to FTP March 2007 + + +3.3. FEAT Response for MDTM + + When replying to the FEAT command [6], a server-FTP process that + supports the MDTM command MUST include a line containing the single + word "MDTM". This MAY be sent in upper or lower case or a mixture of + both (it is case insensitive), but SHOULD be transmitted in upper + case only. That is, the response SHOULD be: + + C> Feat + S> 211- + S> ... + S> MDTM + S> ... + S> 211 End + + The ellipses indicate place holders where other features may be + included, but are not required. The one-space indentation of the + feature lines is mandatory [6]. + +3.4. MDTM Examples + + If we assume the existence of three files, A B and C, a directory D, + two files with names that end with the string "ile6", and no other + files at all, then the MDTM command may behave as indicated. The + "C>" lines are commands from user-PI to server-PI, the "S>" lines are + server-PI replies. + + C> MDTM A + S> 213 19980615100045.014 + C> MDTM B + S> 213 19980615100045.014 + C> MDTM C + S> 213 19980705132316 + C> MDTM D + S> 550 D is not retrievable + C> MDTM E + S> 550 No file named "E" + C> mdtm file6 + S> 213 19990929003355 + C> MdTm 19990929043300 File6 + S> 213 19991005213102 + C> MdTm 19990929043300 file6 + S> 550 19990929043300 file6: No such file or directory. + + From that we can conclude that both A and B were last modified at the + same time (to the nearest millisecond), and that C was modified 20 + days and several hours later. + + + + +Hethmon Standards Track [Page 10] + +RFC 3659 Extensions to FTP March 2007 + + + The times are in GMT, so file A was modified on the 15th of June, + 1998, at approximately 11am in London (summer time was then in + effect), or perhaps at 8pm in Melbourne, Australia, or at 6am in New + York. All of those represent the same absolute time, of course. The + location where the file was modified, and consequently the local wall + clock time at that location, is not available. + + There is no file named "E" in the current directory, but there are + files named both "file6" and "19990929043300 File6". The + modification times of those files were obtained. There is no file + named "19990929043300 file6". + +4. File SIZE + + The FTP command, SIZE OF FILE (SIZE), is used to obtain the transfer + size of a file from the server-FTP process. This is the exact number + of octets (8 bit bytes) that would be transmitted over the data + connection should that file be transmitted. This value will change + depending on the current STRUcture, MODE, and TYPE of the data + connection or of a data connection that would be created were one + created now. Thus, the result of the SIZE command is dependent on + the currently established STRU, MODE, and TYPE parameters. + + The SIZE command returns how many octets would be transferred if the + file were to be transferred using the current transfer structure, + mode, and type. This command is normally used in conjunction with + the RESTART (REST) command when STORing a file to a remote server in + STREAM mode, to determine the restart point. The server-PI might + need to read the partially transferred file, do any appropriate + conversion, and count the number of octets that would be generated + when sending the file in order to correctly respond to this command. + Estimates of the file transfer size MUST NOT be returned; only + precise information is acceptable. + +4.1. Syntax + + The syntax of the SIZE command is: + + size = "Size" SP pathname CRLF + + The server-PI will respond to the SIZE command with a 213 reply + giving the transfer size of the file whose pathname was supplied, or + an error response if the file does not exist, the size is + unavailable, or some other error has occurred. The value returned is + in a format suitable for use with the RESTART (REST) command for mode + STREAM, provided the transfer mode and type are not altered. + + + + + +Hethmon Standards Track [Page 11] + +RFC 3659 Extensions to FTP March 2007 + + + size-response = "213" SP 1*DIGIT CRLF / + error-response + + Note that when the 213 response is issued, that is, when there is no + error, the format MUST be exactly as specified. Multi-line responses + are not permitted. + +4.2. Error Responses + + Where the command is correctly parsed but the size is not available, + perhaps because the pathname identifies no existing entity or because + the entity named cannot be transferred in the current MODE and TYPE + (or at all), then a 550 reply should be sent. Where the command + cannot be correctly parsed, a 500 or 501 reply should be sent, as + specified in [3]. The presence of the 550 error response to a SIZE + command MUST NOT be taken by the client as an indication that the + file cannot be transferred in the current MODE and TYPE. A server + may generate this error for other reasons -- for instance if the + processing overhead is considered too great. Various 4xy replies are + also possible in appropriate circumstances. + +4.3. FEAT Response for SIZE + + When replying to the FEAT command [6], a server-FTP process that + supports the SIZE command MUST include a line containing the single + word "SIZE". This word is case insensitive, and MAY be sent in any + mixture of upper or lower case, however it SHOULD be sent in upper + case. That is, the response SHOULD be: + + C> FEAT + S> 211- + S> ... + S> SIZE + S> ... + S> 211 END + + The ellipses indicate place holders where other features may be + included, and are not required. The one-space indentation of the + feature lines is mandatory [6]. + +4.4. Size Examples + + Consider a text file "Example" stored on a Unix(TM) server where each + end of line is represented by a single octet. Assume the file + contains 112 lines, and 1830 octets total. Then the SIZE command + would produce: + + + + + +Hethmon Standards Track [Page 12] + +RFC 3659 Extensions to FTP March 2007 + + + C> TYPE I + S> 200 Type set to I. + C> size Example + S> 213 1830 + C> TYPE A + S> 200 Type set to A. + C> Size Example + S> 213 1942 + + Notice that with TYPE=A the SIZE command reports an extra 112 octets. + Those are the extra octets that need to be inserted, one at the end + of each line, to provide correct end-of-line semantics for a transfer + using TYPE=A. Other systems might need to make other changes to the + transfer format of files when converting between TYPEs and MODEs. + The SIZE command takes all of that into account. + + Since calculating the size of a file with this degree of precision + may take considerable effort on the part of the server-PI, user-PIs + should not used this command unless this precision is essential (such + as when about to restart an interrupted transfer). For other uses, + the "Size" fact of the MLST command (see section 7.5.7) ought be + requested. + +5. Restart of Interrupted Transfer (REST) + + To avoid having to resend the entire file if the file is only + partially transferred, both sides need some way to agree on where in + the data stream to restart the data transfer. + + The FTP specification [3] includes three modes of data transfer, + STREAM, Block, and Compressed. In Block and Compressed modes, the + data stream that is transferred over the data connection is + formatted, allowing the embedding of restart markers into the stream. + The sending DTP can include a restart marker with whatever + information it needs to be able to restart a file transfer at that + point. The receiving DTP can keep a list of these restart markers, + and correlate them with how the file is being saved. To restart the + file transfer, the receiver just sends back that last restart marker, + and both sides know how to resume the data transfer. Note that there + are some flaws in the description of the restart mechanism in STD 9, + RFC 959 [3]. See section 4.1.3.4 of RFC 1123 [9] for the + corrections. + + + + + + + + + +Hethmon Standards Track [Page 13] + +RFC 3659 Extensions to FTP March 2007 + + +5.1. Restarting in STREAM Mode + + In STREAM mode, the data connection contains just a stream of + unformatted octets of data. Explicit restart markers thus cannot be + inserted into the data stream, they would be indistinguishable from + data. For this reason, the FTP specification [3] did not provide the + ability to do restarts in stream mode. However, there is not really + a need to have explicit restart markers in this case, as restart + markers can be implied by the octet offset into the data stream. + + Because the data stream defines the file in STREAM mode, a different + data stream would represent a different file. Thus, an offset will + always represent the same position within a file. On the other hand, + in other modes than STREAM, the same file can be transferred using + quite different octet sequences and yet be reconstructed into the one + identical file. Thus an offset into the data stream in transfer + modes other than STREAM would not give an unambiguous restart point. + + If the data representation TYPE is IMAGE and the STRUcture is File, + for many systems the file will be stored exactly in the same format + as it is sent across the data connection. It is then usually very + easy for the receiver to determine how much data was previously + received, and notify the sender of the offset where the transfer + should be restarted. In other representation types and structures + more effort will be required, but it remains always possible to + determine the offset with finite, but perhaps non-negligible, effort. + In the worst case, an FTP process may need to open a data connection + to itself, set the appropriate transfer type and structure, and + actually transmit the file, counting the transmitted octets. + + If the user-FTP process is intending to restart a retrieve, it will + directly calculate the restart marker and send that information in + the RESTart command. However, if the user-FTP process is intending + to restart sending the file, it needs to be able to determine how + much data was previously sent, and correctly received and saved. A + new FTP command is needed to get this information. This is the + purpose of the SIZE command, as documented in section 4. + +5.2. Error Recovery and Restart + + STREAM mode transfers with FILE STRUcture may be restarted even + though no restart marker has been transferred in addition to the data + itself. This is done by using the SIZE command, if needed, in + combination with the RESTART (REST) command, and one of the standard + file transfer commands. + + When using TYPE ASCII or IMAGE, the SIZE command will return the + number of octets that would actually be transferred if the file were + + + +Hethmon Standards Track [Page 14] + +RFC 3659 Extensions to FTP March 2007 + + + to be sent between the two systems, i.e., with type IMAGE, the SIZE + normally would be the number of octets in the file. With type ASCII, + the SIZE would be the number of octets in the file including any + modifications required to satisfy the TYPE ASCII CR-LF end-of-line + convention. + +5.3. Syntax + + The syntax for the REST command when the current transfer mode is + STREAM is: + + rest = "Rest" SP 1*DIGIT CRLF + + The numeric value gives the number of octets of the immediately- + following transfer to not actually send, effectively causing the + transmission to be restarted at a later point. A value of zero + effectively disables restart, causing the entire file to be + transmitted. The server-PI will respond to the REST command with a + 350 reply, indicating that the REST parameter has been saved, and + that another command, which should be either RETR or STOR, should + then follow to complete the restart. + + rest-response = "350" SP *TCHAR CRLF / + error-response + + Server-FTP processes may permit transfer commands other than RETR and + STOR, such as APPE and STOU, to complete a restart; however, this is + not recommended. STOU (store unique) is undefined in this usage, as + storing the remainder of a file into a unique file name is rarely + going to be useful. If APPE (append) is permitted, it MUST act + identically to STOR when a restart marker has been set. That is, in + both cases, octets from the data connection are placed into the file + at the location indicated by the restart marker value. + + The REST command is intended to complete a failed transfer. Use with + RETR is comparatively well defined in all cases, as the client bears + the responsibility of merging the retrieved data with the partially + retrieved file. It may choose to use the data obtained other than to + complete an earlier transfer, or to re-retrieve data that had been + retrieved before. With STOR, however, the server must insert the + data into the file named. The results are undefined if a client uses + REST to do other than restart to complete a transfer of a file that + had previously failed to completely transfer. In particular, if the + restart marker set with a REST command is not at the end of the data + currently stored at the server, as reported by the server, or if + insufficient data are provided in a STOR that follows a REST to + extend the destination file to at least its previous size, then the + effects are undefined. + + + +Hethmon Standards Track [Page 15] + +RFC 3659 Extensions to FTP March 2007 + + + The REST command must be the last command issued before the data + transfer command that is to cause a restarted, rather than a + complete, file transfer. The effect of issuing a REST command at any + other time is undefined. The server-PI may react to a badly + positioned REST command by issuing an error response to the following + command, not being a restartable data transfer command, or it may + save the restart value and apply it to the next data transfer + command, or it may silently ignore the inappropriate restart attempt. + Because of this, a user-PI that has issued a REST command, but that + has not successfully transmitted the following data transfer command + for any reason, should send another REST command before the next data + transfer command. If that transfer is not to be restarted, then + "REST 0" should be issued. + + An error response will follow a REST command only when the server + does not implement the command, or when the restart marker value is + syntactically invalid for the current transfer mode (e.g., in STREAM + mode, something other than one or more digits appears in the + parameter to the REST command). Any other errors, including such + problems as restart marker out of range, should be reported when the + following transfer command is issued. Such errors will cause that + transfer request to be rejected with an error indicating the invalid + restart attempt. + +5.4. FEAT Response for REST + + Where a server-FTP process supports RESTart in STREAM mode, as + specified here, it MUST include, in the response to the FEAT command + [6], a line containing exactly the string "REST STREAM". This string + is not case sensitive, but it SHOULD be transmitted in upper case. + Where REST is not supported at all or supported only in block or + compressed modes, the REST line MUST NOT be included in the FEAT + response. Where required, the response SHOULD be: + + C> feat + S> 211- + S> ... + S> REST STREAM + S> ... + S> 211 end + + The ellipses indicate place holders where other features may be + included, and are not required. The one-space indentation of the + feature lines is mandatory [6]. + + + + + + + +Hethmon Standards Track [Page 16] + +RFC 3659 Extensions to FTP March 2007 + + +5.5. REST Example + + Assume that the transfer of a largish file has previously been + interrupted after 802816 octets had been received, that the previous + transfer was with TYPE=I, and that it has been verified that the file + on the server has not since changed. + + C> TYPE I + S> 200 Type set to I. + C> PORT 127,0,0,1,15,107 + S> 200 PORT command successful. + C> REST 802816 + S> 350 Restarting at 802816. Send STORE or RETRIEVE + C> RETR cap60.pl198.tar + S> 150 Opening BINARY mode data connection + [...] + S> 226 Transfer complete. + +6. A Trivial Virtual File Store (TVFS) + + Traditionally, FTP has placed almost no constraints upon the file + store (NVFS) provided by a server. This specification does not alter + that. However, it has become common for servers to attempt to + provide at least file system naming conventions modeled loosely upon + those of the UNIX(TM) file system. This is a tree-structured file + system, built of directories, each of which can contain other + directories, or other kinds of files, or both. Each file and + directory has a name relative to the directory that contains it, + except for the directory at the root of the tree, which is contained + in no other directory, and hence has no name of its own. + + That which has so far been described is perfectly consistent with the + standard FTP NVFS and access mechanisms. The "CWD" command is used + to move from one directory to an embedded directory. "CDUP" may be + provided to return to the parent directory, and the various file + manipulation commands ("RETR", "STOR", the rename commands, etc.) are + used to manipulate files within the current directory. + + However, it is often useful to be able to reference files other than + by changing directories, especially as FTP provides no guaranteed + mechanism to return to a previous directory. The Trivial Virtual + File Store (TVFS), if implemented, provides that mechanism. + + + + + + + + + +Hethmon Standards Track [Page 17] + +RFC 3659 Extensions to FTP March 2007 + + +6.1. TVFS File Names + + Where a server implements the TVFS, no elementary file name shall + contain the character "/". Where the underlying natural file store + permits files, or directories, to contain the "/" character in their + names, a server-PI implementing TVFS must encode that character in + some manner whenever file or directory names are being returned to + the user-PI, and reverse that encoding whenever such names are being + accepted from the user-PI. + + The encoding method to be used is not specified here. Where some + other character is illegal in file and directory names in the + underlying file store, a simple transliteration may be sufficient. + Where there is no suitable substitute character a more complex + encoding scheme, possibly using an escape character, is likely to be + required. + + With the one exception of the unnamed root directory, a TVFS file + name may not be empty. That is, all other file names contain at + least one character. + + With the sole exception of the "/" character, any valid IS10646 + character [10] may be used in a TVFS file name. When transmitted, + file name characters are encoded using the UTF-8 encoding [2]. Note + that the two-character sequence CR LF occurring in a file name will + make that name impossible to transmit over a data connection. + Consequently, it should be avoided, or if that is impossible to + achieve, it MUST be encoded in some reversible way. + +6.2. TVFS Pathnames + + A TVFS "Pathname" combines the file or directory name of a target + file or directory, with the directory names of zero or more enclosing + directories, so as to allow the target file or directory to be + referenced other than when the server's "current working directory" + is the directory directly containing the target file or directory. + + By definition, every TVFS file or directory name is also a TVFS + pathname. Such a pathname is valid to reference the file from the + directory containing the name, that is, when that directory is the + server-FTP's current working directory. + + Other TVFS pathnames are constructed by prefixing a pathname by a + name of a directory from which the path is valid, and separating the + two with the "/" character. Such a pathname is valid to reference + the file or directory from the directory containing the newly added + directory name. + + + + +Hethmon Standards Track [Page 18] + +RFC 3659 Extensions to FTP March 2007 + + + Where a pathname has been extended to the point where the directory + added is the unnamed root directory, the pathname will begin with the + "/" character. Such a path is known as a fully qualified pathname. + Fully qualified paths may, obviously, not be further extended, as, by + definition, no directory contains the root directory. Being unnamed, + it cannot be represented in any other directory. A fully qualified + pathname is valid to reference the named file or directory from any + location (that is, regardless of what the current working directory + may be) in the virtual file store. + + Any pathname that is not a fully qualified pathname may be referred + to as a "relative pathname" and will only correctly reference the + intended file when the current working directory of the server-FTP is + a directory from which the relative pathname is valid. + + As a special case, the pathname "/" is defined to be a fully + qualified pathname referring to the root directory. That is, the + root directory does not have a directory (or file) name, but does + have a pathname. This special pathname may be used only as is as a + reference to the root directory. It may not be combined with other + pathnames using the rules above, as doing so would lead to a pathname + containing two consecutive "/" characters, which is an undefined + sequence. + +6.2.1. Notes + + + It is not required, or expected, that there be only one fully + qualified pathname that will reference any particular file or + directory. + + + As a caveat, though the TVFS file store is basically tree + structured, there is no requirement that any file or directory + have only one parent directory. + + + As defined, no TVFS pathname will ever contain two consecutive "/" + characters. Such a name is not illegal however, and may be + defined by the server for any purpose that suits it. Clients + implementing this specification should not assume any semantics + for such names. + + + Similarly, other than the special case path that refers to the + root directory, no TVFS pathname constructed as defined here will + ever end with the "/" character. Such names are also not illegal, + but are undefined. + + + While any legal IS10646 character is permitted to occur in a TVFS + file or directory name, other than "/", server FTP implementations + are not required to support all possible IS10646 characters. The + + + +Hethmon Standards Track [Page 19] + +RFC 3659 Extensions to FTP March 2007 + + + subset supported is entirely at the discretion of the server. The + case (where it exists) of the characters that make up file, + directory, and pathnames may be significant. Unless determined + otherwise by means unspecified here, clients should assume that + all such names are comprised of characters whose case is + significant. Servers are free to treat case (or any other + attribute) of a name as irrelevant, and hence map two names that + appear to be distinct onto the same underlying file. + + + There are no defined "magic" names, like ".", ".." or "C:". + Servers may implement such names, with any semantics they choose, + but are not required to do so. + + + TVFS imposes no particular semantics or properties upon files, + guarantees no access control schemes, or any of the other common + properties of a file store. Only the naming scheme is defined. + +6.3. FEAT Response for TVFS + + In response to the FEAT command [6] a server that wishes to indicate + support for the TVFS as defined here will include a line that begins + with the four characters "TVFS" (in any case, or mixture of cases, + upper case is not required). Servers SHOULD send upper case. + + Such a response to the FEAT command MUST NOT be returned unless the + server implements TVFS as defined here. + + Later specifications may add to the TVFS definition. Such additions + should be notified by means of additional text appended to the TVFS + feature line. Such specifications, if any, will define the extra + text. + + Until such a specification is defined, servers should not include + anything after "TVFS" in the TVFS feature line. Clients, however, + should be prepared to deal with arbitrary text following the four + defined characters, and simply ignore it if unrecognized. + + A typical response to the FEAT command issued by a server + implementing only this specification would be: + + C> feat + S> 211- + S> ... + S> TVFS + S> ... + S> 211 end + + + + + +Hethmon Standards Track [Page 20] + +RFC 3659 Extensions to FTP March 2007 + + + The ellipses indicate place holders where other features may be + included, but are not required. The one-space indentation of the + feature lines is mandatory [6] and is not counted as one of the first + four characters for the purposes of this feature listing. + + The TVFS feature adds no new commands to the FTP command repertoire. + +6.4. OPTS for TVFS + + There are no options in this TVFS specification, and hence there is + no OPTS command defined. + +6.5. TVFS Examples + + Assume a TVFS file store is comprised of a root directory, which + contains two directories (A and B) and two non-directory files (X and + Y). The A directory contains two directories (C and D) and one other + file (Z). The B directory contains just two non-directory files (P + and Q) and the C directory also two non-directory files (also named P + and Q, by chance). The D directory is empty, that is, contains no + files or directories. This structure may depicted graphically as... + + (unnamed root) + / | \ \ + / | \ \ + A X B Y + /|\ / \ + / | \ / \ + C D Z P Q + / \ + / \ + P Q + + Given this structure, the following fully qualified pathnames exist. + + / + /A + /B + /X + /Y + /A/C + /A/D + /A/Z + /A/C/P + /A/C/Q + /B/P + /B/Q + + + + +Hethmon Standards Track [Page 21] + +RFC 3659 Extensions to FTP March 2007 + + + It is clear that none of the paths / /A /B or /A/D refer to the same + directory, as the contents of each is different. Nor do any of / /A + /A/C or /A/D. However /A/C and /B might be the same directory, there + is insufficient information given to tell. Any of the other + pathnames (/X /Y /A/Z /A/C/P /A/C/Q /B/P and /B/Q) may refer to the + same underlying files, in almost any combination. + + If the current working directory of the server-FTP is /A then the + following pathnames, in addition to all the fully qualified + pathnames, are valid + + C + D + Z + C/P + C/Q + + These all refer to the same files or directories as the corresponding + fully qualified path with "/A/" prepended. + + That those pathnames all exist does not imply that the TVFS sever + will necessarily grant any kind of access rights to the named paths, + or that access to the same file via different pathnames will + necessarily be granted equal rights. + + None of the following relative paths are valid when the current + directory is /A + + A + B + X + Y + B/P + B/Q + P + Q + + Any of those could be made valid by changing the server-FTP's current + working directory to the appropriate directory. Note that the paths + "P" and "Q" might refer to different files depending upon which + directory is selected to cause those to become valid TVFS relative + paths. + + + + + + + + + +Hethmon Standards Track [Page 22] + +RFC 3659 Extensions to FTP March 2007 + + +7. Listings for Machine Processing (MLST and MLSD) + + The MLST and MLSD commands are intended to standardize the file and + directory information returned by the server-FTP process. These + commands differ from the LIST command in that the format of the + replies is strictly defined although extensible. + + Two commands are defined, MLST and MLSD. MLST provides data about + exactly the object named on its command line, and no others. MLSD, + on the other, lists the contents of a directory if a directory is + named, otherwise a 501 reply is returned. In either case, if no + object is named, the current directory is assumed. That will cause + MLST to send a one-line response, describing the current directory + itself, and MLSD to list the contents of the current directory. + + In the following, the term MLSx will be used wherever either MLST or + MLSD may be inserted. + + The MLST and MLSD commands also extend the FTP protocol as presented + in STD 9, RFC 959 [3] and STD 3, RFC 1123 [9] to allow that + transmission of 8-bit data over the control connection. Note this is + not specifying character sets which are 8-bit, but specifying that + FTP implementations are to specifically allow the transmission and + reception of 8-bit bytes, with all bits significant, over the control + connection. That is, all 256 possible octet values are permitted. + The MLSx command allows both UTF-8/Unicode and "raw" forms as + arguments, and in responses both to the MLST and MLSD commands, and + all other FTP commands which take pathnames as arguments. + +7.1. Format of MLSx Requests + + The MLST and MLSD commands each allow a single optional argument. + This argument may be either a directory name or, for MLST only, a + file name. For these purposes, a "file name" is the name of any + entity in the server NVFS which is not a directory. Where TVFS is + supported, any TVFS relative pathname valid in the current working + directory, or any TVFS fully qualified pathname, may be given. If a + directory name is given then MLSD must return a listing of the + contents of the named directory, otherwise it issues a 501 reply, and + does not open a data connection. In all cases for MLST, a single set + of fact lines (usually a single fact line) containing the information + about the named file or directory shall be returned over the control + connection, without opening a data connection. + + If no argument is given then MLSD must return a listing of the + contents of the current working directory, and MLST must return a + listing giving information about the current working directory + itself. For these purposes, the contents of a directory are whatever + + + +Hethmon Standards Track [Page 23] + +RFC 3659 Extensions to FTP March 2007 + + + file or directory names (not pathnames) the server-PI will allow to + be referenced when the current working directory is the directory + named, and which the server-PI desires to reveal to the user-PI. + Note that omitting the argument is the only defined way to obtain a + listing of the current directory, unless a pathname that represents + the directory happens to be known. In particular, there is no + defined shorthand name for the current directory. This does not + prohibit any particular server-PI implementing such a shorthand. + + No title, header, or summary, lines, or any other formatting, other + than as is specified below, is ever returned in the output of an MLST + or MLSD command. + + If the Client-FTP sends an invalid argument, the server-FTP MUST + reply with an error code of 501. + + The syntax for the MLSx command is: + + mlst = "MLst" [ SP pathname ] CRLF + mlsd = "MLsD" [ SP pathname ] CRLF + +7.2. Format of MLSx Response + + The format of a response to an MLSx command is as follows: + + mlst-response = control-response / error-response + mlsd-response = ( initial-response final-response ) / + error-response + + control-response = "250-" [ response-message ] CRLF + 1*( SP entry CRLF ) + "250" [ SP response-message ] CRLF + + initial-response = "150" [ SP response-message ] CRLF + final-response = "226" SP response-message CRLF + + response-message = *TCHAR + + data-response = *( entry CRLF ) + + entry = [ facts ] SP pathname + facts = 1*( fact ";" ) + fact = factname "=" value + factname = "Size" / "Modify" / "Create" / + "Type" / "Unique" / "Perm" / + "Lang" / "Media-Type" / "CharSet" / + os-depend-fact / local-fact + os-depend-fact = "." token + + + +Hethmon Standards Track [Page 24] + +RFC 3659 Extensions to FTP March 2007 + + + local-fact = "X." token + value = *SCHAR + + Upon receipt of an MLSx command, the server will verify the + parameter, and if invalid return an error-response. For this + purpose, the parameter should be considered to be invalid if the + client issuing the command does not have permission to perform the + requested operation. + + If the parameter is valid, then for an MLST command, the server-PI + will send the first (leading) line of the control response, the entry + for the pathname given, or the current directory if no pathname was + provided, and the terminating line. Normally exactly one entry would + be returned, more entries are permitted only when required to + represent a file that is to have multiple "Type" facts returned. In + this case, the pathname component of every response MUST be + identical. + + Note that for MLST the fact set is preceded by a space. That is + provided to guarantee that the fact set cannot be accidentally + interpreted as the terminating line of the control response, but is + required even when that would not be possible. Exactly one space + exists between the set of facts and the pathname. Where no facts are + present, there will be exactly two leading spaces before the + pathname. No spaces are permitted in the facts, any other spaces in + the response are to be treated as being a part of the pathname. + + If the command was an MLSD command, the server will open a data + connection as indicated in section 3.2 of STD 9, RFC 959 [3]. If + that fails, the server will return an error-response. If all is OK, + the server will return the initial-response, send the appropriate + data-response over the new data connection, close that connection, + and then send the final-response over the control connection. The + grammar above defines the format for the data-response, which defines + the format of the data returned over the data connection established. + + The data connection opened for a MLSD response shall be a connection + as if the "TYPE L 8", "MODE S", and "STRU F" commands had been given, + whatever FTP transfer type, mode and structure had actually been set, + and without causing those settings to be altered for future commands. + That is, this transfer type shall be set for the duration of the data + connection established for this command only. While the content of + the data sent can be viewed as a series of lines, implementations + should note that there is no maximum line length defined. + Implementations should be prepared to deal with arbitrarily long + lines. + + + + + +Hethmon Standards Track [Page 25] + +RFC 3659 Extensions to FTP March 2007 + + + The facts part of the specification would contain a series of "file + facts" about the file or directory named on the same line. Typical + information to be presented would include file size, last + modification time, creation time, a unique identifier, and a + file/directory flag. + + The complete format for a successful reply to the MLSD command would + be: + + facts SP pathname CRLF + facts SP pathname CRLF + facts SP pathname CRLF + ... + + Note that the format is intended for machine processing, not human + viewing, and as such the format is very rigid. Implementations MUST + NOT vary the format by, for example, inserting extra spaces for + readability, replacing spaces by tabs, including header or title + lines, or inserting blank lines, or in any other way alter this + format. Exactly one space is always required after the set of facts + (which may be empty). More spaces may be present on a line if, and + only if, the pathname presented contains significant spaces. The set + of facts must not contain any spaces anywhere inside it. Facts + should be provided in each output line only if they both provide + relevant information about the file named on the same line, and they + are in the set requested by the user-PI. See section 7.9 (page 51). + There is no requirement that the same set of facts be provided for + each file, or that the facts presented occur in the same order for + each file. + +7.2.1. Error Responses to MLSx commands + + Many of the 4xy and 5xy responses defined in section 4.2 of STD 9, + RFC 959 [3] are possible in response to the MLST and MLSD commands. + In particular, syntax errors can generate 500 or 501 replies. Giving + a pathname that exists but is not a directory as the argument to a + MLSD command generates a 501 reply. Giving a name that does not + exist, or for which access permission (to obtain directory + information as requested) is not granted will elicit a 550 reply. + Other replies (530, 553, 503, 504, and any of the 4xy replies) are + also possible in appropriate circumstances. + +7.3. File Name Encoding + + An FTP implementation supporting the MLSx commands must be 8-bit + clean. This is necessary in order to transmit UTF-8 encoded file + names. This specification recommends the use of UTF-8 encoded file + + + + +Hethmon Standards Track [Page 26] + +RFC 3659 Extensions to FTP March 2007 + + + names. FTP implementations SHOULD use UTF-8 whenever possible to + encourage the maximum inter-operability. + + File names are not restricted to UTF-8, however treatment of + arbitrary character encodings is not specified by this standard. + Applications are encouraged to treat non-UTF-8 encodings of file + names as octet sequences. + + Note that this encoding is unrelated to that of the contents of the + file, even if the file contains character data. + + Further information about file name encoding for FTP may be found in + "Internationalization of the File Transfer Protocol" [7]. + +7.3.1. Notes about the File Name + + The file name returned in the MLST response should be the same name + as was specified in the MLST command, or, where TVFS is supported, a + fully qualified TVFS path naming the same file. Where no argument + was given to the MLST command, the server-PI may either include an + empty file name in the response, or it may supply a name that refers + to the current directory, if such a name is available. Where TVFS is + supported, a fully qualified pathname of the current directory SHOULD + be returned. + + File names returned in the output from an MLSD command SHOULD be + unqualified names within the directory named, or the current + directory if no argument was given. That is, the directory named in + the MLSD command SHOULD NOT appear as a component of the file names + returned. + + If the server-FTP process is able, and the "type" fact is being + returned, it MAY return in the MLSD response, an entry whose type is + "cdir", which names the directory from which the contents of the + listing were obtained. Where TVFS is supported, the name MAY be the + fully qualified pathname of the directory, or MAY be any other + pathname that is valid to refer to that directory from the current + working directory of the server-FTP. Where more than one name + exists, multiple of these entries may be returned. In a sense, the + "cdir" entry can be viewed as a heading for the MLSD output. + However, it is not required to be the first entry returned, and may + occur anywhere within the listing. + + When TVFS is supported, a user-PI can refer to any file or directory + in the listing by combining a type "cdir" name, with the appropriate + name from the directory listing using the procedure defined in + section 6.2. + + + + +Hethmon Standards Track [Page 27] + +RFC 3659 Extensions to FTP March 2007 + + + Alternatively, whether TVFS is supported or not, the user-PI can + issue a CWD command ([3]) giving a name of type "cdir" from the + listing returned, and from that point reference the files returned in + the MLSD response from which the cdir was obtained by using the file + name components of the listing. + +7.4. Format of Facts + + The "facts" for a file in a reply to a MLSx command consist of + information about that file. The facts are a series of keyword=value + pairs each followed by semi-colon (";") characters. An individual + fact may not contain a semi-colon in its name or value. The complete + series of facts may not contain the space character. See the + definition or "RCHAR" in section 2.1 for a list of the characters + that can occur in a fact value. Not all are applicable to all facts. + + A sample of a typical series of facts would be: (spread over two + lines for presentation here only) + + size=4161;lang=en-US;modify=19970214165800;create=19961001124534; + type=file;x.myfact=foo,bar; + +7.5. Standard Facts + + This document defines a standard set of facts as follows: + + size -- Size in octets + modify -- Last modification time + create -- Creation time + type -- Entry type + unique -- Unique id of file/directory + perm -- File permissions, whether read, write, execute is + allowed for the login id. + lang -- Language of the file name per IANA [11] registry. + media-type -- MIME media-type of file contents per IANA registry. + charset -- Character set per IANA registry (if not UTF-8) + + Fact names are case-insensitive. Size, size, SIZE, and SiZe are the + same fact. + + Further operating system specific keywords could be specified by + using the IANA operating system name as a prefix (examples only): + + OS/2.ea -- OS/2 extended attributes + MACOS.rf -- MacIntosh resource forks + UNIX.mode -- Unix file modes (permissions) + + + + + +Hethmon Standards Track [Page 28] + +RFC 3659 Extensions to FTP March 2007 + + + Implementations may define keywords for experimental, or private use. + All such keywords MUST begin with the two character sequence "x.". + As type names are case independent, "x." and "X." are equivalent. + For example: + + x.ver -- Version information + x.desc -- File description + x.type -- File type + +7.5.1. The Type Fact + + The type fact needs a special description. Part of the problem with + current practices is deciding when a file is a directory. If it is a + directory, is it the current directory, a regular directory, or a + parent directory? The MLST specification makes this unambiguous + using the type fact. The type fact given specifies information about + the object listed on the same line of the MLST response. + + Five values are possible for the type fact: + + file -- a file entry + cdir -- the listed directory + pdir -- a parent directory + dir -- a directory or sub-directory + OS.name=type -- an OS or file system dependent file type + + The syntax is defined to be: + + type-fact = type-label "=" type-val + type-label = "Type" + type-val = "File" / "cdir" / "pdir" / "dir" / + os-type + + The value of the type fact (the "type-val") is a case independent + string. + +7.5.1.1. type=file + + The presence of the type=file fact indicates the listed entry is a + file containing non-system data. That is, it may be transferred from + one system to another of quite different characteristics, and perhaps + still be meaningful. + +7.5.1.2. type=cdir + + The type=cdir fact indicates the listed entry contains a pathname of + the directory whose contents are listed. An entry of this type will + only be returned as a part of the result of an MLSD command when the + + + +Hethmon Standards Track [Page 29] + +RFC 3659 Extensions to FTP March 2007 + + + type fact is included, and provides a name for the listed directory, + and facts about that directory. In a sense, it can be viewed as + representing the title of the listing, in a machine friendly format. + It may appear at any point of the listing, it is not restricted to + appearing at the start, though frequently may do so, and may occur + multiple times. It MUST NOT be included if the type fact is not + included, or there would be no way for the user-PI to distinguish the + name of the directory from an entry in the directory. + + Where TVFS is supported by the server-FTP, this name may be used to + construct pathnames with which to refer to the files and directories + returned in the same MLSD output (see section 6.2). These pathnames + are only expected to work when the server-PI's position in the NVFS + file tree is the same as its position when the MLSD command was + issued, unless a fully qualified pathname results. + + Where TVFS is not supported, the only defined semantics associated + with a "type=cdir" entry are that, provided the current working + directory of the server-PI has not been changed, a pathname of type + "cdir" may be used as an argument to a CWD command, which will cause + the current directory of the server-PI to change so that the + directory that was listed in its current working directory. + +7.5.1.3. type=dir + + If present, the type=dir entry gives the name of a directory. Such + an entry typically cannot be transferred from one system to another + using RETR, etc., but should (permissions permitting) be able to be + the object of an MLSD command. + +7.5.1.4. type=pdir + + If present, which will occur only in the response to a MLSD command + when the type fact is included, the type=pdir entry represents a + pathname of the parent directory of the listed directory. As well as + having the properties of a type=dir, a CWD command that uses the + pathname from this entry should change the user to a parent directory + of the listed directory. If the listed directory is the current + directory, a CDUP command may also have the effect of changing to the + named directory. User-FTP processes should note not all responses + will include this information, and that some systems may provide + multiple type=pdir responses. + + Where TVFS is supported, a "type=pdir" name may be a relative + pathname, or a fully qualified pathname. A relative pathname will be + relative to the directory being listed, not to the current directory + of the server-PI at the time. + + + + +Hethmon Standards Track [Page 30] + +RFC 3659 Extensions to FTP March 2007 + + + For the purposes of this type value, a "parent directory" is any + directory in which there is an entry of type=dir that refers to the + directory in which the type=pdir entity was found. Thus it is not + required that all entities with type=pdir refer to the same + directory. The "unique" fact (if supported and supplied) can be used + to determine whether there is a relationship between the type=pdir + entries or not. + +7.5.1.5. System Defined Types + + Files types that are specific to a specific operating system, or file + system, can be encoded using the "OS." type names. The format is: + + os-type = "OS." os-name "=" os-kind + os-name = + os-kind = token + + The "os-name" indicates the specific system type that supports the + particular localtype. OS specific types are registered by the IANA + using the procedures specified in section 10. The "os-kind" provides + the system dependent information as to the type of the file listed. + The os-name and os-kind strings in an os-type are case independent. + "OS.unix=block" and "OS.Unix=BLOCK" represent the same type (or + would, if such a type were registered.) + + Note: Where the underlying system supports a file type that is + essentially an indirect pointer to another file, the NVFS + representation of that type should normally be to represent the file + that the reference indicates. That is, the underlying basic file + will appear more than once in the NVFS, each time with the "unique" + fact (see immediately following section) containing the same value, + indicating that the same file is represented by all such names. + User-PIs transferring the file need then transfer it only once, and + then insert their own form of indirect reference to construct + alternate names where desired, or perhaps even copy the local file if + that is the only way to provide two names with the same content. A + file which would be a reference to another file, if only the other + file actually existed, may be represented in any OS dependent manner + appropriate, or not represented at all. + +7.5.1.6. Multiple Types + + Where a file is such that it may validly, and sensibly, treated by + the server-PI as being of more than one of the above types, then + multiple entries should be returned, each with its own "Type" fact of + the appropriate type, and each containing the same pathname. This + may occur, for example, with a structured file, which may contain + sub-files, and where the server-PI permits the structured file to be + + + +Hethmon Standards Track [Page 31] + +RFC 3659 Extensions to FTP March 2007 + + + treated as a unit, or treated as a directory allowing the sub-files + within it to be referenced. When this is done, the pathname returned + with each entry MUST be identical to the others representing the same + file. + +7.5.2. The unique Fact + + The unique fact is used to present a unique identifier for a file or + directory in the NVFS accessed via a server-FTP process. The value + of this fact should be the same for any number of pathnames that + refer to the same underlying file. The fact should have different + values for names that reference distinct files. The mapping between + files, and unique fact tokens should be maintained, and remain + consistent, for at least the lifetime of the control connection from + user-PI to server-PI. + + unique-fact = "Unique" "=" token + + This fact would be expected to be used by server-FTPs whose host + system allows things such as symbolic links so that the same file may + be represented in more than one directory on the server. The only + conclusion that should be drawn is that if two different names each + have the same value for the unique fact, they refer to the same + underlying object. The value of the unique fact (the token) should + be considered an opaque string for comparison purposes, and is a case + dependent value. The tokens "A" and "a" do not represent the same + underlying object. + +7.5.3. The modify Fact + + The modify fact is used to determine the last time the content of the + file (or directory) indicated was modified. Any change of substance + to the file should cause this value to alter. That is, if a change + is made to a file such that the results of a RETR command would + differ, then the value of the modify fact should alter. User-PIs + should not assume that a different modify fact value indicates that + the file contents are necessarily different than when last retrieved. + Some systems may alter the value of the modify fact for other + reasons, though this is discouraged wherever possible. Also a file + may alter, and then be returned to its previous content, which would + often be indicated as two incremental alterations to the value of the + modify fact. + + For directories, this value should alter whenever a change occurs to + the directory such that different file names would (or might) be + included in MLSD output of that directory. + + modify-fact = "Modify" "=" time-val + + + +Hethmon Standards Track [Page 32] + +RFC 3659 Extensions to FTP March 2007 + + +7.5.4. The create Fact + + The create fact indicates when a file, or directory, was first + created. Exactly what "creation" is for this purpose is not + specified here, and may vary from server to server. About all that + can be said about the value returned is that it can never indicate a + later time than the modify fact. + + create-fact = "Create" "=" time-val + + Implementation Note: Implementors of this fact on UNIX(TM) systems + should note that the unix "stat" "st_ctime" field does not give + creation time, and that unix file systems do not record creation + time at all. Unix (and POSIX) implementations will normally not + include this fact. + +7.5.5. The perm Fact + + The perm fact is used to indicate access rights the current FTP user + has over the object listed. Its value is always an unordered + sequence of alphabetic characters. + + perm-fact = "Perm" "=" *pvals + pvals = "a" / "c" / "d" / "e" / "f" / + "l" / "m" / "p" / "r" / "w" + + There are ten permission indicators currently defined. Many are + meaningful only when used with a particular type of object. The + indicators are case independent, "d" and "D" are the same indicator. + + The "a" permission applies to objects of type=file, and indicates + that the APPE (append) command may be applied to the file named. + + The "c" permission applies to objects of type=dir (and type=pdir, + type=cdir). It indicates that files may be created in the directory + named. That is, that a STOU command is likely to succeed, and that + STOR and APPE commands might succeed if the file named did not + previously exist, but is to be created in the directory object that + has the "c" permission. It also indicates that the RNTO command is + likely to succeed for names in the directory. + + The "d" permission applies to all types. It indicates that the + object named may be deleted, that is, that the RMD command may be + applied to it if it is a directory, and otherwise that the DELE + command may be applied to it. + + The "e" permission applies to the directory types. When set on an + object of type=dir, type=cdir, or type=pdir it indicates that a CWD + + + +Hethmon Standards Track [Page 33] + +RFC 3659 Extensions to FTP March 2007 + + + command naming the object should succeed, and the user should be able + to enter the directory named. For type=pdir it also indicates that + the CDUP command may succeed (if this particular pathname is the one + to which a CDUP would apply.) + + The "f" permission for objects indicates that the object named may be + renamed - that is, may be the object of an RNFR command. + + The "l" permission applies to the directory file types, and indicates + that the listing commands, LIST, NLST, and MLSD may be applied to the + directory in question. + + The "m" permission applies to directory types, and indicates that the + MKD command may be used to create a new directory within the + directory under consideration. + + The "p" permission applies to directory types, and indicates that + objects in the directory may be deleted, or (stretching naming a + little) that the directory may be purged. Note: it does not indicate + that the RMD command may be used to remove the directory named + itself, the "d" permission indicator indicates that. + + The "r" permission applies to type=file objects, and for some + systems, perhaps to other types of objects, and indicates that the + RETR command may be applied to that object. + + The "w" permission applies to type=file objects, and for some + systems, perhaps to other types of objects, and indicates that the + STOR command may be applied to the object named. + + Note: That a permission indicator is set can never imply that the + appropriate command is guaranteed to work -- just that it might. + Other system specific limitations, such as limitations on + available space for storing files, may cause an operation to fail, + where the permission flags may have indicated that it was likely + to succeed. The permissions are a guide only. + + Implementation note: The permissions are described here as they apply + to FTP commands. They may not map easily into particular + permissions available on the server's operating system. Servers + are expected to synthesize these permission bits from the + permission information available from operating system. For + example, to correctly determine whether the "D" permission bit + should be set on a directory for a server running on the UNIX(TM) + operating system, the server should check that the directory named + is empty, and that the user has write permission on both the + directory under consideration, and its parent directory. + + + + +Hethmon Standards Track [Page 34] + +RFC 3659 Extensions to FTP March 2007 + + + Some systems may have more specific permissions than those listed + here, such systems should map those to the flags defined as best + they are able. Other systems may have only more broad access + controls. They will generally have just a few possible + permutations of permission flags, however they should attempt to + correctly represent what is permitted. + +7.5.6. The lang Fact + + The lang fact describes the natural language of the file name for use + in display purposes. Values used here should be taken from the + language registry of the IANA. See [12] for the syntax, and + procedures, related to language tags. + + lang-fact = "Lang" "=" token + + Server-FTP implementations MUST NOT guess language values. Language + values must be determined in an unambiguous way such as file system + tagging of language or by user configuration. Note that the lang + fact provides no information at all about the content of a file, only + about the encoding of its name. + +7.5.7. The size Fact + + The size fact applies to non-directory file types and should always + reflect the approximate size of the file. This should be as accurate + as the server can make it, without going to extraordinary lengths, + such as reading the entire file. The size is expressed in units of + octets of data in the file. + + Given limitations in some systems, Client-FTP implementations must + understand this size may not be precise and may change between the + time of a MLST and RETR operation. + + Clients that need highly accurate size information for some + particular reason should use the SIZE command as defined in section + 4. The most common need for this accuracy is likely to be in + conjunction with the REST command described in section 5. The size + fact, on the other hand, should be used for purposes such as + indicating to a human user the approximate size of the file to be + transferred, and perhaps to give an idea of expected transfer + completion time. + + size-fact = "Size" "=" 1*DIGIT + + + + + + + +Hethmon Standards Track [Page 35] + +RFC 3659 Extensions to FTP March 2007 + + +7.5.8. The media-type Fact + + The media-type fact represents the IANA media type of the file named, + and applies only to non-directory types. The list of values used + must follow the guidelines set by the IANA registry. + + media-type = "Media-Type" "=" + + Server-FTP implementations MUST NOT guess media type values. Media + type values must be determined in an unambiguous way such as file + system tagging of media-type or by user configuration. This fact + gives information about the content of the file named. Both the + primary media type, and any appropriate subtype should be given, + separated by a slash "/" as is traditional. + +7.5.9. The charset Fact + + The charset fact provides the IANA character set name, or alias, for + the encoded pathnames in a MLSx response. The default character set + is UTF-8 unless specified otherwise. FTP implementations SHOULD use + UTF-8 if possible to encourage maximum inter-operability. The value + of this fact applies to the pathname only, and provides no + information about the contents of the file. + + charset-type = "Charset" "=" token + +7.5.10. Required Facts + + Servers are not required to support any particular set of the + available facts. However, servers SHOULD, if conceivably possible, + support at least the type, perm, size, unique, and modify facts. + +7.6. System Dependent and Local Facts + + By using an system dependent fact, or a local fact, a server-PI may + communicate to the user-PI information about the file named that is + peculiar to the underlying file system. + +7.6.1. System Dependent Facts + + System dependent fact names are labeled by prefixing a label + identifying the specific information returned by the name of the + appropriate operating system from the IANA maintained list of + operating system names. + + The value of an OS dependent fact may be whatever is appropriate to + convey the information available. It must be encoded as a "token" as + defined in section 2.1 however. + + + +Hethmon Standards Track [Page 36] + +RFC 3659 Extensions to FTP March 2007 + + + In order to allow reliable inter-operation between users of system + dependent facts, the IANA will maintain a registry of system + dependent fact names, their syntax, and the interpretation to be + given to their values. Registrations of system dependent facts are + to be accomplished according to the procedures of section 10. + +7.6.2. Local Facts + + Implementations may also make available other facts of their own + choosing. As the method of interpretation of such information will + generally not be widely understood, server-PIs should be aware that + clients will typically ignore any local facts provided. As there is + no registration of locally defined facts, it is entirely possible + that different servers will use the same local fact name to provide + vastly different information. Hence user-PIs should be hesitant + about making any use of any information in a locally defined fact + without some other specific assurance that the particular fact is one + that they do comprehend. + + Local fact names all begin with the sequence "X.". The rest of the + name is a "token" (see section 2.1). The value of a local fact can + be anything at all, provided it can be encoded as a "token". + +7.7. MLSx Examples + + The following examples are all taken from dialogues between existing + FTP clients and servers. Because of this, not all possible + variations of possible response formats are shown in the examples. + This should not be taken as limiting the options of other server + implementors. Where the examples show OS dependent information, that + is to be treated as being purely for the purposes of demonstration of + some possible OS specific information that could be defined. As at + the time of the writing of this document, no OS specific facts or + file types have been defined, the examples shown here should not be + treated as in any way to be preferred over other possible similar + definitions. Consult the IANA registries to determine what types and + facts have been defined. Finally also beware that as the examples + shown are taken from existing implementations, coded before this + document was completed, the possibility of variations between the + text of this document and the examples exists. In any such case of + inconsistency, the example is to be treated as incorrect. + + In the examples shown, only relevant commands and responses have been + included. This is not to imply that other commands (including + authentication, directory modification, PORT or PASV commands, or + similar) would not be present in an actual connection, or were not, + in fact, actually used in the examples before editing. Note also + that the formats shown are those that are transmitted between client + + + +Hethmon Standards Track [Page 37] + +RFC 3659 Extensions to FTP March 2007 + + + and server, not formats that would normally ever be reported to the + user of the client. + +7.7.1. Simple MLST + +C> PWD +S> 257 "/tmp" is current directory. +C> MLst cap60.pl198.tar.gz +S> 250- Listing cap60.pl198.tar.gz +S> Type=file;Size=1024990;Perm=r; /tmp/cap60.pl198.tar.gz +S> 250 End + + The client first asked to be told the current directory of the + server. This was purely for the purposes of clarity of this example. + The client then requested facts about a specific file. The server + returned the "250-" first control-response line, followed by a single + line of facts about the file, followed by the terminating "250 " + line. The text on the control-response line and the terminating line + can be anything the server decides to send. Notice that the fact + line is indented by a single space. Notice also that there are no + spaces in the set of facts returned, until the single space before + the file name. The file name returned on the fact line is a fully + qualified pathname of the file listed. The facts returned show that + the line refers to a file, that file contains approximately 1024990 + bytes, though more or less than that may be transferred if the file + is retrieved, and a different number may be required to store the + file at the client's file store, and the connected user has + permission to retrieve the file but not to do anything else + particularly interesting. + +7.7.2. MLST of a directory + +C> PWD +S> 257 "/" is current directory. +C> MLst tmp +S> 250- Listing tmp +S> Type=dir;Modify=19981107085215;Perm=el; /tmp +S> 250 End + + Again the PWD is just for the purposes of demonstration for the + example. The MLST fact line this time shows that the file listed is + a directory, that it was last modified at 08:52:15 on the 7th of + November, 1998 UTC, and that the user has permission to enter the + directory, and to list its contents, but not to modify it in any way. + Again, the fully qualified pathname of the directory listed is given. + + + + + + +Hethmon Standards Track [Page 38] + +RFC 3659 Extensions to FTP March 2007 + + +7.7.3. MLSD of a directory + +C> MLSD tmp +S> 150 BINARY connection open for MLSD tmp +D> Type=cdir;Modify=19981107085215;Perm=el; tmp +D> Type=cdir;Modify=19981107085215;Perm=el; /tmp +D> Type=pdir;Modify=19990112030508;Perm=el; .. +D> Type=file;Size=25730;Modify=19940728095854;Perm=; capmux.tar.z +D> Type=file;Size=1830;Modify=19940916055648;Perm=r; hatch.c +D> Type=file;Size=25624;Modify=19951003165342;Perm=r; MacIP-02.txt +D> Type=file;Size=2154;Modify=19950501105033;Perm=r; uar.netbsd.patch +D> Type=file;Size=54757;Modify=19951105101754;Perm=r; iptnnladev.1.0.sit.hqx +D> Type=file;Size=226546;Modify=19970515023901;Perm=r; melbcs.tif +D> Type=file;Size=12927;Modify=19961025135602;Perm=r; tardis.1.6.sit.hqx +D> Type=file;Size=17867;Modify=19961025135602;Perm=r; timelord.1.4.sit.hqx +D> Type=file;Size=224907;Modify=19980615100045;Perm=r; uar.1.2.3.sit.hqx +D> Type=file;Size=1024990;Modify=19980130010322;Perm=r; cap60.pl198.tar.gz +S> 226 MLSD completed + + In this example notice that there is no leading space on the fact + lines returned over the data connection. Also notice that two lines + of "type=cdir" have been given. These show two alternate names for + the directory listed, one a fully qualified pathname, and the other a + local name relative to the servers current directory when the MLSD + was performed. Note that all other file names in the output are + relative to the directory listed, though the server could, if it + chose, give a fully qualified pathname for the "type=pdir" line. + This server has chosen not to. The other files listed present a + fairly boring set of files that are present in the listed directory. + Note that there is no particular order in which they are listed. + They are not sorted by file name, by size, or by modify time. Note + also that the "perm" fact has an empty value for the file + "capmux.tar.z" indicating that the connected user has no permissions + at all for that file. This server has chosen to present the "cdir" + and "pdir" lines before the lines showing the content of the + directory, it is not required to do so. The "size" fact does not + provide any meaningful information for a directory, so is not + included in the fact lines for the directory types shown. + + + + + + + + + + + + + +Hethmon Standards Track [Page 39] + +RFC 3659 Extensions to FTP March 2007 + + +7.7.4. A More Complex Example + +C> MLst test +S> 250- Listing test +S> Type=dir;Perm=el;Unique=keVO1+ZF4 test +S> 250 End +C> MLSD test +S> 150 BINARY connection open for MLSD test +D> Type=cdir;Perm=el;Unique=keVO1+ZF4; test +D> Type=pdir;Perm=e;Unique=keVO1+d?3; .. +D> Type=OS.unix=slink:/foobar;Perm=;Unique=keVO1+4G4; foobar +D> Type=OS.unix=chr-13/29;Perm=;Unique=keVO1+5G4; device +D> Type=OS.unix=blk-11/108;Perm=;Unique=keVO1+6G4; block +D> Type=file;Perm=awr;Unique=keVO1+8G4; writable +D> Type=dir;Perm=cpmel;Unique=keVO1+7G4; promiscuous +D> Type=dir;Perm=;Unique=keVO1+1t2; no-exec +D> Type=file;Perm=r;Unique=keVO1+EG4; two words +D> Type=file;Perm=r;Unique=keVO1+IH4; leading space +D> Type=file;Perm=r;Unique=keVO1+1G4; file1 +D> Type=dir;Perm=cpmel;Unique=keVO1+7G4; incoming +D> Type=file;Perm=r;Unique=keVO1+1G4; file2 +D> Type=file;Perm=r;Unique=keVO1+1G4; file3 +D> Type=file;Perm=r;Unique=keVO1+1G4; file4 +S> 226 MLSD completed +C> MLSD test/incoming +S> 150 BINARY connection open for MLSD test/incoming +D> Type=cdir;Perm=cpmel;Unique=keVO1+7G4; test/incoming +D> Type=pdir;Perm=el;Unique=keVO1+ZF4; .. +D> Type=file;Perm=awdrf;Unique=keVO1+EH4; bar +D> Type=file;Perm=awdrf;Unique=keVO1+LH4; +D> Type=file;Perm=rf;Unique=keVO1+1G4; file5 +D> Type=file;Perm=rf;Unique=keVO1+1G4; file6 +D> Type=dir;Perm=cpmdelf;Unique=keVO1+!s2; empty +S> 226 MLSD completed + + For the purposes of this example the fact set requested has been + modified to delete the "size" and "modify" facts, and add the + "unique" fact. First, facts about a file name have been obtained via + MLST. Note that no fully qualified pathname was given this time. + That was because the server was unable to determine that information. + Then having determined that the file name represents a directory, + that directory has been listed. That listing also shows no fully + qualified pathname, for the same reason, thus has but a single + "type=cdir" line. This directory (which was created especially for + the purpose) contains several interesting files. There are some with + OS dependent file types, several sub-directories, and several + ordinary files. + + + + +Hethmon Standards Track [Page 40] + +RFC 3659 Extensions to FTP March 2007 + + + Not much can be said here about the OS dependent file types, as none + of the information shown there should be treated as any more than + possibilities. It can be seen that the OS type of the server is + "unix" though, which is one of the OS types in the IANA registry of + Operating System names. + + Of the three directories listed, "no-exec" has no permission granted + to this user to access at all. From the "Unique" fact values, it can + be determined that "promiscuous" and "incoming" in fact represent the + same directory. Its permissions show that the connected user has + permission to do essentially anything other than to delete the + directory. That directory was later listed. It happens that the + directory can not be deleted because it is not empty. + + Of the normal files listed, two contain spaces in their names. The + file called " leading space" actually contains two spaces in its + name, one before the "l" and one between the "g" and the "s". The + two spaces that separate the facts from the visible part of the + pathname make that clear. The file "writable" has the "a" and "w" + permission bits set, and consequently the connected user should be + able to STOR or APPE to that file. + + The other four file names, "file1", "file2", "file3", and "file4" all + represent the same underlying file, as can be seen from the values of + the "unique" facts of each. It happens that "file1" and "file2" are + Unix "hard" links, and that "file3" and "file4" are "soft" or + "symbolic" links to the first two. None of that information is + available via standard MLST facts, it is sufficient for the purposes + of FTP to note that all represent the same file, and that the same + data would be fetched no matter which of them was retrieved, and that + all would be simultaneously modified were data stored in any. + + Finally, the sub-directory "incoming" is listed. Since "promiscuous" + is the same directory there would be no point listing it as well. In + that directory, the files "file5" and "file6" represent still more + names for the "file1" file we have seen before. Notice the entry + between that for "bar" and "file5". Though it is not possible to + easily represent it in this document, that shows a file with a name + comprising exactly three spaces (" "). A client will have no + difficulty determining that name from the output presented to it + however. The directory "empty" is, as its name implies, empty, + though that is not shown here. It can, however, be deleted, as can + file "bar" and the file whose name is three spaces. All the files + that reside in this directory can be renamed. This is a consequence + of the UNIX semantics of the directory that contains them being + modifiable. + + + + + +Hethmon Standards Track [Page 41] + +RFC 3659 Extensions to FTP March 2007 + + +7.7.5. More Accurate Time Information + +C> MLst file1 +S> 250- Listing file1 +S> Type=file;Modify=19990929003355.237; file1 +S> 250 End + + In this example, the server-FTP is indicating that "file1" was last + modified 237 milliseconds after 00:33:55 UTC on the 29th of + September, 1999. + +7.7.6. A Different Server + +C> MLST +S> 250-Begin +S> type=dir;unique=AQkAAAAAAAABCAAA; / +S> 250 End. +C> MLSD +S> 150 Opening ASCII mode data connection for MLS. +D> type=cdir;unique=AQkAAAAAAAABCAAA; / +D> type=dir;unique=AQkAAAAAAAABEAAA; bin +D> type=dir;unique=AQkAAAAAAAABGAAA; etc +D> type=dir;unique=AQkAAAAAAAAB8AwA; halflife +D> type=dir;unique=AQkAAAAAAAABoAAA; incoming +D> type=dir;unique=AQkAAAAAAAABIAAA; lib +D> type=dir;unique=AQkAAAAAAAABWAEA; linux +D> type=dir;unique=AQkAAAAAAAABKAEA; ncftpd +D> type=dir;unique=AQkAAAAAAAABGAEA; outbox +D> type=dir;unique=AQkAAAAAAAABuAAA; quake2 +D> type=dir;unique=AQkAAAAAAAABQAEA; winstuff +S> 226 Listing completed. +C> MLSD linux +S> 150 Opening ASCII mode data connection for MLS. +D> type=cdir;unique=AQkAAAAAAAABWAEA; /linux +D> type=pdir;unique=AQkAAAAAAAABCAAA; / +D> type=dir;unique=AQkAAAAAAAABeAEA; firewall +D> type=file;size=12;unique=AQkAAAAAAAACWAEA; helo_world +D> type=dir;unique=AQkAAAAAAAABYAEA; kernel +D> type=dir;unique=AQkAAAAAAAABmAEA; scripts +D> type=dir;unique=AQkAAAAAAAABkAEA; security +S> 226 Listing completed. +C> MLSD linux/kernel +S> 150 Opening ASCII mode data connection for MLS. +D> type=cdir;unique=AQkAAAAAAAABYAEA; /linux/kernel +D> type=pdir;unique=AQkAAAAAAAABWAEA; /linux +D> type=file;size=6704;unique=AQkAAAAAAAADYAEA; k.config +D> type=file;size=7269221;unique=AQkAAAAAAAACYAEA; linux-2.0.36.tar.gz +D> type=file;size=12514594;unique=AQkAAAAAAAAEYAEA; linux-2.1.130.tar.gz + + + +Hethmon Standards Track [Page 42] + +RFC 3659 Extensions to FTP March 2007 + + +S> 226 Listing completed. + + Note that this server returns its "unique" fact value in quite a + different format. It also returns fully qualified pathnames for the + "pdir" entry. + +7.7.7. Some IANA Files + +C> MLSD +S> 150 BINARY connection open for MLSD . +D> Type=cdir;Modify=19990219183438; /iana/assignments +D> Type=pdir;Modify=19990112030453; .. +D> Type=dir;Modify=19990219073522; media-types +D> Type=dir;Modify=19990112033515; character-set-info +D> Type=dir;Modify=19990112033529; languages +D> Type=file;Size=44242;Modify=19990217230400; character-sets +D> Type=file;Size=1947;Modify=19990209215600; operating-system-names +S> 226 MLSD completed +C> MLSD media-types +S> 150 BINARY connection open for MLSD media-types +D> Type=cdir;Modify=19990219073522; media-types +D> Type=cdir;Modify=19990219073522; /iana/assignments/media-types +D> Type=pdir;Modify=19990219183438; .. +D> Type=dir;Modify=19990112033045; text +D> Type=dir;Modify=19990219183442; image +D> Type=dir;Modify=19990112033216; multipart +D> Type=dir;Modify=19990112033254; video +D> Type=file;Size=30249;Modify=19990218032700; media-types +S> 226 MLSD completed +C> MLSD character-set-info +S> 150 BINARY connection open for MLSD character-set-info +D> Type=cdir;Modify=19990112033515; character-set-info +D> Type=cdir;Modify=19990112033515; /iana/assignments/character-set-info +D> Type=pdir;Modify=19990219183438; .. +D> Type=file;Size=1234;Modify=19980903020400; windows-1251 +D> Type=file;Size=4557;Modify=19980922001400; tis-620 +D> Type=file;Size=801;Modify=19970324130000; ibm775 +D> Type=file;Size=552;Modify=19970320130000; ibm866 +D> Type=file;Size=922;Modify=19960505140000; windows-1258 +S> 226 MLSD completed +C> MLSD languages +S> 150 BINARY connection open for MLSD languages +D> Type=cdir;Modify=19990112033529; languages +D> Type=cdir;Modify=19990112033529; /iana/assignments/languages +D> Type=pdir;Modify=19990219183438; .. +D> Type=file;Size=2391;Modify=19980309130000; default +D> Type=file;Size=943;Modify=19980309130000; tags +D> Type=file;Size=870;Modify=19971026130000; navajo + + + +Hethmon Standards Track [Page 43] + +RFC 3659 Extensions to FTP March 2007 + + +D> Type=file;Size=699;Modify=19950911140000; no-bok +S> 226 MLSD completed +C> PWD +S> 257 "/iana/assignments" is current directory. + + This example shows some of the IANA maintained files that are + relevant for this specification in MLSD format. Note that these + listings have been edited by deleting many entries, the actual + listings are much longer. + +7.7.8. A Stress Test of Case (In)dependence + + The following example is intended to make clear some cases where case + dependent strings are permitted in the MLSx commands, and where case + independent strings are required. + + Note first that the "MLSD" command, shown here as "MlsD" is case + independent. Clients may issue this command in any case, or + combination of cases, they desire. This is the case for all FTP + commands. + +C> MlsD +S> 150 BINARY connection open for MLSD . +D> Type=pdir;Modify=19990929011228;Perm=el;Unique=keVO1+ZF4; .. +D> Type=file;Size=4096;Modify=19990929011440;Perm=r;Unique=keVO1+Bd8; FILE2 +D> Type=file;Size=4096;Modify=19990929011440;Perm=r;Unique=keVO1+aG8; file3 +D> Type=file;Size=4096;Modify=19990929011440;Perm=r;Unique=keVO1+ag8; FILE3 +D> Type=file;Size=4096;Modify=19990929011440;Perm=r;Unique=keVO1+bD8; file1 +D> Type=file;Size=4096;Modify=19990929011440;Perm=r;Unique=keVO1+bD8; file2 +D> Type=file;Size=4096;Modify=19990929011440;Perm=r;Unique=keVO1+Ag8; File3 +D> Type=file;Size=4096;Modify=19990929011440;Perm=r;Unique=keVO1+bD8; File1 +D> Type=file;Size=4096;Modify=19990929011440;Perm=r;Unique=keVO1+Bd8; File2 +D> Type=file;Size=4096;Modify=19990929011440;Perm=r;Unique=keVO1+bd8; FILE1 +S> 226 MLSD completed + + Next, notice the labels of the facts. These are also case- + independent strings; the server-FTP is permitted to return them in + any case desired. User-FTP must be prepared to deal with any case, + though it may do this by mapping the labels to a common case if + desired. + + Then, notice that there are nine objects of "type" file returned. In + a case-independent NVFS these would represent three different file + names, "file1", "file2", and "file3". With a case-dependent NVFS all + nine represent different file names. Either is possible, server-FTPs + may implement a case dependent or a case independent NVFS. User-FTPs + must allow for case dependent selection of files to manipulate on the + server. + + + +Hethmon Standards Track [Page 44] + +RFC 3659 Extensions to FTP March 2007 + + + Lastly, notice that the value of the "unique" fact is case dependent. + In the example shown, "file1", "File1", and "file2" all have the same + "unique" fact value "keVO1+bD8", and thus all represent the same + underlying file. On the other hand, "FILE1" has a different "unique" + fact value ("keVO1+bd8") and hence represents a different file. + Similarly, "FILE2" and "File2" are two names for the same underlying + file, whereas "file3", "File3" and "FILE3" all represent different + underlying files. + + That the approximate sizes ("size" fact) and last modification times + ("modify" fact) are the same in all cases might be no more than a + coincidence. + + It is not suggested that the operators of server-FTPs create an NVFS + that stresses the protocols to this extent; however, both user and + server implementations must be prepared to deal with such extreme + examples. + +7.7.9. Example from Another Server + +C> MlsD +S> 150 File Listing Follows in IMAGE / Binary mode. +D> type=cdir;modify=19990426150227;perm=el; /MISC +D> type=pdir;modify=19791231130000;perm=el; / +D> type=dir;modify=19990426150227;perm=el; CVS +D> type=dir;modify=19990426150228;perm=el; SRC +S> 226 Transfer finished successfully. +C> MlsD src +S> 150 File Listing Follows in IMAGE / Binary mode. +D> type=cdir;modify=19990426150228;perm=el; /MISC/src +D> type=pdir;modify=19990426150227;perm=el; /MISC +D> type=dir;modify=19990426150228;perm=el; CVS +D> type=dir;modify=19990426150228;perm=el; INSTALL +D> type=dir;modify=19990426150230;perm=el; INSTALLI +D> type=dir;modify=19990426150230;perm=el; TREES +S> 226 Transfer finished successfully. +C> MlsD src/install +S> 150 File Listing Follows in IMAGE / Binary mode. +D> type=cdir;modify=19990426150228;perm=el; /MISC/src/install +D> type=pdir;modify=19990426150228;perm=el; /MISC/src +D> type=file;modify=19990406234304;perm=r;size=20059; BOOTPC.C +D> type=file;modify=19980401170153;perm=r;size=278; BOOTPC.H +D> type=file;modify=19990413153736;perm=r;size=54220; BOOTPC.O +D> type=file;modify=19990223044003;perm=r;size=3389; CDROM.C +D> type=file;modify=19990413153739;perm=r;size=30192; CDROM.O +D> type=file;modify=19981119155324;perm=r;size=1055; CHANGELO +D> type=file;modify=19981204171040;perm=r;size=8297; COMMANDS.C +D> type=file;modify=19980508041749;perm=r;size=580; COMMANDS.H + + + +Hethmon Standards Track [Page 45] + +RFC 3659 Extensions to FTP March 2007 + + + ... +D> type=file;modify=19990419052351;perm=r;size=54264; URLMETHO.O +D> type=file;modify=19980218161629;perm=r;size=993; WINDOWS.C +D> type=file;modify=19970912154859;perm=r;size=146; WINDOWS.H +D> type=file;modify=19990413153731;perm=r;size=16812; WINDOWS.O +D> type=file;modify=19990322174959;perm=r;size=129; _CVSIGNO +D> type=file;modify=19990413153640;perm=r;size=82536; _DEPEND +S> 226 Transfer finished successfully. +C> MLst src/install/windows.c +S> 250-Listing src/install/windows.c +S> type=file;perm=r;size=993; /misc/src/install/windows.c +S> 250 End +S> ftp> mlst SRC/INSTALL/WINDOWS.C +C> MLst SRC/INSTALL/WINDOWS.C +S> 250-Listing SRC/INSTALL/WINDOWS.C +S> type=file;perm=r;size=993; /misc/SRC/INSTALL/WINDOWS.C +S> 250 End + + Note that this server gives fully qualified pathnames for the "pdir" + and "cdir" entries in MLSD listings. Also notice that this server + does, though it is not required to, sort its directory listing + outputs. That may be an artifact of the underlying file system + access mechanisms of course. Finally notice that the NVFS supported + by this server, in contrast to the earlier ones, implements its + pathnames in a case independent manner. The server seems to return + files using the case in which they were requested, when the name was + sent by the client, and otherwise uses an algorithm known only to + itself to select the case of the names it returns. + +7.7.10. A Server Listing Itself + +C> MLst f +S> 250-MLST f +S> Type=dir;Modify=20000710052229;Unique=AAD/AAAABIA; f +S> 250 End +C> CWD f +S> 250 CWD command successful. +C> MLSD +S> 150 Opening ASCII mode data connection for 'MLSD'. +D> Type=cdir;Unique=AAD/AAAABIA; . +D> Type=pdir;Unique=AAD/AAAAAAI; .. +D> Type=file;Size=987;Unique=AAD/AAAABIE; Makefile +D> Type=file;Size=20148;Unique=AAD/AAAABII; conf.c +D> Type=file;Size=11111;Unique=AAD/AAAABIM; extern.h +D> Type=file;Size=38721;Unique=AAD/AAAABIQ; ftpcmd.y +D> Type=file;Size=17922;Unique=AAD/AAAABIU; ftpd.8 +D> Type=file;Size=60732;Unique=AAD/AAAABIY; ftpd.c +D> Type=file;Size=3127;Unique=AAD/AAAABIc; logwtmp.c + + + +Hethmon Standards Track [Page 46] + +RFC 3659 Extensions to FTP March 2007 + + +D> Type=file;Size=2294;Unique=AAD/AAAABIg; pathnames.h +D> Type=file;Size=7605;Unique=AAD/AAAABIk; popen.c +D> Type=file;Size=9951;Unique=AAD/AAAABIo; ftpd.conf.5 +D> Type=file;Size=5023;Unique=AAD/AAAABIs; ftpusers.5 +D> Type=file;Size=3547;Unique=AAD/AAAABIw; logutmp.c +D> Type=file;Size=2064;Unique=AAD/AAAABI0; version.h +D> Type=file;Size=20420;Unique=AAD/AAAAAAM; cmds.c +D> Type=file;Size=15864;Unique=AAD/AAAAAAg; ls.c +D> Type=file;Size=2898;Unique=AAD/AAAAAAk; ls.h +D> Type=file;Size=2769;Unique=AAD/AAAAAAo; lsextern.h +D> Type=file;Size=2042;Unique=AAD/AAAAAAs; stat_flags.h +D> Type=file;Size=5708;Unique=AAD/AAAAAAw; cmp.c +D> Type=file;Size=9280;Unique=AAD/AAAAAA0; print.c +D> Type=file;Size=4657;Unique=AAD/AAAAAA4; stat_flags.c +D> Type=file;Size=2664;Unique=AAD/AAAAAA8; util.c +D> Type=file;Size=10383;Unique=AAD/AAAABJ0; ftpd.conf.cat5 +D> Type=file;Size=3631;Unique=AAD/AAAABJ4; ftpusers.cat5 +D> Type=file;Size=17729;Unique=AAD/AAAABJ8; ftpd.cat8 +S> 226 MLSD complete. + + This examples shows yet another server implementation, showing a + listing of its own source code. Note that this implementation does + not include the fully qualified path name in its "cdir" and "pdir" + entries, nor in the output from "MLST". Also note that the facts + requested were modified between the "MLST" and "MLSD" commands, + though that exchange has not been shown here. + +7.7.11. A Server with a Difference + +C> PASV +S> 227 Entering Passive Mode (127,0,0,1,255,46) +C> MLSD +S> 150 I tink I tee a trisector tree +D> Type=file;Unique=aaaaafUYqaaa;Perm=rf;Size=15741; x +D> Type=cdir;Unique=aaaaacUYqaaa;Perm=cpmel; / +D> Type=file;Unique=aaaaajUYqaaa;Perm=rf;Size=5760; x4 +D> Type=dir;Unique=aaabcaUYqaaa;Perm=elf; sub +D> Type=file;Unique=aaaaagUYqaaa;Perm=rf;Size=8043; x1 +D> Type=dir;Unique=aaab8aUYqaaa;Perm=cpmelf; files +D> Type=file;Unique=aaaaahUYqaaa;Perm=rf;Size=4983; x2 +D> Type=file;Unique=aaaaaiUYqaaa;Perm=rf;Size=6854; x3 +S> 226 That's all folks... +C> CWD sub +S> 250 CWD command successful. +C> PWD +S> 257 "/sub" is current directory. +C> PASV +S> 227 Entering Passive Mode (127,0,0,1,255,44) + + + +Hethmon Standards Track [Page 47] + +RFC 3659 Extensions to FTP March 2007 + + +C> MLSD +S> 150 I tink I tee a trisector tree +D> Type=dir;Unique=aaabceUYqaaa;Perm=elf; dir +D> Type=file;Unique=aaabcbUYqaaa;Perm=rf;Size=0; y1 +D> Type=file;Unique=aaabccUYqaaa;Perm=rf;Size=0; y2 +D> Type=file;Unique=aaabcdUYqaaa;Perm=rf;Size=0; y3 +D> Type=pdir;Unique=aaaaacUYqaaa;Perm=cpmel; / +D> Type=pdir;Unique=aaaaacUYqaaa;Perm=cpmel; .. +D> Type=cdir;Unique=aaabcaUYqaaa;Perm=el; /sub +S> 226 That's all folks... +C> PASV +S> 227 Entering Passive Mode (127,0,0,1,255,42) +C> MLSD dir +S> 150 I tink I tee a trisector tree +D> Type=pdir;Unique=aaabcaUYqaaa;Perm=el; /sub +D> Type=pdir;Unique=aaabcaUYqaaa;Perm=el; .. +D> Type=file;Unique=aaab8cUYqaaa;Perm=r;Size=15039; mlst.c +D> Type=dir;Unique=aaabcfUYqaaa;Perm=el; ect +D> Type=cdir;Unique=aaabceUYqaaa;Perm=el; dir +D> Type=cdir;Unique=aaabceUYqaaa;Perm=el; /sub/dir +D> Type=dir;Unique=aaabchUYqaaa;Perm=el; misc +D> Type=file;Unique=aaab8bUYqaaa;Perm=r;Size=34589; ftpd.c +S> 226 That's all folks... +C> CWD dir/ect +S> 250 CWD command successful. +C> PWD +S> 257 "/sub/dir/ect" is current directory. +C> PASV +S> 227 Entering Passive Mode (127,0,0,1,255,40) +C> MLSD +S> 150 I tink I tee a trisector tree +D> Type=dir;Unique=aaabcgUYqaaa;Perm=el; ory +D> Type=pdir;Unique=aaabceUYqaaa;Perm=el; /sub/dir +D> Type=pdir;Unique=aaabceUYqaaa;Perm=el; .. +D> Type=cdir;Unique=aaabcfUYqaaa;Perm=el; /sub/dir/ect +S> 226 That's all folks... +C> CWD /files +S> 250 CWD command successful. +C> PASV +S> 227 Entering Passive Mode (127,0,0,1,255,36) +C> MLSD +S> 150 I tink I tee a trisector tree +D> Type=cdir;Unique=aaab8aUYqaaa;Perm=cpmel; /files +D> Type=pdir;Unique=aaaaacUYqaaa;Perm=cpmel; / +D> Type=pdir;Unique=aaaaacUYqaaa;Perm=cpmel; .. +D> Type=file;Unique=aaab8cUYqaaa;Perm=rf;Size=15039; mlst.c +D> Type=file;Unique=aaab8bUYqaaa;Perm=rf;Size=34589; ftpd.c +S> 226 That's all folks... + + + +Hethmon Standards Track [Page 48] + +RFC 3659 Extensions to FTP March 2007 + + +C> RNFR mlst.c +S> 350 File exists, ready for destination name +C> RNTO list.c +S> 250 RNTO command successful. +C> PASV +S> 227 Entering Passive Mode (127,0,0,1,255,34) +C> MLSD +S> 150 I tink I tee a trisector tree +D> Type=file;Unique=aaab8cUYqaaa;Perm=rf;Size=15039; list.c +D> Type=pdir;Unique=aaaaacUYqaaa;Perm=cpmel; / +D> Type=pdir;Unique=aaaaacUYqaaa;Perm=cpmel; .. +D> Type=file;Unique=aaab8bUYqaaa;Perm=rf;Size=34589; ftpd.c +D> Type=cdir;Unique=aaab8aUYqaaa;Perm=cpmel; /files +S> 226 That's all folks... + + The server shown here returns its directory listings in seemingly + random order, and even seems to modify the order of the directory as + its contents change -- perhaps the underlying directory structure is + based upon hashing of some kind. Note that the "pdir" and "cdir" + entries are interspersed with other entries in the directory. Note + also that this server does not show a "pdir" entry when listing the + contents of the root directory of the virtual filestore; however, it + does however include multiple "cdir" and "pdir" entries when it feels + inclined. The server also uses obnoxiously "cute" messages. + +7.8. FEAT Response for MLSx + + When responding to the FEAT command, a server-FTP process that + supports MLST, and MLSD, plus internationalization of pathnames, MUST + indicate that this support exists. It does this by including a MLST + feature line. As well as indicating the basic support, the MLST + feature line indicates which MLST facts are available from the + server, and which of those will be returned if no subsequent "OPTS + MLST" command is sent. + + mlst-feat = SP "MLST" [SP factlist] CRLF + factlist = 1*( factname ["*"] ";" ) + + The initial space shown in the mlst-feat response is that required by + the FEAT command, two spaces are not permitted. If no factlist is + given, then the server-FTP process is indicating that it supports + MLST, but implements no facts. Only pathnames can be returned. This + would be a minimal MLST implementation, and useless for most + practical purposes. Where the factlist is present, the factnames + included indicate the facts supported by the server. Where the + optional asterisk appears after a factname, that fact will be + included in MLST format responses, until an "OPTS MLST" is given to + alter the list of facts returned. After that, subsequent FEAT + + + +Hethmon Standards Track [Page 49] + +RFC 3659 Extensions to FTP March 2007 + + + commands will return the asterisk to show the facts selected by the + most recent "OPTS MLST". + + Note that there is no distinct FEAT output for MLSD. The presence of + the MLST feature indicates that both MLST and MLSD are supported. + +7.8.1. Examples + +C> Feat +S> 211- Features supported +S> REST STREAM +S> MDTM +S> SIZE +S> TVFS +S> UTF8 +S> MLST Type*;Size*;Modify*;Perm*;Unique*;UNIX.mode;UNIX.chgd;X.hidden; +S> 211 End + + Aside from some features irrelevant here, this server indicates that + it supports MLST including several, but not all, standard facts, all + of which it will send by default. It also supports two OS dependent + facts, and one locally defined fact. The latter three must be + requested expressly by the client for this server to supply them. + +C> Feat +S> 211-Extensions supported: +S> CLNT +S> MDTM +S> MLST type*;size*;modify*;UNIX.mode*;UNIX.owner;UNIX.group;unique; +S> PASV +S> REST STREAM +S> SIZE +S> TVFS +S> Compliance Level: 19981201 (IETF mlst-05) +S> 211 End. + + Again, in addition to some irrelevant features here, this server + indicates that it supports MLST, four of the standard facts, one of + which ("unique") is not enabled by default, and several OS dependent + facts, one of which is provided by the server by default. This + server actually supported more OS dependent facts. Others were + deleted for the purposes of this document to comply with document + formatting restrictions. + + + + + + + + +Hethmon Standards Track [Page 50] + +RFC 3659 Extensions to FTP March 2007 + + +C> FEAT +S> 211-Features supported +S> MDTM +S> MLST Type*;Size*;Modify*;Perm;Unique*; +S> REST STREAM +S> SIZE +S> TVFS +S> 211 End + + This server has wisely chosen not to implement any OS dependent + facts. At the time of writing this document, no such facts have been + defined (using the mechanisms of section 10.1) so rational support + for them would be difficult at best. All but one of the facts + supported by this server are enabled by default. + +7.9. OPTS Parameters for MLST + + For the MLSx commands, the Client-FTP may specify a list of facts it + wishes to be returned in all subsequent MLSx commands until another + OPTS MLST command is sent. The format is specified by: + + mlst-opts = "OPTS" SP "MLST" + [ SP 1*( factname ";" ) ] + + By sending the "OPTS MLST" command, the client requests the server to + include only the facts listed as arguments to the command in + subsequent output from MLSx commands. Facts not included in the + "OPTS MLST" command MUST NOT be returned by the server. Facts that + are included should be returned for each entry returned from the MLSx + command where they meaningfully apply. Facts requested that are not + supported, or that are inappropriate to the file or directory being + listed should simply be omitted from the MLSx output. This is not an + error. Note that where no factname arguments are present, the client + is requesting that only the file names be returned. In this case, + and in any other case where no facts are included in the result, the + space that separates the fact names and their values from the file + name is still required. That is, the first character of the output + line will be a space, (or two characters will be spaces when the line + is returned over the control connection) and the file name will start + immediately thereafter. + + Clients should note that generating values for some facts can be + possible, but very expensive, for some servers. It is generally + acceptable to retrieve any of the facts that the server offers as its + default set before any "OPTS MLST" command has been given, however + clients should use particular caution before requesting any facts not + in that set. That is, while other facts may be available from the + server, clients should refrain from requesting such facts unless + + + +Hethmon Standards Track [Page 51] + +RFC 3659 Extensions to FTP March 2007 + + + there is a particular operational requirement for that particular + information, which ought be more significant than perhaps simply + improving the information displayed to an end user. + + Note, there is no "OPTS MLSD" command, the fact names set with the + "OPTS MLST" command apply to both MLST and MLSD commands. + + Servers are not required to accept "OPTS MLST" commands before + authentication of the user-PI, but may choose to permit them. + +7.9.1. OPTS MLST Response + + The "response-message" from [6] to a successful OPTS MLST command has + the following syntax. + + mlst-opt-resp = "MLST OPTS" [ SP 1*( factname ";" ) ] + + This defines the "response-message" as used in the "opts-good" + message in RFC 2389 [6]. + + The facts named in the response are those that the server will now + include in MLST (and MLSD) response, after the processing of the + "OPTS MLST" command. Any facts from the request not supported by the + server will be omitted from this response message. If no facts will + be included, the list of facts will be empty. Note that the list of + facts returned will be the same as those marked by a trailing + asterisk ("*") in a subsequent FEAT command response. There is no + requirement that the order of the facts returned be the same as that + in which they were requested, or that in which they will be listed in + a FEAT command response, or that in which facts are returned in MLST + responses. The fixed string "MLST OPTS" in the response may be + returned in any case, or mixture of cases. + +7.9.2. Examples + +C> Feat +S> 211- Features supported +S> MLST Type*;Size;Modify*;Perm;Unique;UNIX.mode;UNIX.chgd;X.hidden; +S> 211 End +C> OptS Mlst Type;UNIX.mode;Perm; +S> 200 MLST OPTS Type;Perm;UNIX.mode; +C> Feat +S> 211- Features supported +S> MLST Type*;Size;Modify;Perm*;Unique;UNIX.mode*;UNIX.chgd;X.hidden; +S> 211 End +C> opts MLst lang;type;charset;create; +S> 200 MLST OPTS Type; +C> Feat + + + +Hethmon Standards Track [Page 52] + +RFC 3659 Extensions to FTP March 2007 + + +S> 211- Features supported +S> MLST Type*;Size;Modify;Perm;Unique;UNIX.mode;UNIX.chgd;X.hidden; +S> 211 End +C> OPTS mlst size;frogs; +S> 200 MLST OPTS Size; +C> Feat +S> 211- Features supported +S> MLST Type;Size*;Modify;Perm;Unique;UNIX.mode;UNIX.chgd;X.hidden; +S> 211 End +C> opts MLst unique type; +S> 501 Invalid MLST options +C> Feat +S> 211- Features supported +S> MLST Type;Size*;Modify;Perm;Unique;UNIX.mode;UNIX.chgd;X.hidden; +S> 211 End + + For the purposes of this example, features other than MLST have been + deleted from the output to avoid clutter. The example shows the + initial default feature output for MLST. The facts requested are + then changed by the client. The first change shows facts that are + available from the server being selected. Subsequent FEAT output + shows the altered features as being returned. The client then + attempts to select some standard features that the server does not + support. This is not an error, however the server simply ignores the + requests for unsupported features, as the FEAT output that follows + shows. Then, the client attempts to request a non-standard, and + unsupported, feature. The server ignores that, and selects only the + supported features requested. Lastly, the client sends a request + containing a syntax error (spaces cannot appear in the factlist.) + The server-FTP sends an error response and completely ignores the + request, leaving the fact set selected as it had been previously. + + Note that in all cases, except the error response, the response lists + the facts that have been selected. + +C> Feat +S> 211- Features supported +S> MLST Type*;Size*;Modify*;Perm*;Unique*;UNIX.mode;UNIX.chgd;X.hidden; +S> 211 End +C> Opts MLST +S> 200 MLST OPTS +C> Feat +S> 211- Features supported +S> MLST Type;Size;Modify;Perm;Unique;UNIX.mode;UNIX.chgd;X.hidden; +S> 211 End +C> MLst tmp +S> 250- Listing tmp +S> /tmp + + + +Hethmon Standards Track [Page 53] + +RFC 3659 Extensions to FTP March 2007 + + +S> 250 End +C> OPTS mlst unique;size; +S> 200 MLST OPTS Size;Unique; +C> MLst tmp +S> 250- Listing tmp +S> Unique=keVO1+YZ5; /tmp +S> 250 End +C> OPTS mlst unique;type;modify; +S> 200 MLST OPTS Type;Modify;Unique; +C> MLst tmp +S> 250- Listing tmp +S> Type=dir;Modify=19990930152225;Unique=keVO1+YZ5; /tmp +S> 250 End +C> OPTS mlst fish;cakes; +S> 200 MLST OPTS +C> MLst tmp +S> 250- Listing tmp +S> /tmp +S> 250 End +C> OptS Mlst Modify;Unique; +S> 200 MLST OPTS Modify;Unique; +C> MLst tmp +S> 250- Listing tmp +S> Modify=19990930152225;Unique=keVO1+YZ5; /tmp +S> 250 End +C> opts MLst fish cakes; +S> 501 Invalid MLST options +C> MLst tmp +S> 250- Listing tmp +S> Modify=19990930152225;Unique=keVO1+YZ5; /tmp +S> 250 End + + This example shows the effect of changing the facts requested upon + subsequent MLST commands. Notice that a syntax error leaves the set + of selected facts unchanged. Also notice exactly two spaces + preceding the pathname when no facts were selected, either + deliberately, or because none of the facts requested were available. + +8. Impact on Other FTP Commands + + Along with the introduction of MLST, traditional FTP commands must be + extended to allow for the use of more than US-ASCII [1] or EBCDIC + character sets. In general, the support of MLST requires support for + arbitrary character sets wherever file names and directory names are + allowed. This applies equally to both arguments given to the + following commands and to the replies from them, as appropriate. + + + + + +Hethmon Standards Track [Page 54] + +RFC 3659 Extensions to FTP March 2007 + + + APPE RMD + CWD RNFR + DELE RNTO + MKD STAT + PWD STOR + RETR STOU + + The arguments to all of these commands should be processed the same + way that MLST commands and responses are processed with respect to + handling embedded spaces, CRs and NULs. See section 2.2. + +9. Character Sets and Internationalization + + FTP commands are protocol elements, and are always expressed in + ASCII. FTP responses are composed of the numeric code, which is a + protocol element, and a message, which is often expected to convey + information to the user. It is not expected that users normally + interact directly with the protocol elements, rather the user-FTP + process constructs the commands, and interprets the results, in the + manner best suited for the particular user. Explanatory text in + responses generally has no particular meaning to the protocol. The + numeric codes provide all necessary information. Server-PIs are free + to provide the text in any language that can be adequately + represented in ASCII, or where an alternative language and + representation has been negotiated (see [7]) in that language and + representation. + + Pathnames are expected to be encoded in UTF-8 allowing essentially + any character to be represented in a pathname. Meaningful pathnames + are defined by the server NVFS. + + No restrictions at all are placed upon the contents of files + transferred using the FTP protocols. Unless the "media-type" fact is + provided in a MLSx response nor is any advice given here that would + allow determining the content type. That information is assumed to + be obtained via other means. + +10. IANA Considerations + + This specification makes use of some lists of values currently + maintained by the IANA, and creates two new lists for the IANA to + maintain. It does not add any values to any existing registries. + + The existing IANA registries used by this specification are modified + using mechanisms specified elsewhere. + + + + + + +Hethmon Standards Track [Page 55] + +RFC 3659 Extensions to FTP March 2007 + + +10.1. The OS-Specific Fact Registry + + A registry of OS specific fact names shall be maintained by the IANA. + The OS names for the OS portion of the fact name must be taken from + the IANA's list of registered OS names. To add a fact name to this + OS specific registry of OS specific facts, an applicant must send to + the IANA a request, in which is specified the OS name, the OS + specific fact name, a definition of the syntax of the fact value, + which must conform to the syntax of a token as given in this + document, and a specification of the semantics to be associated with + the particular fact and its values. Upon receipt of such an + application, and if the combination of OS name and OS specific fact + name has not been previously defined, the IANA will add the + specification to the registry. + + Any examples of OS specific facts found in this document are to be + treated as examples of possible OS specific facts, and do not form a + part of the IANA's registry merely because of being included in this + document. + +10.2. The OS-Specific Filetype Registry + + A registry of OS specific file types shall be maintained by the IANA. + The OS names for the OS portion of the fact name must be taken from + the IANA's list of registered OS names. To add a file type to this + OS specific registry of OS specific file types, an applicant must + send to the IANA a request, in which is specified the OS name, the OS + specific file type, a definition of the syntax of the fact value, + which must conform to the syntax of a token as given in this + document, and a specification of the semantics to be associated with + the particular fact and its values. Upon receipt of such an + application, and if the combination of OS name and OS specific file + type has not been previously defined, the IANA will add the + specification to the registry. + + Any examples of OS specific file types found in this document are to + be treated as potential OS specific file types only, and do not form + a part of the IANA's registry merely because of being included in + this document. + + + + + + + + + + + + +Hethmon Standards Track [Page 56] + +RFC 3659 Extensions to FTP March 2007 + + +11. Security Considerations + + This memo does not directly concern security. It is not believed + that any of the mechanisms documented here impact in any particular + way upon the security of FTP. + + Implementing the SIZE command, and perhaps some of the facts of the + MLSx commands, may impose a considerable load on the server, which + could lead to denial of service attacks. Servers have, however, + implemented this for many years, without significant reported + difficulties. + + The server-FTP should take care not to reveal sensitive information + about files to unauthorised parties. In particular, some underlying + filesystems provide a file identifier that, if known, can allow many + of the filesystem protection mechanisms to be by-passed. That + identifier would not be a suitable choice to use as the basis of the + value of the unique fact. + + The FEAT and OPTS commands may be issued before the FTP + authentication has occurred [6]. This allows unauthenticated clients + to determine which of the features defined here are supported, and to + negotiate the fact list for MLSx output. No actual MLSx commands may + be issued however, and no problems with permitting the selection of + the format prior to authentication are foreseen. + + A general discussion of issues related to the security of FTP can be + found in [13]. + + + + + + + + + + + + + + + + + + + + + + + +Hethmon Standards Track [Page 57] + +RFC 3659 Extensions to FTP March 2007 + + +12. Normative References + + [1] Coded Character Set--7-bit American Standard Code for + Information Interchange, ANSI X3.4-1986. + + [2] Yergeau, F., "UTF-8, a transformation format of ISO 10646", RFC + 3629, November 2003. + + [3] Postel, J. and J. Reynolds, "File Transfer Protocol (FTP)", STD + 9, RFC 959, October 1985. + + [4] Bradner, S., "Key words for use in RFCs to Indicate Requirement + Levels", BCP 14, RFC 2119, March 1997. + + [5] Crocker, D. and P. Overell, "Augmented BNF for Syntax + Specifications: ABNF", RFC 4234, October 2005. + + [6] Hethmon, P. and R. Elz, "Feature negotiation mechanism for the + File Transfer Protocol", RFC 2389, August 1998. + + [7] Curtin, B., "Internationalization of the File Transfer + Protocol", RFC 2640, July 1999. + + [8] Postel, J. and J. Reynolds, "Telnet protocol Specification", STD + 8, RFC 854, May 1983. + + [9] Braden, R,. "Requirements for Internet Hosts -- Application and + Support", STD 3, RFC 1123, October 1989. + + [10] ISO/IEC 10646-1:1993 "Universal multiple-octet coded character + set (UCS) -- Part 1: Architecture and basic multilingual plane", + International Standard -- Information Technology, 1993. + + [11] Internet Assigned Numbers Authority. http://www.iana.org + Email: iana@iana.org. + + [12] Phillips, A. and M. Davis, "Tags for Identifying Languages", BCP + 47, RFC 4646, September 2006. + + [13] Allman, M. and S. Ostermann, "FTP Security Considerations" RFC + 2577, May 1999. + + + + + + + + + + +Hethmon Standards Track [Page 58] + +RFC 3659 Extensions to FTP March 2007 + + +Acknowledgments + + This document is a product of the FTPEXT working group of the IETF. + + The following people are among those who have contributed to this + document: + + Alex Belits + D. J. Bernstein + Dave Cridland + Martin J. Duerst + Bill Fenner (and the rest of the IESG) + Paul Ford-Hutchinson + Mike Gleason + Mark Harris + Stephen Head + Alun Jones + Andrew Main + James Matthews + Luke Mewburn + Jan Mikkelsen + Keith Moore + Buz Owen + Mark Symons + Stephen Tihor + and the entire FTPEXT working group of the IETF. + + Apologies are offered to any inadvertently omitted. + + The description of the modifications to the REST command and the MDTM + and SIZE commands comes from a set of modifications suggested for STD + 9, RFC 959 by Rick Adams in 1989. A document containing just those + commands, edited by David Borman, has been merged with this document. + + Mike Gleason, Alun Jones and Luke Mewburn provided access to FTP + servers used in some of the examples. + + All of the examples in this document are taken from actual + client/server exchanges, though some have been edited for brevity, or + to meet document formatting requirements. + +RFC Editor Note: + + Several of the examples in this document exceed the RFC standard line + length of 72 characters. Since this document is a standards-track + result of an IETF working group and is important to an IETF sub- + community, the RFC Editor is publishing it with the margin + violations. This is not a precedent. + + + +Hethmon Standards Track [Page 59] + +RFC 3659 Extensions to FTP March 2007 + + +Author's Address + + Paul Hethmon + Hethmon Software + 10420 Jackson Oaks Way, Suite 201 + Knoxville, TN 37922 + + EMail: phethmon@hethmon.com + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +Hethmon Standards Track [Page 60] + +RFC 3659 Extensions to FTP March 2007 + + +Full Copyright Statement + + Copyright (C) The IETF Trust (2007). + + This document is subject to the rights, licenses and restrictions + contained in BCP 78, and except as set forth therein, the authors + retain all their rights. + + This document and the information contained herein are provided on an + "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS + OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY, THE IETF TRUST AND + THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS + OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF + THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED + WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. + +Intellectual Property + + The IETF takes no position regarding the validity or scope of any + Intellectual Property Rights or other rights that might be claimed to + pertain to the implementation or use of the technology described in + this document or the extent to which any license under such rights + might or might not be available; nor does it represent that it has + made any independent effort to identify any such rights. Information + on the procedures with respect to rights in RFC documents can be + found in BCP 78 and BCP 79. + + Copies of IPR disclosures made to the IETF Secretariat and any + assurances of licenses to be made available, or the result of an + attempt made to obtain a general license or permission for the use of + such proprietary rights by implementers or users of this + specification can be obtained from the IETF on-line IPR repository at + http://www.ietf.org/ipr. + + The IETF invites any interested party to bring to its attention any + copyrights, patents or patent applications, or other proprietary + rights that may cover technology that may be required to implement + this standard. Please address the information to the IETF at + ietf-ipr@ietf.org. + +Acknowledgement + + Funding for the RFC Editor function is currently provided by the + Internet Society. + + + + + + + +Hethmon Standards Track [Page 61] + -- 2.11.4.GIT