| blob | 6e07310dff1a5370572915204658933bb9879637 |
2 <html>
3 <head>
11 </head>
12 <body>
18 </p>
21 <!-- ####################################################### -->
22 <hr>
24 <ul>
33 </ul>
35 <hr>
37 <!-- ####################################################### -->
40 <p>So you want to copy a file quickly and easily, and most important,
41 completely portably. And since this is C++, you have an open
42 ifstream (call it IN) and an open ofstream (call it OUT):
43 <pre>
46 std::ifstream IN ("input_file");
48 </p>
49 <p>Here's the easiest way to get it completely wrong:
50 <pre>
52 For those of you who don't already know why this doesn't work
53 (probably from having done it before), I invite you to quickly
55 the sentence
56 <pre>
57 The quick brown fox jumped over the lazy dog.</pre>
58 surrounded by blank lines. Code it up and try it. The contents
60 </p>
61 <p>Seriously, go do it. Get surprised, then come back. It's worth it.
62 </p>
65 handle formatting, nothing else. In particular, they break up on
66 whitespace. The actual reading, writing, and storing of data is
69 a pointer-to-streambuf, in order to help with just this kind of
71 </p>
73 the [io]streams hold pointers (or references, depending on the
74 implementation) to their buffers, not the actual
75 buffers. This allows polymorphic behavior on the part of the buffers
76 as well as the streams themselves. The pointer is easily retrieved
78 way to copy the file is:
79 <pre>
81 </p>
83 behavior, since that particular << isn't defined by the Standard.
84 I have seen instances where it is implemented, but the character
85 extraction process removes all the whitespace, leaving you with no
87 libraries that do not define that operator, IN (or one of IN's
88 member pointers) sometimes gets converted to a void*, and the output
89 file then contains a perfect text representation of a hexidecimal
90 address (quite a big surprise). Others don't compile at all.
91 </p>
93 The operators shown above are all defined in the parent
94 basic_ostream class and are therefore available with all possible
95 descendents.
96 </p>
99 </p>
101 <hr>
103 <!--
104 This is not written very well. I need to redo this section.
105 -->
106 <p>First, are you sure that you understand buffering? Particularly
107 the fact that C++ may not, in fact, have anything to do with it?
108 </p>
109 <p>The rules for buffering can be a little odd, but they aren't any
110 different from those of C. (Maybe that's why they can be a bit
111 odd.) Many people think that writing a newline to an output
112 stream automatically flushes the output buffer. This is true only
113 when the output stream is, in fact, a terminal and not a file
115 since C++ says nothing about files nor terminals. All of that is
116 system-dependent. (The "newline-buffer-flushing only occurring
117 on terminals" thing is mostly true on Unix systems, though.)
118 </p>
120 output stream only writes a newline. This is incorrect; after a
121 newline is written, the buffer is also flushed. Perhaps this
122 is the effect you want when writing to a screen -- get the text
123 out as soon as possible, etc -- but the buffering is largely
124 wasted when doing this to a file:
125 <pre>
129 The proper thing to do in this case to just write the data out
130 and let the libraries and the system worry about the buffering.
131 If you need a newline, just write a newline:
132 <pre>
136 I have also joined the output statements into a single statement.
137 You could make the code prettier by moving the single newline to
138 the start of the quoted text on the thing line, for example.
139 </p>
140 <p>If you do need to flush the buffer above, you can send an
142 yourself:
143 <pre>
145 output.flush(); // or call a member fn </pre>
146 </p>
147 <p>On the other hand, there are times when writing to a file should
148 be like writing to standard error; no buffering should be done
149 because the data needs to appear quickly (a prime example is a
150 log file for security-related information). The way to do this is
151 just to turn off the buffering <em>before any I/O operations at
152 all</em> have been done, i.e., as soon as possible after opening:
153 <pre>
156 int i;
160 ...
163 </p>
164 <p>Since all aspects of buffering are handled by a streambuf-derived
167 arguments are the same as those for the Standard C I/O Library
168 function (a buffer area followed by its size).
169 </p>
170 <p>A great deal of this is implementation-dependent. For example,
177 varying effects. Other user-defined class derived from streambuf can
181 which you must allocate and deallocate.)
182 </p>
183 <p>A last reminder: there are usually more buffers involved than
184 just those at the language/library level. Kernel buffers, disk
185 buffers, and the like will also have an effect. Inspecting and
186 changing those are system-dependent.
187 </p>
190 </p>
192 <hr>
194 <p>The first and most important thing to remember about binary I/O is
198 operators of the normal fstreams to do binary I/O.
199 </p>
200 <p>Sorry. Them's the breaks.
201 </p>
202 <p>This isn't going to try and be a complete tutorial on reading and
205 up a couple of misconceptions and common errors.
206 </p>
208 and no less. Normal text mode has to be concerned with the newline
209 characters, and the runtime system will translate between (for
210 example) '\n' and the appropriate end-of-line sequence (LF on Unix,
211 CRLF on DOS, CR on Macintosh, etc). (There are other things that
212 normal mode does, but that's the most obvious.) Opening a file in
213 binary mode disables this conversion, so reading a CRLF sequence
214 under Windows won't accidentally get mapped to a '\n' character, etc.
215 Binary mode is not supposed to suddenly give you a bitstream, and
216 if it is doing so in your program then you've discovered a bug in
217 your vendor's compiler (or some other part of the C++ implementation,
218 possibly the runtime system).
219 </p>
221 read isn't going to work with the standard file stream classes, even
224 not reading and writing. Their job is to interpret the data into
225 text characters, and that's exactly what you don't want to happen
226 during binary I/O.
227 </p>
229 functions still aren't guaranteed to help you. These are
231 (This may or may not be what you want, see below.)
232 </p>
233 <p>Notice how all the problems here are due to the inappropriate use
236 seemingly infinite number of solutions, and a few are listed here:
237 <ul>
240 types you're using." This is a Bad Thing, because while
241 the compiler would probably be just fine with it, other humans
242 are going to be confused. The overloaded bitshift operators
243 have a well-defined meaning (formatting), and this breaks it.
245 the file and copy the structure." Well, this is easy to
246 make work, and easy to break, and is pretty equivalent to
248 makes no use of the iostream library at all...
250 While not trivial for the beginner, this is the best of all
251 solutions. The streambuf/filebuf layer is the layer that is
252 responsible for actual I/O. If you want to use the C++
253 library for binary I/O, this is where you start.
254 </ul>
255 </p>
256 <p>How to go about using streambufs is a bit beyond the scope of this
257 document (at least for now), but while streambufs go a long way,
258 they still leave a couple of things up to you, the programmer.
259 As an example, byte ordering is completely between you and the
260 operating system, and you have to handle it yourself.
261 </p>
262 <p>Deriving a streambuf or filebuf
263 class from the standard ones, one that is specific to your data
264 types (or an abstraction thereof) is probably a good idea, and
265 lots of examples exist in journals and on Usenet. Using the
266 standard filebufs directly (either by declaring your own or by
268 is certainly feasible as well.
269 </p>
270 <p>One area that causes problems is trying to do bit-by-bit operations
271 with filebufs. C++ is no different from C in this respect: I/O
272 must be done at the byte level. If you're trying to read or write
273 a few bits at a time, you're going about it the wrong way. You
274 must read/write an integral number of bytes and then process the
275 bytes. (For example, the streambuf functions take and return
277 </p>
278 <p>Another area of problems is opening text files in binary mode.
279 Generally, binary mode is intended for binary files, and opening
280 text files in binary mode means that you now have to deal with all of
281 those end-of-line and end-of-file problems that we mentioned before.
282 An instructive thread from comp.lang.c++.moderated delved off into
283 this topic starting more or less at
285 article and continuing to the end of the thread. (You'll have to
286 sort through some flames every couple of paragraphs, but the points
287 made are good ones.)
288 </p>
290 <hr>
293 are in this author's opinion one of the coolest things since
294 sliced time. An example of their use is in the Received Wisdom
295 section for Chapter 21 (Strings),
297 format strings</a>.
298 </p>
299 <p>The quick definition is: they are siblings of ifstream and ofstream,
305 ostringstream. You've read a string as input and need to parse it?
306 Initialize an istringstream with that string, and then pull pieces
309 member function.
310 </p>
311 <p>This only works if you've written your
313 and correctly means that they take istreams and ostreams as
315 take the latter, then your I/O operators will work fine with
316 file streams, but with nothing else -- including stringstreams.
317 </p>
318 <p>If you are a user of the strstream classes, you need to update
320 terminate the C-style character array, you don't have to mess with
322 memory yourself. The strstreams have been officially deprecated,
323 which means that 1) future revisions of the C++ Standard won't
324 support them, and 2) if you use them, people will laugh at you.
325 </p>
327 <hr>
329 <p>Creating your own stream buffers for I/O can be remarkably easy.
330 If you are interested in doing so, we highly recommend two very
331 excellent books:
336 Addison-Wesley, who isn't paying us a cent for saying that, honest.
337 </p>
338 <p>Here is a simple example, io/outbuf1, from the Josuttis text. It
339 transforms everything sent through it to uppercase. This version
340 assumes many things about the nature of the character type being
341 used (for more information, read the books or the newsgroups):
342 <pre>
348 class outbuf : public std::streambuf
349 {
350 protected:
351 /* central output function
352 * - print characters in uppercase mode
353 */
354 virtual int_type overflow (int_type c) {
355 if (c != EOF) {
356 // convert lowercase to uppercase
359 // and write the character to the standard output
360 if (putchar(c) == EOF) {
361 return EOF;
362 }
363 }
364 return c;
365 }
366 };
368 int main()
369 {
370 // create special output buffer
371 outbuf ob;
372 // initialize output stream with that output buffer
373 std::ostream out(&ob);
377 return 0;
378 }
379 </pre>
380 Try it yourself!
381 </p>
383 <hr>
387 same time. One notable place was Usenet, where James Kanze and
388 Dietmar Kühl separately posted articles on why attempting
389 generic binary I/O was not a good idea. (Here are copies of
392 </p>
393 <p>Briefly, the problems of byte ordering and type sizes mean that
396 between arbitrary programs, or across a network, or from one
397 invocation of a program to another invocation of the same program
398 on a different platform, etc.
399 </p>
400 <p>The entire Usenet thread is instructive, and took place under the
402 and comp.lang.c++.moderated in parallel. Also in that thread,
403 Dietmar Kühl mentioned that he had written a pair of stream
404 classes that would read and write XDR, which is a good step towards
405 a portable binary format.
406 </p>
408 <hr>
410 <p>It sounds like a flame on C, but it isn't. Really. Calm down.
411 I'm just saying it to get your attention.
412 </p>
413 <p>Because the C++ library includes the C library, both C-style and
414 C++-style I/O have to work at the same time. For example:
415 <pre>
422 </pre>
423 This must do what you think it does.
424 </p>
425 <p>Alert members of the audience will immediately notice that buffering
426 is going to make a hash of the output unless special steps are taken.
427 </p>
429 involve doing very little buffering for the standard streams, leaving
430 most of the buffering to the underlying C library. (This kind of
432 The upside is that correctness is ensured. The downside is that
434 performance when the C++ I/O library is layered on top of the C I/O
435 library (as it is for 3.0 by default). Some patches are in the
436 works which should improve the situation for 3.1.
437 </p>
438 <p>However, the C and C++ standard streams only need to be kept in sync
439 when both libraries' facilities are in use. If your program only uses
440 C++ I/O, then there's no need to sync with the C streams. The right
441 thing to do in this case is to call
442 <pre>
445 std::ios::sync_with_stdio(false);
446 </pre>
447 </p>
448 <p>You must do this before performing any I/O via the C++ stream objects.
449 Once you call this, the C++ streams will operate independently of the
451 company will become fully buffered on their own.
452 </p>
453 <p>Note, by the way, that the synchronization requirement only applies to
457 objects that you declare yourself have no such requirement and are fully
458 buffered.
459 </p>
461 <hr>
463 <p>I'll assume that you have already read the
465 and the
467 access</a> (you might not think of an I/O stream as a container, but
468 the points made there also hold here). If you have not read them,
469 please do so first.
470 </p>
471 <p>This gets a bit tricky. Please read carefully, and bear with me.
472 </p>
478 </p>
479 <p>A generic locking mechanism is somewhat in place at the filebuf layer,
480 but is not used in the current code. Providing locking at any higher
481 level is akin to providing locking within containers, and is not done
482 for the same reasons (see the links above).
483 </p>
485 <p>The __basic_file type is simply a collection of small wrappers around
486 the C stdio layer (again, see the link under Structure). We do no
489 </p>
491 must be answered with, "is your platform's C library threadsafe
492 for I/O?" Some are by default, some are not; many offer multiple
493 implementations of the C library with varying tradeoffs of threadsafety
494 and efficiency. You, the programmer, are always required to take care
495 with multiple threads.
496 </p>
497 <p>(As an example, the POSIX standard requires that C stdio FILE*
498 operations are atomic. POSIX-conforming C libraries (e.g, on Solaris
499 and GNU/Linux) have an internal mutex to serialize operations on
500 FILE*s. However, you still need to not do stupid things like calling
503 </p>
504 <p>So, if your platform's C library is threadsafe, then your
506 level. For higher-level operations, such as manipulating the data
507 contained in the stream formatting classes (e.g., setting up callbacks
509 like any other critical shared resource.
510 </p>
513 second choice is available for I/O implementations: libio. This is
514 disabled by default, and in fact will not currently work due to other
515 issues. It will be revisited, however.
516 </p>
517 <p>The libio code is a subset of the guts of the GNU libc (glibc) I/O
519 type is basically derived from FILE. (The real situation is more
520 complex than that... it's derived from an internal type used to
521 implement FILE. See libio/libioP.h to see scary things done with
523 to go through; the filebuf makes calls directly into the same
525 and so forth, using internal data structures. (And when I say
527 replaced by a jump into an internal function. Fast but frightening.
528 *grin*)
529 </p>
530 <p>Also, the libio internal locks are used. This requires pulling in
531 large chunks of glibc, such as a pthreads implementation, and is one
532 of the issues preventing widespread use of libio as the libstdc++
533 cstdio implementation.
534 </p>
535 <p>But we plan to make this work, at least as an option if not a future
536 default. Platforms running a copy of glibc with a recent-enough
537 version will see calls from libstdc++ directly into the glibc already
538 installed. For other platforms, a copy of the libio subsection will
539 be built and included in libstdc++.
540 </p>
542 <p>Don't forget that other cstdio implemenations are possible. You could
543 easily write one to perform your own forms of locking, to solve your
545 </p>
548 <!-- ####################################################### -->
550 <hr>
553 Comments and suggestions are welcome, and may be sent to
555 </em></p>
558 </body>
559 </html>