| blob | 4a0b6a927fbcbf745ce6e9c746df37639941e7bd |
2 <!DOCTYPE html
3 PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
7 <head>
15 </head>
16 <body>
22 </p>
25 <!-- ####################################################### -->
26 <hr />
28 <ul>
39 </ul>
41 <hr />
43 <!-- ####################################################### -->
46 <p>So you want to copy a file quickly and easily, and most important,
47 completely portably. And since this is C++, you have an open
48 ifstream (call it IN) and an open ofstream (call it OUT):
49 </p>
50 <pre>
53 std::ifstream IN ("input_file");
55 <p>Here's the easiest way to get it completely wrong:
56 </p>
57 <pre>
59 <p>For those of you who don't already know why this doesn't work
60 (probably from having done it before), I invite you to quickly
62 the sentence
63 </p>
64 <pre>
65 The quick brown fox jumped over the lazy dog.</pre>
66 <p>surrounded by blank lines. Code it up and try it. The contents
68 </p>
69 <p>Seriously, go do it. Get surprised, then come back. It's worth it.
70 </p>
73 handle formatting, nothing else. In particular, they break up on
74 whitespace. The actual reading, writing, and storing of data is
77 a pointer-to-streambuf, in order to help with just this kind of
79 </p>
81 the [io]streams hold pointers (or references, depending on the
82 implementation) to their buffers, not the actual
83 buffers. This allows polymorphic behavior on the part of the buffers
84 as well as the streams themselves. The pointer is easily retrieved
86 way to copy the file is:
87 </p>
88 <pre>
91 behavior, since that particular << isn't defined by the Standard.
92 I have seen instances where it is implemented, but the character
93 extraction process removes all the whitespace, leaving you with no
95 libraries that do not define that operator, IN (or one of IN's
96 member pointers) sometimes gets converted to a void*, and the output
97 file then contains a perfect text representation of a hexidecimal
98 address (quite a big surprise). Others don't compile at all.
99 </p>
101 The operators shown above are all defined in the parent
102 basic_ostream class and are therefore available with all possible
103 descendents.
104 </p>
107 </p>
109 <hr />
111 <!--
112 This is not written very well. I need to redo this section.
113 -->
114 <p>First, are you sure that you understand buffering? Particularly
115 the fact that C++ may not, in fact, have anything to do with it?
116 </p>
117 <p>The rules for buffering can be a little odd, but they aren't any
118 different from those of C. (Maybe that's why they can be a bit
119 odd.) Many people think that writing a newline to an output
120 stream automatically flushes the output buffer. This is true only
121 when the output stream is, in fact, a terminal and not a file
123 since C++ says nothing about files nor terminals. All of that is
124 system-dependent. (The "newline-buffer-flushing only occurring
125 on terminals" thing is mostly true on Unix systems, though.)
126 </p>
128 output stream only writes a newline. This is incorrect; after a
129 newline is written, the buffer is also flushed. Perhaps this
130 is the effect you want when writing to a screen -- get the text
131 out as soon as possible, etc -- but the buffering is largely
132 wasted when doing this to a file:
133 </p>
134 <pre>
138 <p>The proper thing to do in this case to just write the data out
139 and let the libraries and the system worry about the buffering.
140 If you need a newline, just write a newline:
141 </p>
142 <pre>
146 <p>I have also joined the output statements into a single statement.
147 You could make the code prettier by moving the single newline to
148 the start of the quoted text on the thing line, for example.
149 </p>
150 <p>If you do need to flush the buffer above, you can send an
152 yourself:
153 </p>
154 <pre>
156 output.flush(); // or call a member fn </pre>
157 <p>On the other hand, there are times when writing to a file should
158 be like writing to standard error; no buffering should be done
159 because the data needs to appear quickly (a prime example is a
160 log file for security-related information). The way to do this is
161 just to turn off the buffering <em>before any I/O operations at
162 all</em> have been done (note that opening counts as an I/O operation):
163 </p>
164 <pre>
165 std::ofstream os;
166 std::ifstream is;
167 int i;
174 ...
177 <p>Since all aspects of buffering are handled by a streambuf-derived
180 arguments are the same as those for the Standard C I/O Library
181 function (a buffer area followed by its size).
182 </p>
183 <p>A great deal of this is implementation-dependent. For example,
190 varying effects. Other user-defined class derived from streambuf can
194 which you must allocate and deallocate.)
195 </p>
196 <p>A last reminder: there are usually more buffers involved than
197 just those at the language/library level. Kernel buffers, disk
198 buffers, and the like will also have an effect. Inspecting and
199 changing those are system-dependent.
200 </p>
203 </p>
205 <hr />
207 <p>The first and most important thing to remember about binary I/O is
211 operators of the normal fstreams to do binary I/O.
212 </p>
213 <p>Sorry. Them's the breaks.
214 </p>
215 <p>This isn't going to try and be a complete tutorial on reading and
218 up a couple of misconceptions and common errors.
219 </p>
221 and no less. Normal text mode has to be concerned with the newline
222 characters, and the runtime system will translate between (for
223 example) '\n' and the appropriate end-of-line sequence (LF on Unix,
224 CRLF on DOS, CR on Macintosh, etc). (There are other things that
225 normal mode does, but that's the most obvious.) Opening a file in
226 binary mode disables this conversion, so reading a CRLF sequence
227 under Windows won't accidentally get mapped to a '\n' character, etc.
228 Binary mode is not supposed to suddenly give you a bitstream, and
229 if it is doing so in your program then you've discovered a bug in
230 your vendor's compiler (or some other part of the C++ implementation,
231 possibly the runtime system).
232 </p>
234 read isn't going to work with the standard file stream classes, even
237 not reading and writing. Their job is to interpret the data into
238 text characters, and that's exactly what you don't want to happen
239 during binary I/O.
240 </p>
242 functions still aren't guaranteed to help you. These are
244 (This may or may not be what you want, see below.)
245 </p>
246 <p>Notice how all the problems here are due to the inappropriate use
249 seemingly infinite number of solutions, and a few are listed here:
250 </p>
251 <ul>
254 types you're using." This is a Bad Thing, because while
255 the compiler would probably be just fine with it, other humans
256 are going to be confused. The overloaded bitshift operators
257 have a well-defined meaning (formatting), and this breaks it.
258 </li>
260 the file and copy the structure." Well, this is easy to
261 make work, and easy to break, and is pretty equivalent to
263 makes no use of the iostream library at all...
264 </li>
266 While not trivial for the beginner, this is the best of all
267 solutions. The streambuf/filebuf layer is the layer that is
268 responsible for actual I/O. If you want to use the C++
269 library for binary I/O, this is where you start.
270 </li>
271 </ul>
272 <p>How to go about using streambufs is a bit beyond the scope of this
273 document (at least for now), but while streambufs go a long way,
274 they still leave a couple of things up to you, the programmer.
275 As an example, byte ordering is completely between you and the
276 operating system, and you have to handle it yourself.
277 </p>
278 <p>Deriving a streambuf or filebuf
279 class from the standard ones, one that is specific to your data
280 types (or an abstraction thereof) is probably a good idea, and
281 lots of examples exist in journals and on Usenet. Using the
282 standard filebufs directly (either by declaring your own or by
284 is certainly feasible as well.
285 </p>
286 <p>One area that causes problems is trying to do bit-by-bit operations
287 with filebufs. C++ is no different from C in this respect: I/O
288 must be done at the byte level. If you're trying to read or write
289 a few bits at a time, you're going about it the wrong way. You
290 must read/write an integral number of bytes and then process the
291 bytes. (For example, the streambuf functions take and return
293 </p>
294 <p>Another area of problems is opening text files in binary mode.
295 Generally, binary mode is intended for binary files, and opening
296 text files in binary mode means that you now have to deal with all of
297 those end-of-line and end-of-file problems that we mentioned before.
298 An instructive thread from comp.lang.c++.moderated delved off into
299 this topic starting more or less at
301 article and continuing to the end of the thread. (You'll have to
302 sort through some flames every couple of paragraphs, but the points
303 made are good ones.)
304 </p>
306 <hr />
309 are in this author's opinion one of the coolest things since
310 sliced time. An example of their use is in the Received Wisdom
311 section for Chapter 21 (Strings),
313 format strings</a>.
314 </p>
315 <p>The quick definition is: they are siblings of ifstream and ofstream,
321 ostringstream. You've read a string as input and need to parse it?
322 Initialize an istringstream with that string, and then pull pieces
325 member function.
326 </p>
327 <p>This only works if you've written your
329 and correctly means that they take istreams and ostreams as
331 take the latter, then your I/O operators will work fine with
332 file streams, but with nothing else -- including stringstreams.
333 </p>
334 <p>If you are a user of the strstream classes, you need to update
336 terminate the C-style character array, you don't have to mess with
338 memory yourself. The strstreams have been officially deprecated,
339 which means that 1) future revisions of the C++ Standard won't
340 support them, and 2) if you use them, people will laugh at you.
341 </p>
343 <hr />
345 <p>Creating your own stream buffers for I/O can be remarkably easy.
346 If you are interested in doing so, we highly recommend two very
347 excellent books:
352 Addison-Wesley, who isn't paying us a cent for saying that, honest.
353 </p>
354 <p>Here is a simple example, io/outbuf1, from the Josuttis text. It
355 transforms everything sent through it to uppercase. This version
356 assumes many things about the nature of the character type being
357 used (for more information, read the books or the newsgroups):
358 </p>
359 <pre>
365 class outbuf : public std::streambuf
366 {
367 protected:
368 /* central output function
369 * - print characters in uppercase mode
370 */
371 virtual int_type overflow (int_type c) {
372 if (c != EOF) {
373 // convert lowercase to uppercase
376 // and write the character to the standard output
377 if (putchar(c) == EOF) {
378 return EOF;
379 }
380 }
381 return c;
382 }
383 };
385 int main()
386 {
387 // create special output buffer
388 outbuf ob;
389 // initialize output stream with that output buffer
390 std::ostream out(&ob);
394 return 0;
395 }
396 </pre>
401 </p>
403 <hr />
407 same time. One notable place was Usenet, where James Kanze and
408 Dietmar Kühl separately posted articles on why attempting
409 generic binary I/O was not a good idea. (Here are copies of
412 </p>
413 <p>Briefly, the problems of byte ordering and type sizes mean that
416 between arbitrary programs, or across a network, or from one
417 invocation of a program to another invocation of the same program
418 on a different platform, etc.
419 </p>
420 <p>The entire Usenet thread is instructive, and took place under the
422 and comp.lang.c++.moderated in parallel. Also in that thread,
423 Dietmar Kühl mentioned that he had written a pair of stream
424 classes that would read and write XDR, which is a good step towards
425 a portable binary format.
426 </p>
428 <hr />
430 <p>It sounds like a flame on C, but it isn't. Really. Calm down.
431 I'm just saying it to get your attention.
432 </p>
433 <p>Because the C++ library includes the C library, both C-style and
434 C++-style I/O have to work at the same time. For example:
435 </p>
436 <pre>
443 </pre>
444 <p>This must do what you think it does.
445 </p>
446 <p>Alert members of the audience will immediately notice that buffering
447 is going to make a hash of the output unless special steps are taken.
448 </p>
450 involve doing very little buffering for the standard streams, leaving
451 most of the buffering to the underlying C library. (This kind of
453 The upside is that correctness is ensured. The downside is that
455 performance when the C++ I/O library is layered on top of the C I/O
456 library (as it is for 3.0 by default). Some patches have been applied
457 which improve the situation for 3.1.
458 </p>
459 <p>However, the C and C++ standard streams only need to be kept in sync
460 when both libraries' facilities are in use. If your program only uses
461 C++ I/O, then there's no need to sync with the C streams. The right
462 thing to do in this case is to call
463 </p>
464 <pre>
467 std::ios::sync_with_stdio(false);
468 </pre>
469 <p>You must do this before performing any I/O via the C++ stream objects.
470 Once you call this, the C++ streams will operate independently of the
472 company will become fully buffered on their own.
473 </p>
474 <p>Note, by the way, that the synchronization requirement only applies to
478 objects that you declare yourself have no such requirement and are fully
479 buffered.
480 </p>
482 <hr />
484 <p>I'll assume that you have already read the
486 and the
488 access</a> (you might not think of an I/O stream as a container, but
489 the points made there also hold here). If you have not read them,
490 please do so first.
491 </p>
492 <p>This gets a bit tricky. Please read carefully, and bear with me.
493 </p>
499 </p>
500 <p>A generic locking mechanism is somewhat in place at the filebuf layer,
501 but is not used in the current code. Providing locking at any higher
502 level is akin to providing locking within containers, and is not done
503 for the same reasons (see the links above).
504 </p>
506 <p>The __basic_file type is simply a collection of small wrappers around
507 the C stdio layer (again, see the link under Structure). We do no
510 </p>
512 must be answered with, "is your platform's C library threadsafe
513 for I/O?" Some are by default, some are not; many offer multiple
514 implementations of the C library with varying tradeoffs of threadsafety
515 and efficiency. You, the programmer, are always required to take care
516 with multiple threads.
517 </p>
518 <p>(As an example, the POSIX standard requires that C stdio FILE*
519 operations are atomic. POSIX-conforming C libraries (e.g, on Solaris
520 and GNU/Linux) have an internal mutex to serialize operations on
521 FILE*s. However, you still need to not do stupid things like calling
524 </p>
525 <p>So, if your platform's C library is threadsafe, then your
527 level. For higher-level operations, such as manipulating the data
528 contained in the stream formatting classes (e.g., setting up callbacks
530 like any other critical shared resource.
531 </p>
534 second choice is available for I/O implementations: libio. This is
535 disabled by default, and in fact will not currently work due to other
536 issues. It will be revisited, however.
537 </p>
538 <p>The libio code is a subset of the guts of the GNU libc (glibc) I/O
540 type is basically derived from FILE. (The real situation is more
541 complex than that... it's derived from an internal type used to
542 implement FILE. See libio/libioP.h to see scary things done with
544 to go through; the filebuf makes calls directly into the same
546 and so forth, using internal data structures. (And when I say
548 replaced by a jump into an internal function. Fast but frightening.
549 *grin*)
550 </p>
551 <p>Also, the libio internal locks are used. This requires pulling in
552 large chunks of glibc, such as a pthreads implementation, and is one
553 of the issues preventing widespread use of libio as the libstdc++
554 cstdio implementation.
555 </p>
556 <p>But we plan to make this work, at least as an option if not a future
557 default. Platforms running a copy of glibc with a recent-enough
558 version will see calls from libstdc++ directly into the glibc already
559 installed. For other platforms, a copy of the libio subsection will
560 be built and included in libstdc++.
561 </p>
563 <p>Don't forget that other cstdio implemenations are possible. You could
564 easily write one to perform your own forms of locking, to solve your
566 </p>
568 <hr />
570 <p>To minimize the time you have to wait on the compiler, it's good to
571 only include the headers you really need. Many people simply include
573 your runtime as well.</em> Here are some tips on which header to use
574 for which situations, starting with the simplest.
575 </p>
579 implies, these are forward declarations. (A word to all you fellow
580 old school programmers: trying to forward declare classes like
582 you'd like to know why.) For example,
583 </p>
584 <pre>
587 class MyClass
588 {
589 ....
590 std::ifstream input_file;
591 };
594 </pre>
597 counting types std::streamoff and std::streamsize, the file
598 positioning type std::fpos, and the various manipulators like
599 std::hex, std::fixed, std::noshowbase, and so forth.
600 </p>
601 <p>The ios_base class is what holds the format flags, the state flags,
602 and the functions which change them (setf(), width(), precision(),
603 etc). You can also store extra data and register callback functions
604 through ios_base, but that has been historically underused. Anything
605 which doesn't depend on the type of characters stored is consolidated
606 here.
607 </p>
608 <p>The template class basic_ios is the highest template class in the
609 hierarchy; it is the first one depending on the character type, and
610 holds all general state associated with that type: the pointer to the
611 polymorphic stream buffer, the facet information, etc.
612 </p>
614 basic_streambuf, and two standard instantiations, streambuf and
615 wstreambuf. If you need to work with the vastly useful and capable
616 stream buffer classes, e.g., to create a new form of storage
617 transport, this header is the one to include.
618 </p>
621 interface, or any of the other abstract stream formatting functions.
622 For example,
623 </p>
624 <pre>
628 {
630 }
631 </pre>
632 <p>The std::istream and std::ostream classes are the abstract parents of
633 the various concrete implementations. If you are only using the
634 interfaces, then you only need to use the appropriate interface header.
635 </p>
637 that alter information maintained by class ios_base and its dervied
638 classes," such as std::setprecision and std::setw. If you need
641 </p>
643 declare the six stringstream and fstream classes. As they are the
644 standard concrete descendants of istream and ostream, you will already
645 know about them.
646 </p>
648 global objects (cin, cout, etc). To do this correctly, this header
650 headers, but nothing else. The contents of this header look like
651 </p>
652 <pre>
656 namespace std
657 {
658 extern istream cin;
659 extern ostream cout;
660 ....
662 // this is explained below
664 }
665 </pre>
666 <p>Now, the runtime penalty mentioned previously: the global objects
667 must be initialized before any of your own code uses them; this is
668 guaranteed by the standard. Like any other global object, they must
669 be initialized once and only once. This is typically done with a
670 construct like the one above, and the nested class ios_base::Init is
671 specified in the standard for just this reason.
672 </p>
673 <p>How does it work? Because the header is included before any of your
675 your objects. (Global objects are built in the order in which they
676 are declared, and destroyed in reverse order.) The first time the
677 constructor runs, the eight stream objects are set up.
678 </p>
682 of construction across object files (it's one of those pesky NP
683 problems that make life so interesting), so one copy in each object
684 file means that the stream objects are guaranteed to be set up before
685 any of your code which uses them could run, thereby meeting the
686 requirements of the standard.
687 </p>
688 <p>The penalty, of course, is that after the first copy of
690 processor time. The time spent is merely for an increment-and-test
691 inside a function call, but over several dozen or hundreds of object
692 files, that time can add up. (It's not in a tight loop, either.)
693 </p>
695 the standard objects in that source file; you'll pay less startup
696 time. Only include the header files you need to in general; your
697 compile times will go down when there's less parsing work to do.
698 </p>
701 <hr />
703 <!-- referenced by ext/howto.html#2, update link if numbering changes -->
704 <p>The v2 library included non-standard extensions to construct
707 Today the recommended way to use stdio types with libstdc++-v3
709 but earlier releases provided slightly different mechanisms.
710 </p>
711 <ul>
713 <br />
715 <br />This comes in very handy in a number of places, such as
716 attaching Unix sockets, pipes, and anything else which uses file
717 descriptors, into the IOStream buffering classes. The three
718 arguments are as follows:
719 <ul>
721 // the __c_file_type typedef usually boils down to stdio's FILE
722 </li>
724 // same as all the other uses of openmode
725 </li>
727 // buffer size, defaults to BUFSIZ if not specified
728 </li>
729 </ul>
730 For those wanting to use file descriptors instead of FILE*'s, I
732 </li>
735 integer returned from this function can be used for whatever file
736 descriptors can be used for on your platform. Naturally, the
737 library cannot track what you do on your own with a file descriptor,
738 so if you perform any I/O directly, don't expect the library to be
739 aware of it.
740 </li>
744 a derived class called
745 <a href="http://gcc.gnu.org/onlinedocs/libstdc++/latest-doxygen/class____gnu__cxx_1_1stdio__filebuf.html"><code>__gnu_cxx::stdio_filebuf</code></a>.
748 </li>
749 </ul>
752 call) then you might be interested in Henry Suter's
754 class.
755 </p>
757 <!-- ####################################################### -->
759 <hr />
762 Comments and suggestions are welcome, and may be sent to
764 </em></p>
767 </body>
768 </html>