doc/src/sgml/ref/pg_receivewal.sgml

   1 <!--
   2 doc/src/sgml/ref/pg_receivewal.sgml
   3 PostgreSQL documentation
   4 -->
   5
   6 <refentry id="app-pgreceivewal">
   7  <indexterm zone="app-pgreceivewal">
   8   <primary>pg_receivewal</primary>
   9  </indexterm>
  10
  11  <refmeta>
  12   <refentrytitle><application>pg_receivewal</application></refentrytitle>
  13   <manvolnum>1</manvolnum>
  14   <refmiscinfo>Application</refmiscinfo>
  15  </refmeta>
  16
  17  <refnamediv>
  18   <refname>pg_receivewal</refname>
  19   <refpurpose>stream write-ahead logs from a <productname>PostgreSQL</productname> server</refpurpose>
  20  </refnamediv>
  21
  22  <refsynopsisdiv>
  23   <cmdsynopsis>
  24    <command>pg_receivewal</command>
  25    <arg rep="repeat"><replaceable>option</replaceable></arg>
  26   </cmdsynopsis>
  27  </refsynopsisdiv>
  28
  29  <refsect1>
  30   <title>Description</title>
  31   <para>
  32    <application>pg_receivewal</application> is used to stream the write-ahead log
  33    from a running <productname>PostgreSQL</productname> cluster. The write-ahead
  34    log is streamed using the streaming replication protocol, and is written
  35    to a local directory of files. This directory can be used as the archive
  36    location for doing a restore using point-in-time recovery (see
  37    <xref linkend="continuous-archiving"/>).
  38   </para>
  39
  40   <para>
  41    <application>pg_receivewal</application> streams the write-ahead
  42    log in real time as it's being generated on the server, and does not wait
  43    for segments to complete like <xref linkend="guc-archive-command"/> and
  44    <xref linkend="guc-archive-library"/> do.
  45    For this reason, it is not necessary to set
  46    <xref linkend="guc-archive-timeout"/> when using
  47     <application>pg_receivewal</application>.
  48   </para>
  49
  50   <para>
  51    Unlike the WAL receiver of a PostgreSQL standby server, <application>pg_receivewal</application>
  52    by default flushes WAL data only when a WAL file is closed.
  53    The option <option>--synchronous</option> must be specified to flush WAL data
  54    in real time. Since <application>pg_receivewal</application> does not
  55    apply WAL, you should not allow it to become a synchronous standby when
  56    <xref linkend="guc-synchronous-commit"/> equals
  57    <literal>remote_apply</literal>. If it does, it will appear to be a
  58    standby that never catches up, and will cause transaction commits to
  59    block. To avoid this, you should either configure an appropriate value
  60    for <xref linkend="guc-synchronous-standby-names"/>, or specify
  61    <varname>application_name</varname> for
  62    <application>pg_receivewal</application> that does not match it, or
  63    change the value of <varname>synchronous_commit</varname> to
  64    something other than <literal>remote_apply</literal>.
  65   </para>
  66
  67   <para>
  68    The write-ahead log is streamed over a regular
  69    <productname>PostgreSQL</productname> connection and uses the replication
  70    protocol. The connection must be made with a user having
  71    <literal>REPLICATION</literal> permissions (see
  72    <xref linkend="role-attributes"/>) or a superuser, and
  73    <filename>pg_hba.conf</filename> must permit the replication connection.
  74    The server must also be configured with
  75    <xref linkend="guc-max-wal-senders"/> set high enough to leave at least
  76    one session available for the stream.
  77   </para>
  78
  79   <para>
  80    The starting point of the write-ahead log streaming is calculated when
  81    <application>pg_receivewal</application> starts:
  82    <orderedlist>
  83     <listitem>
  84      <para>
  85       First, scan the directory where the WAL segment files are written and
  86       find the newest completed segment file, using as the starting point the
  87       beginning of the next WAL segment file.
  88      </para>
  89     </listitem>
  90
  91     <listitem>
  92      <para>
  93       If a starting point cannot be calculated with the previous method,
  94       and if a replication slot is used, an extra
  95       <command>READ_REPLICATION_SLOT</command> command is issued to retrieve
  96       the slot's <literal>restart_lsn</literal> to use as the starting point.
  97       This option is only available when streaming write-ahead logs from
  98       <productname>PostgreSQL</productname> 15 and up.
  99      </para>
 100     </listitem>
 101
 102     <listitem>
 103      <para>
 104       If a starting point cannot be calculated with the previous method,
 105       the latest WAL flush location is used as reported by the server from
 106       an <literal>IDENTIFY_SYSTEM</literal> command.
 107      </para>
 108     </listitem>
 109    </orderedlist>
 110   </para>
 111
 112   <para>
 113    If the connection is lost, or if it cannot be initially established,
 114    with a non-fatal error, <application>pg_receivewal</application> will
 115    retry the connection indefinitely, and reestablish streaming as soon
 116    as possible. To avoid this behavior, use the <literal>-n</literal>
 117    parameter.
 118   </para>
 119
 120   <para>
 121    In the absence of fatal errors, <application>pg_receivewal</application>
 122    will run until terminated by the <systemitem>SIGINT</systemitem>
 123    (<keycombo action="simul"><keycap>Control</keycap><keycap>C</keycap></keycombo>)
 124    or <systemitem>SIGTERM</systemitem> signal.
 125   </para>
 126  </refsect1>
 127
 128  <refsect1>
 129   <title>Options</title>
 130
 131     <variablelist>
 132      <varlistentry>
 133       <term><option>-D <replaceable class="parameter">directory</replaceable></option></term>
 134       <term><option>--directory=<replaceable class="parameter">directory</replaceable></option></term>
 135       <listitem>
 136        <para>
 137         Directory to write the output to.
 138        </para>
 139        <para>
 140         This parameter is required.
 141        </para>
 142       </listitem>
 143      </varlistentry>
 144
 145      <varlistentry>
 146       <term><option>-E <replaceable>lsn</replaceable></option></term>
 147       <term><option>--endpos=<replaceable>lsn</replaceable></option></term>
 148       <listitem>
 149        <para>
 150         Automatically stop replication and exit with normal exit status 0 when
 151         receiving reaches the specified LSN.
 152        </para>
 153
 154        <para>
 155         If there is a record with LSN exactly equal to <replaceable>lsn</replaceable>,
 156         the record will be processed.
 157        </para>
 158       </listitem>
 159      </varlistentry>
 160
 161      <varlistentry>
 162       <term><option>--if-not-exists</option></term>
 163       <listitem>
 164        <para>
 165         Do not error out when <option>--create-slot</option> is specified
 166         and a slot with the specified name already exists.
 167        </para>
 168       </listitem>
 169      </varlistentry>
 170
 171      <varlistentry>
 172       <term><option>-n</option></term>
 173       <term><option>--no-loop</option></term>
 174       <listitem>
 175        <para>
 176         Don't loop on connection errors. Instead, exit right away with
 177         an error.
 178        </para>
 179       </listitem>
 180      </varlistentry>
 181
 182      <varlistentry>
 183       <term><option>--no-sync</option></term>
 184       <listitem>
 185        <para>
 186         This option causes <command>pg_receivewal</command> to not force WAL
 187         data to be flushed to disk.  This is faster, but means that a
 188         subsequent operating system crash can leave the WAL segments corrupt.
 189         Generally, this option is useful for testing but should not be used
 190         when doing WAL archiving on a production deployment.
 191        </para>
 192
 193        <para>
 194         This option is incompatible with <literal>--synchronous</literal>.
 195        </para>
 196       </listitem>
 197      </varlistentry>
 198
 199      <varlistentry>
 200       <term><option>-s <replaceable class="parameter">interval</replaceable></option></term>
 201       <term><option>--status-interval=<replaceable class="parameter">interval</replaceable></option></term>
 202       <listitem>
 203        <para>
 204         Specifies the number of seconds between status packets sent back to the
 205         server. This allows for easier monitoring of the progress from server.
 206         A value of zero disables the periodic status updates completely,
 207         although an update will still be sent when requested by the server, to
 208         avoid timeout disconnect. The default value is 10 seconds.
 209        </para>
 210       </listitem>
 211      </varlistentry>
 212
 213      <varlistentry>
 214       <term><option>-S <replaceable>slotname</replaceable></option></term>
 215       <term><option>--slot=<replaceable class="parameter">slotname</replaceable></option></term>
 216       <listitem>
 217         <para>
 218          Require <application>pg_receivewal</application> to use an existing
 219          replication slot (see <xref linkend="streaming-replication-slots"/>).
 220          When this option is used, <application>pg_receivewal</application> will report
 221          a flush position to the server, indicating when each segment has been
 222          synchronized to disk so that the server can remove that segment if it
 223          is not otherwise needed.
 224         </para>
 225
 226         <para>
 227          When the replication client
 228          of <application>pg_receivewal</application> is configured on the
 229          server as a synchronous standby, then using a replication slot will
 230          report the flush position to the server, but only when a WAL file is
 231          closed.  Therefore, that configuration will cause transactions on the
 232          primary to wait for a long time and effectively not work
 233          satisfactorily.  The option <literal>--synchronous</literal> (see
 234          below) must be specified in addition to make this work correctly.
 235         </para>
 236       </listitem>
 237      </varlistentry>
 238
 239      <varlistentry>
 240       <term><option>--synchronous</option></term>
 241       <listitem>
 242        <para>
 243         Flush the WAL data to disk immediately after it has been received. Also
 244         send a status packet back to the server immediately after flushing,
 245         regardless of <literal>--status-interval</literal>.
 246        </para>
 247
 248        <para>
 249         This option should be specified if the replication client
 250         of <application>pg_receivewal</application> is configured on the
 251         server as a synchronous standby, to ensure that timely feedback is
 252         sent to the server.
 253        </para>
 254       </listitem>
 255      </varlistentry>
 256
 257      <varlistentry>
 258       <term><option>-v</option></term>
 259       <term><option>--verbose</option></term>
 260       <listitem>
 261        <para>
 262         Enables verbose mode.
 263        </para>
 264       </listitem>
 265      </varlistentry>
 266
 267      <varlistentry>
 268       <term><option>-Z <replaceable class="parameter">level</replaceable></option></term>
 269       <term><option>-Z <replaceable class="parameter">method</replaceable>[:<replaceable>detail</replaceable>]</option></term>
 270       <term><option>--compress=<replaceable class="parameter">level</replaceable></option></term>
 271       <term><option>--compress=<replaceable class="parameter">method</replaceable>[:<replaceable>detail</replaceable>]</option></term>
 272       <listitem>
 273        <para>
 274         Enables compression of write-ahead logs.
 275        </para>
 276        <para>
 277         The compression method can be set to <literal>gzip</literal>,
 278         <literal>lz4</literal> (if <productname>PostgreSQL</productname>
 279         was compiled with <option>--with-lz4</option>) or
 280         <literal>none</literal> for no compression.
 281         A compression detail string can optionally be specified.  If the
 282         detail string is an integer, it specifies the compression level.
 283         Otherwise, it should be a comma-separated list of items, each of the
 284         form <replaceable>keyword</replaceable> or
 285         <replaceable>keyword=value</replaceable>.
 286         Currently, the only supported keyword is <literal>level</literal>.
 287        </para>
 288        <para>
 289         If no compression level is specified, the default compression level
 290         will be used. If only a level is specified without mentioning an
 291         algorithm, <literal>gzip</literal> compression will be used if the
 292         level is greater than 0, and no compression will be used if the level
 293         is 0.
 294        </para>
 295        <para>
 296         The suffix <filename>.gz</filename> will automatically be added to
 297         all filenames when using <literal>gzip</literal>, and the suffix
 298         <filename>.lz4</filename> is added when using <literal>lz4</literal>.
 299        </para>
 300       </listitem>
 301      </varlistentry>
 302     </variablelist>
 303
 304    <para>
 305     The following command-line options control the database connection parameters.
 306
 307     <variablelist>
 308      <varlistentry>
 309       <term><option>-d <replaceable class="parameter">connstr</replaceable></option></term>
 310       <term><option>--dbname=<replaceable class="parameter">connstr</replaceable></option></term>
 311       <listitem>
 312        <para>
 313         Specifies parameters used to connect to the server, as a <link
 314         linkend="libpq-connstring">connection string</link>;  these
 315         will override any conflicting command line options.
 316        </para>
 317        <para>
 318         The option is called <literal>--dbname</literal> for consistency with other
 319         client applications, but because <application>pg_receivewal</application>
 320         doesn't connect to any particular database in the cluster, any database
 321         name in the connection string will be ignored by
 322         <productname>PostgreSQL</productname>. Middleware, or proxies, used in
 323         connecting to <productname>PostgreSQL</productname> might however
 324         utilize the value.
 325        </para>
 326       </listitem>
 327      </varlistentry>
 328
 329      <varlistentry>
 330       <term><option>-h <replaceable class="parameter">host</replaceable></option></term>
 331       <term><option>--host=<replaceable class="parameter">host</replaceable></option></term>
 332       <listitem>
 333        <para>
 334         Specifies the host name of the machine on which the server is
 335         running.  If the value begins with a slash, it is used as the
 336         directory for the Unix domain socket. The default is taken
 337         from the <envar>PGHOST</envar> environment variable, if set,
 338         else a Unix domain socket connection is attempted.
 339        </para>
 340       </listitem>
 341      </varlistentry>
 342
 343      <varlistentry>
 344       <term><option>-p <replaceable class="parameter">port</replaceable></option></term>
 345       <term><option>--port=<replaceable class="parameter">port</replaceable></option></term>
 346       <listitem>
 347        <para>
 348         Specifies the TCP port or local Unix domain socket file
 349         extension on which the server is listening for connections.
 350         Defaults to the <envar>PGPORT</envar> environment variable, if
 351         set, or a compiled-in default.
 352        </para>
 353       </listitem>
 354      </varlistentry>
 355
 356      <varlistentry>
 357       <term><option>-U <replaceable>username</replaceable></option></term>
 358       <term><option>--username=<replaceable class="parameter">username</replaceable></option></term>
 359       <listitem>
 360        <para>
 361         User name to connect as.
 362        </para>
 363       </listitem>
 364      </varlistentry>
 365
 366      <varlistentry>
 367       <term><option>-w</option></term>
 368       <term><option>--no-password</option></term>
 369       <listitem>
 370        <para>
 371         Never issue a password prompt.  If the server requires
 372         password authentication and a password is not available by
 373         other means such as a <filename>.pgpass</filename> file, the
 374         connection attempt will fail.  This option can be useful in
 375         batch jobs and scripts where no user is present to enter a
 376         password.
 377        </para>
 378       </listitem>
 379      </varlistentry>
 380
 381      <varlistentry>
 382       <term><option>-W</option></term>
 383       <term><option>--password</option></term>
 384       <listitem>
 385        <para>
 386         Force <application>pg_receivewal</application> to prompt for a
 387         password before connecting to a database.
 388        </para>
 389
 390        <para>
 391         This option is never essential, since
 392         <application>pg_receivewal</application> will automatically prompt
 393         for a password if the server demands password authentication.
 394         However, <application>pg_receivewal</application> will waste a
 395         connection attempt finding out that the server wants a password.
 396         In some cases it is worth typing <option>-W</option> to avoid the extra
 397         connection attempt.
 398        </para>
 399       </listitem>
 400      </varlistentry>
 401     </variablelist>
 402    </para>
 403
 404    <para>
 405     <application>pg_receivewal</application> can perform one of the two
 406     following actions in order to control physical replication slots:
 407
 408     <variablelist>
 409      <varlistentry>
 410       <term><option>--create-slot</option></term>
 411       <listitem>
 412        <para>
 413         Create a new physical replication slot with the name specified in
 414         <option>--slot</option>, then exit.
 415        </para>
 416       </listitem>
 417      </varlistentry>
 418
 419      <varlistentry>
 420       <term><option>--drop-slot</option></term>
 421       <listitem>
 422        <para>
 423         Drop the replication slot with the name specified in
 424         <option>--slot</option>, then exit.
 425        </para>
 426       </listitem>
 427      </varlistentry>
 428     </variablelist>
 429    </para>
 430
 431    <para>
 432     Other options are also available:
 433
 434     <variablelist>
 435      <varlistentry>
 436        <term><option>-V</option></term>
 437        <term><option>--version</option></term>
 438        <listitem>
 439        <para>
 440        Print the <application>pg_receivewal</application> version and exit.
 441        </para>
 442        </listitem>
 443      </varlistentry>
 444
 445      <varlistentry>
 446        <term><option>-?</option></term>
 447        <term><option>--help</option></term>
 448        <listitem>
 449        <para>
 450        Show help about <application>pg_receivewal</application> command line
 451        arguments, and exit.
 452        </para>
 453        </listitem>
 454      </varlistentry>
 455
 456     </variablelist>
 457    </para>
 458
 459  </refsect1>
 460
 461  <refsect1>
 462   <title>Exit Status</title>
 463
 464   <para>
 465    <application>pg_receivewal</application> will exit with status 0 when
 466    terminated by the <systemitem>SIGINT</systemitem> or
 467    <systemitem>SIGTERM</systemitem> signal.  (That is the
 468    normal way to end it.  Hence it is not an error.)  For fatal errors or
 469    other signals, the exit status will be nonzero.
 470   </para>
 471
 472  </refsect1>
 473
 474  <refsect1>
 475   <title>Environment</title>
 476
 477   <para>
 478    This utility, like most other <productname>PostgreSQL</productname> utilities,
 479    uses the environment variables supported by <application>libpq</application>
 480    (see <xref linkend="libpq-envars"/>).
 481   </para>
 482
 483   <para>
 484    The environment variable <envar>PG_COLOR</envar> specifies whether to use
 485    color in diagnostic messages. Possible values are
 486    <literal>always</literal>, <literal>auto</literal> and
 487    <literal>never</literal>.
 488   </para>
 489  </refsect1>
 490
 491  <refsect1>
 492   <title>Notes</title>
 493
 494   <para>
 495    When using <application>pg_receivewal</application> instead of
 496    <xref linkend="guc-archive-command"/> or
 497    <xref linkend="guc-archive-library"/> as the main WAL backup method, it is
 498    strongly recommended to use replication slots.  Otherwise, the server is
 499    free to recycle or remove write-ahead log files before they are backed up,
 500    because it does not have any information, either
 501    from <xref linkend="guc-archive-command"/> or
 502    <xref linkend="guc-archive-library"/> or the replication slots, about
 503    how far the WAL stream has been archived.  Note, however, that a
 504    replication slot will fill up the server's disk space if the receiver does
 505    not keep up with fetching the WAL data.
 506   </para>
 507
 508   <para>
 509    <application>pg_receivewal</application> will preserve group permissions on
 510    the received WAL files if group permissions are enabled on the source
 511    cluster.
 512   </para>
 513
 514  </refsect1>
 515
 516  <refsect1>
 517   <title>Examples</title>
 518
 519   <para>
 520    To stream the write-ahead log from the server at
 521    <literal>mydbserver</literal> and store it in the local directory
 522    <filename>/usr/local/pgsql/archive</filename>:
 523 <screen>
 524 <prompt>$</prompt> <userinput>pg_receivewal -h mydbserver -D /usr/local/pgsql/archive</userinput>
 525 </screen></para>
 526  </refsect1>
 527
 528  <refsect1>
 529   <title>See Also</title>
 530
 531   <simplelist type="inline">
 532    <member><xref linkend="app-pgbasebackup"/></member>
 533   </simplelist>
 534  </refsect1>
 535
 536 </refentry>